Computer Vision
StAR: Segment Anything Reasoner
Seokju Yun, Dongheon Lee, Noori Bae et al. (+3 more)
As AI systems are being integrated more rapidly into diverse and complex real-world environments, the ability to perform holistic
reasoning over an implicit query and an image to localize a target is becoming increasingly important. However, recent
reasoning segmentation methods fail to sufficiently elicit the visual
reasoning capabilities of the base mode. In this work, we present Segment Anything Reasoner (
StAR), a comprehensive framework that refines the design space from multiple perspectives-including parameter-tuning scheme, reward functions, learning strategies and answer format-and achieves substantial improvements over recent baselines. In addition, for the first time, we successfully introduce parallel test-time scaling to the
segmentation task, pushing the performance boundary even further. To extend the scope and depth of
reasoning covered by existing
benchmark, we also construct the ReasonSeg-X, which compactly defines
reasoning types and includes samples that require deeper
reasoning. Leveraging this
dataset, we train
StAR with a rollout-expanded selective-tuning approach to activate the base model's latent
reasoning capabilities, and establish a rigorous
benchmark for systematic, fine-grained evaluation of advanced methods. With only 5k training samples,
StAR achieves significant gains over its base counterparts across extensive
benchmarks, demonstrating that our method effectively brings dormant
reasoning competence to the surface.
Read the full abstract ▾
All authors Seokju Yun, Dongheon Lee, Noori Bae, Jaesung Jun, Chanseul Cho, Youngmin Ro
Computer Vision
LoCAtion: Long-time Collaborative Attention Framework for High Dynamic Range Video Reconstruction
Qianyu Zhang, Bolun Zheng, Lingyu Zhu et al. (+3 more)
Prevailing High Dynamic Range (HDR)
video reconstruction methods are fundamentally trapped in a fragile
alignment-and-fusion paradigm. While explicit spatial
alignment can successfully recover fine details in controlled environments, it becomes a severe bottleneck in unconstrained dynamic scenes. By forcing rigid
alignment across unpredictable motions and varying exposures, these methods inevitably translate registration errors into severe ghosting artifacts and temporal flickering. In this paper, we rethink this conventional prerequisite. Recognizing that explicit
alignment is inherently vulnerable to real-world complexities, we propose LoCAtion, a Long-time Collaborative
Attention framework that reformulates HDR
video generation from a fragile spatial warping task into a robust,
alignment-free collaborative feature routing problem. Guided by this new formulation, our architecture explicitly decouples the highly entangled reconstruction task. Rather than struggling to rigidly warp neighboring frames, we anchor the scene on a continuous medium-exposure backbone and utilize collaborative
attention to dynamically harvest and inject reliable irradiance cues from unaligned exposures. Furthermore, we introduce a learned global sequence solver. By leveraging bidirectional context and long-range temporal modeling, it propagates corrective signals and structural features across the entire sequence, inherently enforcing whole-
video coherence and eliminating jitter. Extensive experiments demonstrate that LoCAtion achieves state-of-the-art visual quality and temporal stability, offering a highly competitive balance between accuracy and computational efficiency.
Read the full abstract ▾
All authors Qianyu Zhang, Bolun Zheng, Lingyu Zhu, Aiai Huang, Zongpeng Li, Shiqi Wang
Computer Vision
The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics
Xiangbo Gao, Mingyang Wu, Siyuan Yang et al. (+4 more)
While recent generative
video models have achieved remarkable visual realism and are being explored as world models, true physical simulation requires mastering both space and time. Current models can produce visually smooth kinematics, yet they lack a reliable internal motion pulse to ground these motions in a consistent, real-world time scale. This temporal ambiguity stems from the common practice of indiscriminately training on
videos with vastly different real-world speeds, forcing them into standardized frame rates. This leads to what we term chronometric
hallucination: generated sequences exhibit ambiguous, unstable, and uncontrollable physical motion speeds. To address this, we propose Visual Chronometer, a predictor that recovers the Physical Frames Per Second (PhyFPS) directly from the visual dynamics of an input
video. Trained via controlled temporal resampling, our method estimates the true temporal scale implied by the motion itself, bypassing unreliable metadata. To systematically quantify this issue, we establish two
benchmarks, PhyFPS-Bench-Real and PhyFPS-Bench-Gen. Our evaluations reveal a harsh reality: state-of-the-art
video generators suffer from severe PhyFPS misalignment and temporal instability. Finally, we demonstrate that applying PhyFPS corrections significantly improves the human-perceived naturalness of AI-generated
videos. Our project page is https://xiangbogaobarry.github.io/Visual_Chronometer/.
Read the full abstract ▾
All authors Xiangbo Gao, Mingyang Wu, Siyuan Yang, Jiongze Yu, Pardis Taghavi, Fangzhou Lin, Zhengzhong Tu
Computer Vision
HomeGuard: VLM-based Embodied Safeguard for Identifying Contextual Risk in Household Task
Xiaoya Lu, Yijin Zhou, Zeren Chen et al. (+6 more)
Vision-
Language Models (VLMs) empower
embodied agents to execute complex instructions, yet they remain vulnerable to contextual safety risks where benign commands become hazardous due to subtle environmental states. Existing safeguards often prove inadequate. Rule-based methods lack scalability in object-dense scenes, whereas model-based approaches relying on
prompt engineering suffer from unfocused perception, resulting in missed risks or
hallucinations. To address this, we propose an architecture-agnostic safeguard featuring Context-Guided Chain-of-Thought (CG-CoT). This mechanism decomposes risk assessment into active perception that sequentially anchors
attention to interaction targets and relevant spatial neighborhoods, followed by semantic judgment based on this visual evidence. We support this approach with a curated grounding
dataset and a two-stage training strategy utilizing Reinforcement Fine-Tuning (RFT) with process rewards to enforce precise intermediate grounding. Experiments demonstrate that our model HomeGuard significantly enhances safety, improving risk match rates by over 30% compared to base models while reducing oversafety. Beyond hazard detection, the generated visual anchors serve as actionable spatial constraints for downstream planners, facilitating explicit collision avoidance and safety trajectory generation. Code and data are released under https://github.com/AI45Lab/HomeGuard
Read the full abstract ▾
All authors Xiaoya Lu, Yijin Zhou, Zeren Chen, Ruocheng Wang, Bingrui Sima, Enshen Zhou, Lu Sheng, Dongrui Liu, Jing Shao
Computer Vision
Representation Alignment for Just Image Transformers is not Easier than You Think
Jaeyo Shin, Jiwook Kim and Hyunjung Shim
Representation
Alignment (REPA) has emerged as a simple way to accelerate
Diffusion Transformers training in latent space. At the same time, pixel-space
diffusion transformers such as Just image
Transformers (JiT) have attracted growing
attention because they remove a dependency on a pretrained tokenizer, and then avoid the reconstruction bottleneck of latent
diffusion. This paper shows that the REPA can fail for JiT. REPA yields worse FID for JiT as training proceeds and collapses diversity on image subsets that are tightly clustered in the representation space of pretrained semantic encoder on ImageNet. We trace the failure to an information asymmetry: denoising occurs in the high dimensional image space, while the semantic target is strongly compressed, making direct regression a shortcut objective. We propose PixelREPA, which transforms the
alignment target and constrains
alignment with a Masked
Transformer Adapter that combines a shallow
transformer adapter with partial
token masking. PixelREPA improves both training convergence and final quality. PixelREPA reduces FID from 3.66 to 3.17 for JiT-B$/16$ and improves Inception Score (IS) from 275.1 to 284.6 on ImageNet $256 \times 256$, while achieving $> 2\times$ faster convergence. Finally, PixelREPA-H$/16$ achieves FID$=1.81$ and IS$=317.2$. Our code is available at https://github.com/kaist-cvml/PixelREPA.
Read the full abstract ▾
Computer Vision
AerialVLA: A Vision-Language-Action Model for UAV Navigation via Minimalist End-to-End Control
Peng Xu, Zhengnan Deng, Jiayan Deng et al. (+2 more)
Vision-
Language Navigation (VLN) for Unmanned Aerial Vehicles (
UAVs) demands complex visual interpretation and continuous control in dynamic 3D environments. Existing hierarchical approaches rely on dense oracle guidance or auxiliary object detectors, creating semantic gaps and limiting genuine autonomy. We propose AerialVLA, a minimalist end-to-end
Vision-
Language-Action framework mapping raw visual observations and fuzzy linguistic instructions directly to continuous physical control signals. First, we introduce a streamlined dual-view perception strategy that reduces visual redundancy while preserving essential cues for forward
navigation and precise grounding, which additionally facilitates future simulation-to-reality transfer. To reclaim genuine autonomy, we deploy a fuzzy directional prompting mechanism derived solely from onboard sensors, completely eliminating the dependency on dense oracle guidance. Ultimately, we formulate a unified control space that integrates continuous 3-Degree-of-Freedom (3-DoF) kinematic commands with an intrinsic landing signal, freeing the
agent from external object detectors for precision landing. Extensive experiments on the TravelUAV
benchmark demonstrate that AerialVLA achieves state-of-the-art performance in seen environments. Furthermore, it exhibits superior generalization in unseen scenarios by achieving nearly three times the success rate of leading baselines, validating that a minimalist, autonomy-centric paradigm captures more robust visual-motor representations than complex modular systems.
Read the full abstract ▾
All authors Peng Xu, Zhengnan Deng, Jiayan Deng, Zonghua Gu, Shaohua Wan
Computer Vision
BROTHER: Behavioral Recognition Optimized Through Heterogeneous Ensemble Regularization for Ambivalence and Hesitancy
Alexandre Pereira, Bruno Fernandes and Pablo Barros
Recognizing complex behavioral states such as Ambivalence and Hesitancy (A/H) in naturalistic
video settings remains a significant challenge in affective computing. Unlike basic facial expressions, A/H manifests as subtle,
multimodal conflicts that require deep contextual and temporal understanding. In this paper, we propose a highly regularized,
multimodal fusion pipeline to predict A/H at the
video level. We extract robust unimodal features from visual, acoustic, and linguistic data, introducing a specialized statistical text modality explicitly designed to capture temporal
speech variations and behavioral cues. To identify the most effective representations, we evaluate 15 distinct modality combinations across a committee of machine learning classifiers (MLP, Random Forest, and GBDT), selecting the most well-calibrated models based on validation Binary Cross-
Entropy (BCE) loss. Furthermore, to optimally fuse these heterogeneous models without overfitting to the training distribution, we implement a Particle Swarm Optimization (PSO) hard-voting ensemble. The PSO fitness function dynamically incorporates a train-validation gap penalty (lambda) to actively suppress redundant or overfitted classifiers. Our comprehensive evaluation demonstrates that while linguistic features serve as the strongest independent predictor of A/H, our heavily regularized PSO ensemble (lambda = 0.2) effectively harnesses
multimodal synergies, achieving a peak Macro F1-score of 0.7465 on the unseen test set. These results emphasize that treating ambivalence and hesitancy as a
multimodal conflict, evaluated through an intelligently weighted committee, provides a robust framework for in-the-wild behavioral analysis.
Read the full abstract ▾
Computer Vision
AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning
Jiarui Zhang, Junqi Hu, Zurong Mai et al. (+10 more)
Modern agricultural data is sourced from diverse platforms and spans multiple spatial scales, ranging from ground-level close-up photography to Unmanned Aerial Vehicle (
UAV) aerial observation and satellite remote sensing imagery. Accordingly, agricultural
multimodal reasoning demands robust cross-scale spatial understanding. However, due to the lack of multi-view agricultural
benchmark datasets, existing
multimodal large
language models (MLLMs) exhibit severe ground-level bias, which leads to scale confusion then semantic collapse in agricultural perception tasks, such as misinterpreting farmland imagery as walls or floors. To address this, we introduce AgroOmni, a large-scale multi-view training corpus with 288K Visual Question Answering pairs covering 56 specialized task categories across 14 task types, designed to capture diverse scales in modern precision agriculture. Built on this
dataset, we propose AgroNVILA, which achieves a new state-of-the-art of 62.32% on the AgroMind
benchmark (+15.03% over GPT-5.2), effectively mitigating the multi-view cross-scale gap for holistic agricultural understanding. Diagnostic evaluations on AgMMU further reveal an inherent heterogeneity between macro-priors and micro-diagnostics through constrained zero-shot performance. Meanwhile, even minimal fine-tuning leads to a dramatic performance gain of AgroNVILA on AgMMU, strongly demonstrating its generalization capability empowered by AgroOmni. Full training scripts are publicly available at https://anonymous.4open.science/r/AgroOmni-6510.
Read the full abstract ▾
All authors Jiarui Zhang, Junqi Hu, Zurong Mai, Yang Liu, Yuhang Chen, Shuohong Lou, Henglian Huang, Hong Cheng, Lingyuan Zhao, Jianxi Huang, Yutong Lu, Haohuan Fu, Juepeng Zheng
Computer Vision
On the Nature of Attention Sink that Shapes Decoding Strategy in Omni-LLMs
Suho Yoo, Youngjoon Jang and Joon Son Chung
The goal of this paper is to strengthen the
reasoning of Omnimodal Large
Language Models (Omni-LLMs) at inference time, without additional training. These models jointly process
video, audio, and text, and given the large number of
tokens they consume, how
attention is routed across them is central to their behaviour. We focus specifically on
attention sinks,
tokens that absorb a disproportionate share of
attention mass regardless of their semantic content, to understand how this routing unfolds. To this end, we conduct a systematic analysis of sink behaviour in Omni-LLMs. Our analysis yields two key findings: (i) high sink
attention does not solely indicate head redundancy, suggesting that sink value representations play additional functional roles; (ii) the sink value vector acts as a shared bias added to every
token's output, serving as a global signal that organises the representation as a whole. Building on this, we propose OutRo, which correspondingly aligns non-sink
token representations with the sink in feature space, and relaxes the causal mask for sink
tokens at an early layer to sharpen this bias before the rest of decoding proceeds. This design enhances the
reasoning process without requiring additional forward passes or access to
attention maps. Based on extensive experiments, OutRo consistently improves performance on seven
video QA
benchmarks and demonstrates strong generalisation, while incurring only a 1.1x decoding overhead.
Read the full abstract ▾
Computer Vision
UAVBench and UAVIT-1M: Benchmarking and Enhancing MLLMs for Low-Altitude UAV Vision-Language Understanding
Yang Zhan and Yuan Yuan
Multimodal Large
Language Models (MLLMs) have made significant strides in natural images and satellite remote sensing images. However, understanding low-altitude drone scenarios remains a challenge. Existing
datasets primarily focus on a few specific low-altitude visual tasks, which cannot fully assess the ability of MLLMs in real-world low-altitude
UAV applications. Therefore, we introduce UAVBench, a comprehensive
benchmark, and UAVIT-1M, a large-scale instruction tuning
dataset, designed to evaluate and improve MLLMs' abilities in low-altitude
vision-
language tasks. UAVBench comprises 43 test units and 966k high-quality data samples across 10 tasks at the image-level and region-level. UAVIT-1M consists of approximately 1.24 million diverse instructions, covering 789k multi-scene images and about 2,000 types of spatial resolutions with 11 distinct tasks. UAVBench and UAVIT-1M feature pure real-world visual images and rich weather conditions, and involve manual verification to ensure high quality. Our in-depth analysis of 11 state-of-the-art MLLMs using UAVBench reveals that open-source MLLMs cannot generate accurate conversations about low-altitude visual content, lagging behind closed-source MLLMs. Extensive experiments demonstrate that fine-tuning open-source MLLMs on UAVIT-1M significantly addresses this gap. Our contributions pave the way for bridging the gap between current MLLMs and low-altitude
UAV real-world application demands. (Project page: https://UAVBench.github.io/)
Read the full abstract ▾
Computer Vision
AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising
Liyuan Cui, Wentao Hu, Wenyuan Zhang et al. (+3 more)
Real-time talking avatar generation requires low latency and minute-level temporal stability. Autoregressive (AR) forcing enables streaming inference but suffers from exposure bias, which causes errors to accumulate and become irreversible over long rollouts. In contrast, full-sequence
diffusion transformers mitigate drift but remain computationally prohibitive for real-time long-form synthesis. We present AvatarForcing, a one-step streaming
diffusion framework that denoises a fixed local-future window with heterogeneous noise levels and emits one clean block per step under constant per-step cost. To stabilize unbounded streams, the method introduces dual-anchor temporal forcing: a style anchor that re-indexes RoPE to maintain a fixed relative position with respect to the active window and applies anchor-audio zero-padding, and a temporal anchor that reuses recently emitted clean blocks to ensure smooth transitions. Real-time one-step inference is enabled by two-stage streaming
distillation with offline ODE backfill and distribution matching. Experiments on standard
benchmarks and a new 400-
video long-form
benchmark show strong visual quality and lip synchronization at 34 ms/frame using a 1.3B-parameter student model for realtime streaming. Our page is available at: https://cuiliyuan121.github.io/AvatarForcing/
Read the full abstract ▾
All authors Liyuan Cui, Wentao Hu, Wenyuan Zhang, Zesong Yang, Fan Shi, Xiaoqiang Liu
Computer Vision
How Do Medical MLLMs Fail? A Study on Visual Grounding in Medical Images
Guimeng Liu, Tianze Yu, Somayeh Ebrahimkhani et al. (+3 more)
Generalist
multimodal large
language models (MLLMs) have achieved impressive performance across a wide range of
vision-
language tasks. However, their performance on medical tasks, particularly in zero-shot settings where generalization is critical, remains suboptimal. A key research gap is the limited understanding of why medical MLLMs underperform in medical image interpretation. In this work, we present a pioneering systematic investigation into the visual grounding capabilities of state-of-the-art medical MLLMs. To disentangle visual grounding from semantic grounding, we design VGMED, a novel evaluation
dataset developed with expert clinical guidance, explicitly assessing the visual grounding capability of medical MLLMs. We introduce new quantitative metrics and conduct detailed qualitative analyses. Our study across eight state-of-the-art (SOTA) medical MLLMs validates that they often fail to ground their predictions in clinically relevant image regions. We note that this finding is specific to medical image analysis; in contrast, prior work has shown that MLLMs are capable of grounding their predictions in the correct image regions when applied to natural scene images. Motivated by these findings, we propose VGRefine, a simple yet effective inference-time method that refines
attention distribution to improve visual grounding in medical settings. Our approach achieves SOTA performance across 6 diverse Med-VQA
benchmarks (over 110K VQA samples from 8 imaging modalities) without requiring additional training or external expert models. Overall, our work, for the first time, systematically validates inadequate visual grounding as one of the key contributing factors for medical MLLMs' under-performance. Additional experiments are included in the Supp.
Read the full abstract ▾
All authors Guimeng Liu, Tianze Yu, Somayeh Ebrahimkhani, Lin Zhi Zheng Shawn, Kok Pin Ng, Ngai-Man Cheung
Computer Vision
Bisheng Wang, Jaime S. Cardoso and Lin Wu
Accurate
cell segmentation is critical for biological and medical imaging studies. Although recent deep learning models have advanced this task, most methods are limited to generic
cell segmentation, lacking the ability to differentiate specific
cell types. In this work, we introduce the Personalized
Cell Segmentation (PerCS) task, which aims to segment all
cells of a specific type given a reference
cell. To support this task, we establish a
benchmark by reorganizing publicly available
datasets, yielding 1,372 images and over 110,000 annotated
cells. As a pioneering solution, we propose PerCS-DINO, a framework built on the DINOv2 backbone. By integrating image features and reference
embeddings via a cross-
attention transformer and contrastive learning, PerCS-DINO effectively segments
cells matching the reference. Extensive experiments demonstrate the effectiveness of the proposed PerCS-DINO and highlight the challenges of this new task. We expect PerCS to serve as a useful testbed for advancing research in
cell-based applications.
Read the full abstract ▾
Computer Vision
Early Failure Detection and Intervention in Video Diffusion Models
Byungki Kwon, Sohwi Lim, Nam Hyeon-Woo et al. (+2 more)
Text-to-
video (T2V)
diffusion models have rapidly advanced, yet generations still occasionally fail in practice, such as low text-
video alignment or low perceptual quality. Since
diffusion sampling is non-deterministic, it is difficult to know during inference whether a generation will succeed or fail, incurring high computational cost due to trial-and-error regeneration. To address this, we propose an early failure detection and diagnostic intervention pipeline for latent T2V
diffusion models. For detection, we design a Real-time Inspection (RI) module that converts latents into intermediate
video previews, enabling the use of established text-
video alignment scorers for inspection in the RGB space. The RI module completes the conversion and inspection process in just 39.2ms. This is highly efficient considering that CogVideoX-5B requires 4.3s per denoising step when generating a 480p, 49-frame
video on an NVIDIA A100 GPU. Subsequently, we trigger a hierarchical and early-exit intervention pipeline only when failure is predicted. Experiments on CogVideoX-5B and Wan2.1-1.3B demonstrate consistency gains on VBench with up to 2.64 times less time overhead compared to post-hoc regeneration. Our method also generalizes to a higher-capacity setting, remaining effective on Wan2.1-14B with 720p resolution and 81-frame generation. Furthermore, our pipeline is plug-and-play and orthogonal to existing techniques, showing seamless compatibility with
prompt refinement and sampling guidance methods. We also provide evidence that failure signals emerge early in the denoising process and are detectable within intermediate
video previews using standard
vision-
language evaluators.
Read the full abstract ▾
All authors Byungki Kwon, Sohwi Lim, Nam Hyeon-Woo, Moon Ye-Bin, Tae-Hyun Oh
Computer Vision
Direct Object-Level Reconstruction via Probabilistic Gaussian Splatting
Shuai Guo, Ao Guo, Junchao Zhao et al. (+5 more)
Object-level 3D reconstruction play important roles across domains such as cultural heritage digitization, industrial manufacturing, and virtual reality. However, existing Gaussian Splatting-based approaches generally rely on full-scene reconstruction, in which substantial redundant background information is introduced, leading to increased computational and storage overhead. To address this limitation, we propose an efficient single-object 3D reconstruction method based on 2D Gaussian Splatting. By directly integrating foreground-background probability cues into Gaussian primitives and dynamically
pruning low-probability Gaussians during training, the proposed method fundamentally focuses on an object of interest and improves the
memory and computational efficiency. Our pipeline leverages probability masks generated by YOLO and SAM to supervise probabilistic Gaussian attributes, replacing binary masks with continuous probability values to mitigate boundary ambiguity. Additionally, we propose a dual-stage filtering strategy for training's startup to suppress background Gaussians. And, during training, rendered probability masks are conversely employed to refine supervision and enhance boundary consistency across views. Experiments conducted on the MIP-360, T&T, and NVOS
datasets demonstrate that our method exhibits strong self-correction capability in the presence of mask errors and achieves reconstruction quality comparable to standard 3DGS approaches, while requiring only approximately 1/10 of their Gaussian amount. These results validate the efficiency and robustness of our method for single-object reconstruction and highlight its potential for applications requiring both high fidelity and computational efficiency.
Read the full abstract ▾
All authors Shuai Guo, Ao Guo, Junchao Zhao, Qi Chen, Yuxiang Qi, Zechuan Li, Dong Chen, Mingliang Xu
Computer Vision
In-Field 3D Wheat Head Instance Segmentation From TLS Point Clouds Using Deep Learning Without Manual Labels
Tomislav Medic and Liangliang Nan
3D instance
segmentation for
laser scanning (
LiDAR) point clouds remains a challenge in many remote sensing-related domains. Successful solutions typically rely on supervised deep learning and manual annotations, and consequently focus on objects that can be well delineated through visual inspection and manual labeling of point clouds. However, for tasks with more complex and cluttered scenes, such as in-field plant phenotyping in agriculture, such approaches are often infeasible. In this study, we tackle the task of in-field wheat head instance
segmentation directly from terrestrial
laser scanning (TLS) point clouds. To address the problem and circumvent the need for manual annotations, we propose a novel two-stage pipeline. To obtain the initial 3D instance proposals, the first stage uses 3D-to-2D multi-view projections, the Grounded SAM pipeline for zero-shot 2D object-centric
segmentation, and multi-view label fusion. The second stage uses these initial proposals as noisy pseudo-labels to train a supervised 3D panoptic-style
segmentation neural network. Our results demonstrate the feasibility of the proposed approach and show performance improvementsrelative to Wheat3DGS, a recent alternative solution for in-field wheat head instance
segmentation without manual 3D annotations based on multi-view RGB images and 3D Gaussian Splatting, showcasing TLS as a competitive sensing alternative. Moreover, the results show that both stages of the proposed pipeline can deliver usable 3D instance
segmentation without manual annotations, indicating promising, low-effort transferability to other comparable TLS-based point cloud
segmentation tasks.
Read the full abstract ▾
Computer Vision
A Physically-Grounded Attack and Adaptive Defense Framework for Real-World Low-Light Image Enhancement
Tongshun Zhang, Pingping Liu, Yuqing Lei et al. (+3 more)
Limited illumination often causes severe physical noise and detail degradation in images. Existing Low-Light Image Enhancement (LLIE) methods frequently treat the enhancement process as a blind black-box mapping, overlooking the physical noise transformation during imaging, leading to suboptimal performance. To address this, we propose a novel LLIE approach, conceptually formulated as a physics-based attack and display-adaptive defense paradigm. Specifically, on the attack side, we establish a physics-based Degradation Synthesis (PDS) pipeline. Unlike standard data augmentation, PDS explicitly models Image Signal Processor (ISP) inversion to the RAW domain, injects physically plausible
photon and read noise, and re-projects the data to the sRGB domain. This generates high-fidelity training pairs with explicitly parameterized degradation vectors, effectively simulating realistic attacks on clean signals. On the defense side, we construct a dual-layer fortified system. A noise predictor estimates degradation parameters from the input sRGB image. These estimates guide a degradation-aware Mixture of Experts (DA-MoE), which dynamically routes features to experts specialized in handling specific noise intensities. Furthermore, we introduce an Adaptive Metric Defense (AMD) mechanism, dynamically calibrating the feature
embedding space based on noise severity, ensuring robust representation learning under severe degradation. Extensive experiments demonstrate that our approach offers significant plug-and-play performance enhancement for existing
benchmark LLIE methods, effectively suppressing real-world noise while preserving structural fidelity. The sourced code is available at https://github.com/bywlzts/Attack-defense-llie.
Read the full abstract ▾
All authors Tongshun Zhang, Pingping Liu, Yuqing Lei, Zixuan Zhong, Qiuzhan Zhou, Zhiyuan Zha
Computer Vision
4D Synchronized Fields: Motion-Language Gaussian Splatting for Temporal Scene Understanding
Mohamed Rayan Barhdadi, Samir Abdaljalil, Rasul Khanbayov et al. (+2 more)
Current 4D representations decouple geometry, motion, and semantics: reconstruction methods discard interpretable motion structure;
language-grounded methods attach semantics after motion is learned, blind to how objects move; and motion-aware methods encode dynamics as opaque per-point residuals without object-level organization. We propose 4D Synchronized Fields, a 4D Gaussian representation that learns object-factored motion in-loop during reconstruction and synchronizes
language to the resulting kinematics through a per-object conditioned field. Each Gaussian trajectory is decomposed into shared object motion plus an implicit residual, and a kinematic-conditioned ridge map predicts temporal semantic variation, yielding a single representation in which reconstruction, motion, and semantics are structurally coupled and enabling open-vocabulary temporal queries that retrieve both objects and moments. On HyperNeRF, 4D Synchronized Fields achieves 28.52 dB mean PSNR, the highest among all
language-grounded and motion-aware baselines, within 1.5 dB of reconstruction-only methods. On targeted temporal-state
retrieval, the kinematic-conditioned field attains 0.884 mean accuracy, 0.815 mean vIoU, and 0.733 mean tIoU, surpassing 4D LangSplat (0.620, 0.433, and 0.439 respectively) and LangSplat (0.415, 0.304, and 0.262). Ablation confirms that kinematic conditioning is the primary driver, accounting for +0.45 tIoU over a static-
embedding-only baseline. 4D Synchronized Fields is the only method that jointly exposes interpretable motion primitives and temporally grounded
language fields from a single trained representation. Code will be released.
Read the full abstract ▾
All authors Mohamed Rayan Barhdadi, Samir Abdaljalil, Rasul Khanbayov, Erchin Serpedin, Hasan Kurban
Computer Vision
Show Me When and Where: Towards Referring Video Object Segmentation in the Wild
Mingqi Gao, Jinyu Yang, Jingnan Luo et al. (+4 more)
Referring
video object
segmentation (RVOS) has recently generated great popularity in computer
vision due to its widespread applications. Existing RVOS setting contains elaborately trimmed
videos, with text-referred objects always appearing in all frames, which however fail to fully reflect the realistic challenges of this task. This simplified setting requires RVOS methods to only predict where objects, with no need to show when the objects appear. In this work, we introduce a new setting towards in-the-wild RVOS. To this end, we collect a new
benchmark dataset using Youtube Untrimmed
videos for RVOS - YoURVOS, which contains 1,120 in-the-wild
videos with 7 times more duration and scenes than existing
datasets. Our new
benchmark challenges RVOS methods to show not only where but also when objects appear in
videos. To set a baseline, we propose Object-level
Multimodal TransFormers (OMFormer) to tackle the challenges, which are characterized by encoding object-level
multimodal interactions for efficient and global spatial-temporal localisation. We demonstrate that previous VOS methods struggle on our YoURVOS
benchmark, especially with the increase of target-absent frames, while our OMFormer consistently performs well. Our YoURVOS
dataset offers an imperative
benchmark, which will push forward the advancement of RVOS methods for practical applications.
Read the full abstract ▾
All authors Mingqi Gao, Jinyu Yang, Jingnan Luo, Xiantong Zhen, Jungong Han, Giovanni Montana, Feng Zheng
Computer Vision
RL-ScanIQA: Reinforcement-Learned Scanpaths for Blind 360{\deg}Image Quality Assessment
Yujia Wang, Yuyan Li, Jiuming Liu et al. (+3 more)
Blind 360{\deg}image quality assessment (IQA) aims to predict perceptual quality for panoramic images without a pristine reference. Unlike conventional planar images, 360{\deg}content in immersive environments restricts viewers to a limited viewport at any moment, making viewing behaviors critical to quality perception. Although existing scanpath-based approaches have attempted to model viewing behaviors by approximating the human view-then-rate paradigm, they treat scanpath generation and quality assessment as separate steps, preventing end-to-end optimization and task-aligned exploration. To address this limitation, we propose RL-ScanIQA, a reinforcement-learned framework for blind 360{\deg}IQA. RL-ScanIQA optimize a PPO-trained scanpath policy and a quality assessor, where the policy receives quality-driven feedback to learn task-relevant viewing strategies. To improve training stability and prevent mode collapse, we design multi-level rewards, including scanpath diversity and equator-biased priors. We further boost cross-
dataset robustness using distortion-space augmentation together with rank-consistent losses that preserve intra-image and inter-image quality orderings. Extensive experiments on three
benchmarks show that RL-ScanIQA achieves superior in-
dataset performance and cross-
dataset generalization. Codes are available at https://github.com/wangyuji1/RLScanIQA.git.
Read the full abstract ▾
All authors Yujia Wang, Yuyan Li, Jiuming Liu, Fang-Lue Zhang, Xinhu Zheng, Neil.A Dodgson
Computer Vision
Seeking Physics in Diffusion Noise
Chujun Tang, Lei Zhong and Fangqiang Ding
Do
video diffusion models encode signals predictive of physical plausibility? We probe intermediate denoising representations of pretrained
Diffusion Transformers (DiTs) and find that physically plausible and implausible
videos are partially separable in mid-layer feature space, even at high noise levels. Within-source and perceptual-quality controls suggest that this signal is not fully explained by generator identity or generic visual quality. We distill the signal into a lightweight, backbone-specific physics verifier trained on frozen features and use it in two complementary inference-time mechanisms under a fixed multi-trajectory budget: progressive trajectory selection, which scores trajectories at intermediate checkpoints and prunes weak candidates early, and reward-gradient guidance, which steers surviving trajectories by backpropagating through only the first few DiT blocks. Experiments on PhyGenBench and Physics-IQ across CogVideoX-2B/5B and Wan 2.1-14B show that progressive selection matches verifier-based Best-of-4 on CogVideoX-2B while reducing wall-clock inference time by 37%, whereas reward-gradient guidance substantially improves physical consistency on CogVideoX-5B, all without fine-tuning the
video generator.
Read the full abstract ▾
Computer Vision
RegFormer++: An Efficient Large-Scale 3D LiDAR Point Registration Network with Projection-Aware 2D Transformer
Jiuming Liu, Guangming Wang, Zhe Liu et al. (+7 more)
Although point cloud registration has achieved remarkable advances in object-level and indoor scenes, large-scale
LiDAR registration methods has been rarely explored before. Challenges mainly arise from the huge point scale, complex point distribution, and numerous outliers within outdoor
LiDAR scans. In addition, most existing registration works generally adopt a two-stage paradigm: They first find correspondences by extracting discriminative local descriptors and then leverage robust
estimators (e.g. RANSAC) to filter outliers, which are highly dependent on well-designed descriptors and post-processing choices. To address these problems, we propose a novel end-to-end differential
transformer network, termed RegFormer++, for large-scale point cloud
alignment without requiring any further post-processing. Specifically, a hierarchical projection-aware 2D
transformer with linear complexity is proposed to project raw
LiDAR points onto a cylindrical surface and extract global point features, which can improve resilience to outliers due to long-range dependencies. Because we fill original 3D coordinates into 2D projected positions, our designed
transformer can benefit from both high efficiency in 2D processing and accuracy from 3D
geometric information. Furthermore, to effectively reduce wrong point matching, a Bijective Association
Transformer (BAT) is designed, combining both cross
attention and all-to-all point gathering. To improve training stability and robustness, a feature-transformed optimal transport module is also designed for regressing the final pose transformation. Extensive experiments on KITTI, NuScenes, and Argoverse
datasets demonstrate that our model achieves state-of-the-art performance in terms of both accuracy and efficiency.
Read the full abstract ▾
All authors Jiuming Liu, Guangming Wang, Zhe Liu, Chaokang Jiang, Haoang Li, Mengmeng Liu, Tianchen Deng, Marc Pollefeys, Michael Ying Yang, Hesheng Wang
Computer Vision
Multi-Period Texture Contrast Enhancement for Low-Contrast Wafer Defect Detection and Segmentation
Zihan Zhang
Wafer defect
segmentation is pivotal for
semiconductor yield optimization yet remains challenged by the intrinsic conflict between microscale anomalies and highly periodic, overwhelming background textures. Existing deep learning paradigms often falter due to feature dilution during downsampling and the lack of explicit mechanisms to disentangle low-contrast defects from process-induced noise. To transcend these limitations, we propose TexWDS, a texture-aware framework that harmonizes multi-scale feature retention with frequency-domain perturbation modeling. Our methodology incorporates three strategic innovations: (1) A Multi-scale Receptive Field Reweighting strategy is introduced to mitigate aliasing effects and preserve high-frequency details of micro-defects often lost in standard pyramidal architectures. (2) The Multi-scale Unified Semantic Enhancer (MUSE) integrates local appearance with global context encoding, effectively enhancing feature discriminability in low-visibility regions. (3) Crucially, we design a plug-and-play Multi-Periodic Texture Contrast Enhancement (MPTCE) module. By modeling texture disruptions in the frequency domain, MPTCE explicitly decouples non-periodic anomalies from structured backgrounds, boosting contrast for camouflaged defects. Extensive experiments on real-world industrial
datasets demonstrate that TexWDS achieves a new state-of-the-art, surpassing the baseline by 8.3% in mAP50-95 and 7.7% in recall, while reducing the false positive rate by approximately 8.6%. These results underscore the framework's robustness in handling complex periodic patterns and its suitability for high-precision manufacturing inspection.
Read the full abstract ▾
Computer Vision
DC-ViT: Modulating Spatial and Channel Interactions for Multi-Channel Images
Umar Marikkar, Syed Sameed Husain, Muhammad Awais et al. (+1 more)
Training and evaluation in multi-channel imaging (MCI) remains challenging due to heterogeneous channel configurations arising from varying staining protocols, sensor types, and acquisition settings. This heterogeneity limits the applicability of fixed-channel encoders commonly used in general computer
vision. Recent Multi-Channel
Vision Transformers (MC-ViTs) address this by enabling flexible channel inputs, typically by jointly encoding patch
tokens from all channels within a unified
attention space. However, unrestricted
token interactions across channels can lead to feature dilution, reducing the ability to preserve channel-specific semantics that are critical in MCI data. To address this, we propose Decoupled
Vision Transformer (DC-ViT), which explicitly regulates information sharing using Decoupled Self-
Attention (DSA), which decomposes
token updates into two complementary pathways: spatial updates that model intra-channel structure, and channel-wise updates that adaptively integrate cross-channel information. This decoupling mitigates informational collapse while allowing selective inter-channel interaction. To further exploit these enhanced channel-specific representations, we introduce Decoupled Aggregation (DAG), which allows the model to learn task-specific channel importances. Extensive experiments across three MCI
benchmarks demonstrate consistent improvements over existing MC-ViT approaches. Code is available at https://github.com/umarikkar/DCViT.
Read the full abstract ▾
All authors Umar Marikkar, Syed Sameed Husain, Muhammad Awais, Sara Atito
Computer Vision
All-day Multi-scenes Lifelong Vision-and-Language Navigation with Tucker Adaptation
Xudong Wang, Gan Li, Zhiyu Liu et al. (+3 more)
Deploying
vision-and-
language navigation (VLN)
agents requires adaptation across diverse scenes and environments, but fine-tuning on a specific scenario often causes catastrophic forgetting in others, which severely limits flexible long-term deployment. We formalize this challenge as the all-day multi-scenes lifelong VLN (AML-VLN) problem. Existing parameter-efficient adapters (e.g., LoRA and its variants) are limited by their two-dimensional
matrix form, which fails to capture the multi-hierarchical
navigation knowledge spanning multiple scenes and environments. To address this, we propose Tucker Adaptation (TuKA), which represents the multi-hierarchical
navigation knowledge as a high-order tensor and leverages Tucker decomposition to decouple the knowledge into shared subspaces and scenario-specific experts. We further introduce a decoupled knowledge incremental learning strategy to consolidate shared subspaces while constraining specific experts for decoupled lifelong learning. Building on TuKA, we also develop a VLN
agent named AlldayWalker, which continually learns across multiple
navigation scenarios, achieving all-day multi-scenes
navigation. Extensive experiments show that AlldayWalker consistently outperforms state-of-the-art baselines.
Read the full abstract ▾
All authors Xudong Wang, Gan Li, Zhiyu Liu, Yao Wang, Lianqing Liu, Zhi Han
Computer Vision
Adaptation of Foundation Models in Medical Image Analysis: Mechanisms, Evidence, and Deployment Considerations
Karma Phuntsho, Abdullah, Kyungmi Lee et al. (+2 more)
Foundation models, pretrained on diverse data, objectives, or modalities for reuse across downstream tasks, are increasingly applied to medical image analysis. Their performance depends not only on pretraining but also on how they are adapted to specific tasks, data regimes, and deployment settings. Whereas previous reviews have considered adaptation within broader syntheses, this review focuses specifically on post-pretraining interventions applied to existing FMs. Using a structured narrative review with a systematic, auditable workflow, we identified 719 eligible reports representing 705 independent studies. Of these, 473 provided controlled comparative evidence: 322 formally published studies formed the primary synthesis and 151 preprint-only studies a sensitivity layer. Adaptation was organized into five non-mutually exclusive families: parameter-level, representation-level, objective-level, data-centric, and architectural/sequence-level adaptation, with model use and update regime analyzed separately. Controlled comparisons showed no consistent ranking among frozen, parameter-efficient, partial, or full updating, or across broader adaptation families; outcomes varied by task, data regime, comparator, and evaluation setting. Clinical-readiness evidence remained limited: 173/705 studies (24.5%) had confirmed independent external validation, 10 (1.4%) reported calibration or uncertainty evidence, and 97/322 formally published comparative studies (30.1%) included a qualifying externally validated arm. Overall, FM adaptation is a context-dependent design space rather than a hierarchy of preferred strategies.
Read the full abstract ▾
All authors Karma Phuntsho, Abdullah, Kyungmi Lee, Ickjai Lee, Euijoon Ahn
Computer Vision
DiFlowDubber: Discrete Flow Matching for Automated Video Dubbing via Cross-Modal Alignment and Synchronization
Ngoc-Son Nguyen, Thanh V. T. Tran, Jeongsoo Choi et al. (+3 more)
Video dubbing requires content accuracy, expressive prosody, high-quality acoustics, and precise lip synchronization, yet existing approaches struggle on all four fronts. To address these issues, we propose DiFlowDubber, the first
video dubbing framework built upon a discrete flow matching backbone with a novel two-stage training strategy. In the first stage, a zero-shot text-to-
speech (TTS) system is pre-trained on large-scale corpora, where a deterministic architecture captures linguistic structures, and the Discrete Flow-based Prosody-Acoustic (DFPA) module models expressive prosody and realistic acoustic characteristics. In the second stage, we propose the Content-Consistent Temporal Adaptation (CCTA) to transfer TTS knowledge to the dubbing domain: its Synchronizer enforces cross-modal
alignment for lip-synchronized
speech. Complementarily, the Face-to-Prosody Mapper (FaPro) conditions prosody on facial expressions, whose outputs are then fused with those of the Synchronizer to construct rich, fine-grained
multimodal embeddings that capture prosody-content correlations, guiding the DFPA to generate expressive prosody and acoustic
tokens for content-consistent
speech. Experiments on two
benchmark datasets demonstrate that DiFlowDubber outperforms prior methods across multiple evaluation metrics.
Read the full abstract ▾
All authors Ngoc-Son Nguyen, Thanh V. T. Tran, Jeongsoo Choi, Hieu-Nghia Huynh-Nguyen, Truong-Son Hy, Van Nguyen
Computer Vision · Cross-listed from cs.SE
ITKIT: Feasible CT Image Analysis based on SimpleITK and MMEngine
Yiqin Zhang and Meiling Chen
CT images are widely used in clinical diagnosis and treatment, and their data have formed a de facto standard - DICOM. It is clear and easy to use, and can be efficiently utilized by data-driven analysis methods such as deep learning. In the past decade, many program frameworks for medical image analysis have emerged in the open-source community. ITKIT analyzed the characteristics of these frameworks and hopes to provide a better choice in terms of ease of use and configurability. ITKIT offers a complete pipeline from DICOM to 3D
segmentation inference. Its basic practice only includes some essential steps, enabling users with relatively weak computing capabilities to quickly get started using the CLI according to the documentation. For advanced users, the OneDL-MMEngine framework provides a flexible model configuration and deployment entry. This paper conducted 12 typical experiments to verify that ITKIT can meet the needs of most basic scenarios.
Read the full abstract ▾
Computer Vision
ZOTTA: Test-Time Adaptation with Gradient-Free Zeroth-Order Optimization
Ronghao Zhang, Shuaicheng Niu, Qi Deng et al. (+3 more)
Test-time adaptation (TTA) aims to improve model robustness under distribution shifts by adapting to unlabeled test data, but most existing methods rely on backpropagation (BP), which is computationally costly and incompatible with non-differentiable models such as quantized models, limiting practical deployment on numerous edge devices. Recent BP-free approaches alleviate overhead but remain either architecture-specific or limited in optimization capacity to handle high-dimensional models. We propose ZOTTA, a fully BP-free TTA framework that performs efficient adaptation using only forward passes via Zeroth-Order Optimization (ZOO). While ZOO is theoretically appealing, naive application leads to slow convergence under high-dimensional parameter spaces and unstable optimization due to the lack of labels. ZOTTA overcomes these challenges through 1) Distribution-Robust Layer Selection, which automatically identifies and freezes layers that already extract distribution-invariant features, updating only domain-sensitive layers to reduce the optimization dimensionality and accelerate convergence; 2) Spatial Feature Aggregation
Alignment, which stabilizes ZOO by aligning globally aggregated spatial features between source and target to reduce gradient variance. Together, these components enable architecture-agnostic and stable BP-free adaptation. Extensive experiments on ImageNet-C/R/Sketch/A show that ZOTTA outperforms or matches BP-based methods, e.g., it reduces
memory usage by 84% and improves accuracy by 3.9% over SAR on ImageNet-C. Code:https://github.com/Zhang-Ronghao/zotta.
Read the full abstract ▾
All authors Ronghao Zhang, Shuaicheng Niu, Qi Deng, Yanjie Dong, Jian Chen, Runhao Zeng
Computer Vision
MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos
Sagnik Majumder, Anish Nethi, Ziad Al-Halah et al. (+1 more)
We introduce the task of early mistake detection in
video, where the goal is to determine whether a keystep in a procedural activity is performed correctly while observing as little of the streaming
video as possible. To tackle this problem, we propose a method comprising a mistake detector and a reinforcement learning policy. At each timestep, the detector processes recently observed frames to estimate the keystep's correctness while anticipating future visual features, enabling reliable early mistake estimates. Meanwhile, the policy aggregates the detector outputs and visual observations over time and adaptively decides when to exit (i.e., stop processing incoming frames) while producing the final prediction. Using diverse real-world procedural
video datasets, we demonstrate that our MistExit model achieves superior mistake detection accuracy while reducing the fraction of
video observed compared to state-of-the-art models. Project: https://
vision.cs.utexas.edu/projects/mist_exit.
Read the full abstract ▾
All authors Sagnik Majumder, Anish Nethi, Ziad Al-Halah, Kristen Grauman