Computer Vision
DSAA: Dual-Stage Attribute Activation for Fine-grained Open Vocabulary Detection
Donghong Jiang, Endian Lin, Hanqing Liu et al. (+4 more)
Open-Vocabulary Object Detection (OVD) models break the limitations of closed-set detection, enabling the identification of unseen categories through natural
language prompts. However, they exhibit notable limitations in fine-grained detection tasks involving attributes like color, material, and texture. We attribute this performance bottleneck in OVD models to a core issue: when category signals dominate, OVD models tend to marginalize attribute information during inference. This leads to incorrect binding between attributes and target objects. To address this, we propose the Dual-Stage Attribute Activation (DSAA) framework, which enhances fine-grained detection capabilities by strengthening attribute semantics at two critical stages. In the text
embedding stage, we employ Attribute Prefix Adapter (APA) module to generate attribute prefixes that inject explicit attribute priors. To further amplify the influence of these attributes, our Key/Value (K/V) Modulator module then intervenes during the BERT encoding phase, selectively enhancing the Key and Value vectors of the corresponding attribute
tokens. In addition, we introduce an attribute-aware contrastive loss to improve discrimination among same-category instances with different attributes during training. Experimental results on the FG-OVD
benchmark demonstrate the effectiveness of our method across various mainstream open-vocabulary models.
Read the full abstract ▾
All authors Donghong Jiang, Endian Lin, Hanqing Liu, Mingjie Liu, Luoping Cui, Zhao Yang, Chuang Zhu
Computer Vision
See What I Mean: Aligning Vision and Language Representations for Video Fine-grained Object Understanding
Boyuan Sun, Bowen Yin, Yuanming Li et al. (+2 more)
We present SWIM (See What I Mean), a novel training strategy that aligns
vision and
language representations to enable fine-grained object understanding solely from textual
prompts. Unlike existing approaches that require explicit visual
prompts, such as masks or points, SWIM leverages mask supervision only during training to guide cross-modal
attention, allowing the model to automatically attend to the user-specified object at inference. Our cross-
attention analysis of pretrained
multimodal large languagemodels (MLLMs) reveals a systematic discrepancy: Attribute words produce sharp, localized activations in the visual modality, whereas object nouns yield diffuse and scattered patterns due to semantic reference bias and distributed high-level representations. To address this misalignment, we construct NL-Refer, an enriched
dataset, in which each object mask is paired with a precise natural
language referring expression. SWIM extracts multi-layer cross-
attention maps from object nouns and enforces spatial consistency with ground-truth masks. Experimental results demonstrate that SWIM substantially improves text-visual
alignment and achieves superior performance over visual-
prompt-based methods on fine-grained object understanding
benchmarks. The code and data are available at \href{https://github.com/HumanMLLM/SWIM}{https://github.com/HumanMLLM/SWIM}.
Read the full abstract ▾
All authors Boyuan Sun, Bowen Yin, Yuanming Li, Xihan Wei, Qibin Hou
Computer Vision
TinySAM 2: Extreme Memory Compression for Efficient Track Anything Model
Zhaoyuan Ding, Yijing Yang, Han Shu et al. (+1 more)
Segment Anything Model 2 (SAM 2) serves as a core foundation model in the field of
video segmentation. Building upon the original SAM model, it introduces a
memory bank mechanism and demonstrates outstanding performance in tasks such as semi-supervised
video object
segmentation and tracking anything. However, the complex computational characteristics of SAM 2's multi-stage image encoder and
memory module have raised the barrier to the model's deployment in practical applications. To address this issue, we propose TinySAM 2, a lightweight
video segmentation model that balances performance and efficiency. First, a
memory quality management mechanism is introduced to select and retain high-informative historical frames as the
memory. In addition, a joint-spatial-temporal
token compression is proposed that reduces the
memory storage and computational cost. Specifically, average pooling is employed to first compress redundancy
tokens in the spatial domain. In the temporal domain, informative
tokens are selected across frames in the
memory bank based on
token-level similarity measurement. Besides, we take RepViT as the lightweight image encoder, which further reduces the model parameters. Extensive experiments on challenging
datasets such as DAVIS and SA-V demonstrate that TinySAM 2 achieves 90% of the performance of SAM 2.1, with only 7%
memory tokens and 3% training data. This study effectively alleviates the bottlenecks in parameter count, computational load, and deployment costs associated with SAM 2, providing a resource-efficient solution for the widespread application of
video segmentation models on devices.
Read the full abstract ▾
All authors Zhaoyuan Ding, Yijing Yang, Han Shu, Xinghao Chen
Computer Vision
SAS: Semantic-aware Sampling for Generative Dataset Distillation
Mingzhuo Li, Guang Li, Linfeng Ye et al. (+4 more)
Deep neural networks have achieved impressive performance across a wide range of tasks, but this success often comes with substantial computational and storage costs due to large-scale training data.
Dataset distillation addresses this challenge by constructing compact yet informative
datasets that enable efficient model training while maintaining downstream performance. However, most existing approaches primarily emphasize matching data distributions or downstream training statistics, with limited
attention to preserving high-level semantic information in the distilled data. In this work, we introduce a semantic-aware perspective for
dataset distillation by leveraging Contrastive
Language-Image Pretraining (CLIP) as a semantic prior for post-sampling. Our goal is to obtain distilled
datasets that are not only compact but also semantically class-discriminative and diverse. To this end, we design three semantic scoring functions that quantify class relevance, inter-class separability, and intra-set diversity in a pretrained semantic space. Based on image pools generated by existing
distillation methods, we further develop a two-stage strategy for effective sampling: the first stage filters semantically discriminative samples to form a reliable candidate set, and the second stage performs a dynamic diversity-aware selection to reduce redundancy while preserving semantic coverage. Extensive experiments across multiple
datasets, image pools, and downstream models demonstrate consistent performance gains, highlighting the effectiveness of incorporating semantic information into
dataset distillation.
Read the full abstract ▾
All authors Mingzhuo Li, Guang Li, Linfeng Ye, Jiafeng Mao, Takahiro Ogawa, Konstantinos N. Plataniotis, Miki Haseyama
Computer Vision
Functionalization via Structure Completion and Motion Rectification
Mingrui Zhao, Sai Raj Kishore Perla, Kai Wang et al. (+8 more)
Acquisition and creation of 3D assets have been largely view- or appearance-driven. As a result, existing digital 3D models often lack the requisite structural components to function as intended, such as joints, supports, interiors, or interaction elements. At the same time, even human-annotated motions are frequently error-prone, leading to physically implausible behavior. We introduce object functionalization, a novel task aimed at transforming visually plausible but non-functional 3D models into functional and physically operable ones. We formulate functionalization as a graph completion problem over a new functional graph representation, where labeled nodes represent object parts, labeled edges encode functional and contact relations, and movable nodes carry motion attributes, so that structural functional deficiencies manifest as missing nodes or incorrect edges. We develop a neural Graph Functionalizer (GraFu) to complete an incomplete graph representing a non-functional 3D object. The completed graph then drives a geometry realization stage that instantiates predicted connectors and structural elements in 3D, with the compelling side effect of rectifying erroneous human-annotated and predicted motions. To support training and evaluation, focusing on furniture as a rich and challenging target category, we introduce FurFun-233, a
dataset of 233 paired non-functional and functionalized furniture models. On PartNet-Mobility ("zero-shot") and HSSD test sets, our method matches state-of-the-art methods in motion prediction accuracy while substantially improving functionality in terms of collision and connectivity. Project page: https://mingrui-zhao.github.io/Functionalization/
Read the full abstract ▾
All authors Mingrui Zhao, Sai Raj Kishore Perla, Kai Wang, Sauradip Nag, Duc Anh Nguyen, Jiayi Peng, Ruiqi Wang, Angel X. Chang, Manolis Savva, Ali Mahdavi-Amiri, Hao Zhang
Computer Vision · Cross-listed from eess.IV
Inter-LPCM: Learning-based Inter-Frame Predictive Coding for LiDAR Point Cloud Compression
Chang Sun, Hui Yuan, Shiqi Jiang et al. (+3 more)
Because
LiDAR sensors acquire point clouds with a fixed angular resolution, the resulting data can be systematically parameterized and efficiently compressed in the spherical coordinate system. Traditional spherical coordinate-based point cloud compression methods have demonstrated strong rate-distortion (RD) performance, with the predictive geometry coding (PredGeom) method in the geometry-based point cloud compression (G-PCC) standard being a prominent example. Although PredGeom includes an inter-frame prediction mode, it relies on a simple linear model, which limits its ability to capture complex motion patterns and structural dependencies. Meanwhile, existing learning-based compression methods in the spherical domain do not exploit inter-frame correlations to reduce geometry redundancy. To address these limitations, we propose a learning-based inter-frame predictive coding method, termed Inter-LPCM. For azimuth prediction, we employ a delta coding strategy based on the predefined angular resolution. To improve radius compression, we introduce an inter-frame radius predictive (Inter-RP) model that estimates the current point's radius using neighboring points from both the current frame and the registered reference frame. In addition, we design a lightweight
attention-based prediction (LAEP) model to predict elevation angles by capturing long-range
geometric correlations across different coordinates. For quantization, we propose an RD-optimized method to select quantization steps in the spherical coordinate system. For
entropy coding, we design distinct models for each spherical coordinate component. These models are adapted to the statistical priors of each coordinate, enabling more accurate probability estimation. Our source code is publicly available at https://github.com/SDUChangSun/Inter-LPCM
Read the full abstract ▾
All authors Chang Sun, Hui Yuan, Shiqi Jiang, Chongzhen Tian, Guanghui Zhang, Raouf Hamzaoui
Computer Vision · Cross-listed from cs.LG
MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
Le Su, Xing Luo and Zhi Jin
Recently, residual reconstruction-based model quantization methods have achieved promising performance in low-bit post-training quantization (PTQ) by introducing cross-layer residuals to reduce error accumulated from previous layers.However, these residuals may also introduce additional bias arising from the Hessian-approximation (HA) assumption underlying reconstruction-based PTQ, leading to suboptimal quantization performance.In this work, we analyze that multiplying the residual term by a scaling coefficient provides a direct way to mitigate the HA bias associated with residual strength, while preserving accumulated-error correction. More importantly, we observe that this trade-off is module-dependent, making a single global residual strength insufficient to balance effective correction and residual-related bias across modules.Based on these observations, we propose Module-Adaptive Residual Reconstruction (MARR), which assigns a module-specific scaling coefficient to adaptively balance accumulated-error correction and residual-related HA bias for each module.To avoid expensive per-module coefficient search and obtain a stable coefficient estimate, we design a Proportional-Integral-Derivative (PID)-based adaptive update strategy that uses reconstruction error as feedback to progressively refine this coefficient. Experiments on several typical large
language models (LLMs) and
vision transformers (ViTs) demonstrate the effectiveness of MARR under low-bit quantization (less than or equal to 4-bit), achieving up to 20.2% performance gains on LLMs and up to 4.6% relative gains on ViTs over the residual reconstruction state-of-the-art methods.Code will be made publicly available upon acceptance.
Read the full abstract ▾
Computer Vision
Low Latency Gaze Tracking via Latent Optical Sensing
Yidan Zheng, Matheus Souza, Kaizhang Kang et al. (+3 more)
We present a real-time gaze tracking system that directly acquires task-relevant latent features using a fully passive optical encoder. Instead of forming and processing full-resolution images, our approach leverages a microlens array with a co-designed binary chromium mask to perform spatially multiplexed optical encoding, producing a compact set of measurements sufficient for gaze estimation. By integrating sensing and feature extraction in the optical domain, the proposed system eliminates the need for high-bandwidth image readout and substantially reduces computational overhead. The encoded measurements are captured by a 4 x 4 phototransistor array and mapped to gaze direction using a lightweight neural network. Our proof-of-concept prototype enables an end-to-end sensing-to-inference latency of 3.4 ms, outperforming published research systems. We demonstrate the effectiveness of our approach on both simulated and real-world data, achieving competitive gaze estimation accuracy while significantly reducing end-to-end latency compared to conventional camera-based pipelines. The proposed system establishes a new operating point in the accuracy-latency-compute trade-off for latency- and resource-constrained gaze tracking. This work highlights the potential of task-driven optical sensing for ultra-low-latency, computationally efficient human-computer interaction systems.
Read the full abstract ▾
All authors Yidan Zheng, Matheus Souza, Kaizhang Kang, Qiang Fu, Hadi Amata, Wolfgang Heidrich
Computer Vision · Cross-listed from eess.IV
See Silhouettes in Motion with Neuromorphic Vision
Pei Zhang, Shijie Lin, Zhou Ge et al. (+2 more)
Quasi-bimodal objects, such as text, road signs, and barcodes, play a basic yet vital role in daily visual communication. By boiling these down to clear silhouettes, binarization uses a minimal
language to convey essential
vision cues for maximum downstream efficiency, especially for tasks that require simple
geometric, topological
reasoning rather than heavy appearance modeling. The catch is that frame-based imaging often struggles on mobile platforms like drones, self-driving cars, and
underwater vehicles, in which rapid motion causes severe motion blur and harsh lighting washes out scene details. To overcome these physical limits, neuromorphic
vision via event cameras, featuring microsecond time resolution and high dynamic range, steps in as a natural solution. Building upon this event-driven paradigm, we propose a simple yet effective dual-modal approach that harnesses the synergy between frames and events for training-free, real-time, high-frame-rate binarization on CPU-only devices. Extensive evaluations show that it earns competitive performance against leading techniques in reducing blur artifacts and delivers impressive improvements under challenging illumination at a lower computational cost. Besides, its asynchronous nature bypasses long-standing event-scarcity issues that break traditional time-binning reconstruction at fixed time slots, maintaining clear target shapes even at extreme kilohertz frame rates. Its binary results further serve as reliable representations to facilitate a range of downstream tasks. This work paves the way towards lightweight perception and interaction in
embodied intelligence on resource-constrained edge platforms.
Read the full abstract ▾
All authors Pei Zhang, Shijie Lin, Zhou Ge, Jinpeng Chen, Wei Pu
Computer Vision
Learning to Balance: Decoupled Siamese Diffusion Transformer for Reference-Based Remote Sensing Image Super-Resolution
Bin Luo, Runmin Dong, Zhaoyang Luo et al. (+4 more)
Diffusion-based methods demonstrate significant potential for remote sensing image super-resolution at large scaling factors, particularly in reference-based super-resolution (RefSR), where high-resolution reference images provide critical fine-grained texture priors. However, existing methods often suffer from a trade-off between over-reliance on reference information, which leads to texture artifacts, and under-utilization of such information, which results in insufficient detail recovery. To address these issues, we propose DS-DiT, a Decoupled Siamese
Diffusion Transformer that decouples the interaction between low-resolution (LR) and reference (Ref) conditions within the
attention mechanism. By allowing LR structural priors and Ref texture information to independently interact with the noisy latent, the framework effectively mitigates competition between the two conditional sources. To further compensate for the limited local modeling ability of global
attention, we introduce a Patch-Level Weighting (PLW) module that adaptively modulates the fusion of conditional sources. In addition, the siamese architecture enables an inference-time autoguidance strategy that exploits the prediction discrepancy between strong and weak Ref conditions to improve generation quality without additional training. Experimental results across multiple
datasets and scaling factors show that DS-DiT outperforms existing methods in both quantitative metrics and visual fidelity.
Read the full abstract ▾
All authors Bin Luo, Runmin Dong, Zhaoyang Luo, Jinxiao Zhang, Jiyao Zhao, Fan Wei, Haohuan Fu
Computer Vision
Generation Navigator: A State-Aware Agentic Framework for Image Generation
Jinming Liu, Ruoyu Feng, Yuqi Wang et al. (+2 more)
Despite rapid advances in text-to-image generation, faithfully realizing user intent remains challenging, often requiring manual multi-turn trial and error. To automate this process, existing systems rely on either simple
prompt rewriting or closed-loop
agents driven by hand-crafted rules, rather than learning to adapt actions to the evolving generation process. In this paper, we reformulate image generation as a state-conditioned action-making problem and propose Generation Navigator, a multi-turn T2I
agent that learns to dynamically steer the generation trajectory and output the next action. However, training this
agent via reinforcement learning introduces a critical credit assignment challenge: naively rewarding a trajectory based solely on a single state assigns equal credit to all actions in the rollout, ignores the quality dynamics across turns, and fails to distinguish actions that improve the trajectory from those that degrade it or waste turns without progress. We resolve this with PRE-GRPO (Peak-Retention-Efficiency Group Relative Policy Optimization), a trajectory-level reinforcement learning objective that explicitly rewards discovering a high-quality image (Peak), avoiding subsequent quality degradation across turns (Retention), and minimizing unnecessary turns (Efficiency). Experiments show substantial improvements across
benchmarks, reaching a WISE score of 0.90 and 79.06%
reasoning accuracy on T2I-ReasonBench.
Read the full abstract ▾
All authors Jinming Liu, Ruoyu Feng, Yuqi Wang, Wenjun Zeng, Xin Jin
Computer Vision
A More Word-like Image Tokenization for MLLMs
Hyun Lee, Hyemin Jeong, Yejin Kim et al. (+4 more)
Modern
multimodal large
language models (MLLMs) typically keep the
language model fixed and train a visual projector that maps the pixels into a sequence of
tokens in its
embedding space, so that images can be presented in essentially the same form as text. However, the
language model has been optimized to operate on discrete, semantically meaningful
tokens, while prevailing visual projectors transform an image into a long stream of continuous and highly correlated
embeddings. This causes the visual
tokens to behave differently from the word-like units that LLMs are originally trained to understand. We propose a novel Disentangled Visual Tokenization (DiVT) that clusters patch
embeddings into coherent semantic units, so each
token corresponds to a distinct visual concept instead of a rigid grid
cell. DiVT further adapts its
token budget to image complexity, providing an explicit accuracy-compute trade-off modifying neither the
vision encoder nor the
language model. Across diverse
multimodal benchmarks, DiVT matches or surpasses baselines with significantly fewer visual
tokens, demonstrating robustness under limited
token budgets, significantly reducing
memory cost and latency while making visual inputs more compatible with LLMs. Our code is available at https://github.com/snuviplab/DiVT.
Read the full abstract ▾
All authors Hyun Lee, Hyemin Jeong, Yejin Kim, Hyungwook Choi, Hyunsoo Cho, Soo Kyung Kim, Joonseok Lee
Computer Vision
Counting Machine Parts
Benedict Florance Arockiaraj, Elizabeth Dinella, Ankit Billa et al. (+1 more)
Counting objects in an image is a task applicable across many domains. For instance, crowd counting, inventory counting, and
cell counting have been the focus of recent research. The major challenges in estimating the count of objects include overlapping objects, object scale issues, occlusions, and varying lighting conditions. In this report, we explore the problem of counting machine washer parts. Our technique is an extension of FamNet with an additional loss component, trained on the given
dataset. We compare to three baseline methods: a traditional image processing pipeline, instance
segmentation, and density map estimation. We evaluate the performance of these
algorithms by computing the Mean Absolute Error (MAE) and the Root Mean Squared Error (RMSE) between the true object counts and the model outputs. Our approach achieves a performance of 1.96 MAE.
Read the full abstract ▾
All authors Benedict Florance Arockiaraj, Elizabeth Dinella, Ankit Billa, Ajay Anand
Computer Vision
SkyNative: A Native Multimodal Architecture for Remote Sensing Vision-Language Understanding
Xiao Yang, Ronghao Fu, Zhiwen Lin et al. (+7 more)
Remote sensing
vision-
language models (RS-VLMs) commonly employ a pretrained
vision encoder and a projection module to map image features into the
token space of a large
language model. Although effective, this modular RS-VLMs separates visual representation from
language reasoning, potentially limiting the direct involvement of fine-grained visual evidence in complex spatial inference. This challenge is particularly relevant to remote sensing imagery, which often covers broad geographic areas and contains multi-scale objects, dense target distributions, and intricate spatial layouts. In this paper, we propose SkyNative, the first study to explore a native
multimodal architecture for remote sensing
vision-
language tasks. SkyNative converts remote sensing images into visual
tokens through a lightweight patch
embedding module and places them together with text
tokens in a shared autoregressive sequence, allowing textual
tokens to directly access the preceding visual context. To accommodate the heterogeneous characteristics of the two modalities, we further adopt a modality-aware decoupling mechanism that applies modality-specific projections, normalization, and feed-forward transformations while processing both modalities through shared causal self-
attention. Extensive experiments demonstrate SkyNative's strong capabilities in dense small-object perception, large-format contextual understanding, complex
reasoning, and robustness, with scores of 68.93%, 47.40%, and 63.43% on HRRSD, RSHR
reasoning, and OmniEarth, respectively. These results suggest that the native VLM architecture explored in SkyNative represents a promising approach to RS
vision-
language modeling.
Read the full abstract ▾
All authors Xiao Yang, Ronghao Fu, Zhiwen Lin, Zhuoran Duan, Lang Sun, Jiaqi Liu, Jiashun Zhu, Jiasen Hu, Xu Na, Bo Yang
Computer Vision · Cross-listed from cs.AI
SVFSearch: A Multimodal Knowledge-Intensive Benchmark for Short-Video Frame Search in the Gaming Vertical Domain
Lingtao Mao, Huangyu Dai, Xinyu Sun et al. (+4 more)
Multimodal large
language models are increasingly used as
agent backbones that understand
multimodal inputs, plan
retrieval actions, invoke external tools, and reason over retrieved information. Yet existing
benchmarks rarely evaluate this ability in short-
video applications, where a paused frame is often visually ambiguous and answering requires vertical, long-tail, and fast-evolving domain knowledge. We introduce SVFSearch, the first open
benchmark for short-
video frame search in the Chinese gaming domain. SVFSearch contains 5,000 four-choice test examples and 4,198 auxiliary training examples, each centered on a paused game scene from a real short-
video clip. To support fair and reproducible evaluation, SVFSearch provides a frozen offline
retrieval environment with a game-domain text corpus, a topic-linked image gallery, and text, image, and
multimodal retrieval interfaces, avoiding reliance on uncontrolled web search APIs. We evaluate representative paradigms ranging from direct QA and RAG workflow to Plan-Act-Replan
agents and learned search models. Results reveal a large gap between model-only answering, practical agentic search, and oracle knowledge: the best open-source direct-QA model reaches 66.4%, the best practical
agent achieves 79.1%, and oracle knowledge reaches 95.4%. Further analysis exposes bottlenecks in visual grounding,
retrieval quality, evidence-grounded
reasoning, and tool-use behavior, including over-search, answer-only shortcuts, and
retrieval-induced misleading.
Read the full abstract ▾
All authors Lingtao Mao, Huangyu Dai, Xinyu Sun, Zihan Liang, Ben Chen, Chenyi Lei, Wenwu Ou
Computer Vision
UAVFF3D: A Geometry-Aware Benchmark for Feed-Forward UAV 3D Reconstruction
Xiang Yang, Yongli Wang, HaiFeng Li et al. (+1 more)
Feed-forward 3D reconstruction has advanced rapidly, but current models remain unreliable in
UAV photogrammetric acquisition. We argue that this failure is caused not only by appearance-domain shift, but also by
UAV-specific camera-geometry variations, especially oblique views and HFOV-height ambiguity. Existing
UAV datasets mainly emphasize scene diversity and provide limited coverage of camera configurations, which restricts robustness evaluation and
UAV-domain adaptation. To address this gap, we introduce UAVFF3D, a geometry-aware real-synthetic
benchmark for feed-forward
UAV 3D reconstruction. UAVFF3D contains more than 170k real
UAV images and more than 370k synthetic images rendered from high-quality textured 3D models, covering diverse HFOVs, flight altitudes, viewing directions, and acquisition patterns. It also includes a controlled HFOV-height test subset for diagnosing projection-geometry ambiguity. We further propose an evaluation protocol that jointly assesses camera-geometry estimation and dense scene reconstruction under a shared global
alignment, avoiding the bias caused by separate camera and geometry
alignments. Experiments on representative feed-forward reconstruction models show that UAVFF3D-based domain adaptation consistently improves camera and geometry estimation, reducing Ray Error by up to 84.2%, Pose ATE by up to 76.0%, and Chamfer Distance by up to 41.1%. In oblique scenes, adaptation reduces the oblique-nadir rotation gap by up to 90.7%. Under HFOV-height ambiguity, it improves robustness across HFOV-height configurations and yields more stable performance across HFOV settings. Incorporating camera priors further improves reconstruction under
UAV-specific acquisition geometries. The
dataset and evaluation code are available at https://github.com/yanxian-ll/UAVFF3D .
Read the full abstract ▾
All authors Xiang Yang, Yongli Wang, HaiFeng Li, Yunsheng Zhang
Computer Vision
AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents
Pan Wang, Yihao Hu, Xiujin Liu et al. (+3 more)
Vision-
language model (VLM)
agents increasingly rely on
memory-augmented reinforcement learning to reuse experience across long-horizon tasks, yet most existing frameworks store
memory as text and depend on proprietary teacher models to summarize or refine it. This design is poorly matched to spatial decision making:
geometric priors are compressed into lossy
language, and sparse interaction is often supervised through delayed textual feedback rather than dense visually grounded signals. We argue that reusable experience for VLM
agents should remain visually grounded. Based on this insight, we propose AtlasVA, a teacher-free visual skill
memory framework that organizes
memory into three complementary layers: spatial heatmaps, visual exemplars, and symbolic text skills. AtlasVA further evolves danger and affinity atlases directly from trajectory statistics and lightweight grid heuristics, and reuses these self-evolving atlases as potential-based shaping rewards for reinforcement learning. This unifies perception,
memory, and optimization without external LLM supervision. Experiments on Sokoban, FrozenLake, 3D
embodied navigation, and 3D robotic manipulation
benchmarks show that AtlasVA consistently outperforms text-centric
memory baselines and competitive VLM
agents, with especially strong gains on spatially intensive tasks. Homepage: https://wangpan-ustc.github.io/AtlasvaWeb
Read the full abstract ▾
All authors Pan Wang, Yihao Hu, Xiujin Liu, Jingchu Yang, Hang Wang, Zhihao Wen
Computer Vision
An Efficient Streaming Video Understanding Framework with Agentic Control
Jinming Liu, Jianguo Huang, Zhaoyang Jia et al. (+7 more)
Streaming
video requires handling dynamic information density under strict latency budgets. Yet, existing methods typically employ static strategies, such as fixed
memory compression or reliance on a single model, forcing a trade-off: fast models fail on complex queries, while always-on heavy models violate real-time constraints and overcomplicate simple queries. Rather than fixing these decisions upfront, we propose R3-Streaming (Remember, Respond, Reason), which formulates streaming
video understanding as a cascaded control problem: for each query, the system compresses
memory, judges response readiness, and routes computation sequentially, so that each downstream decision builds on progressively refined information states. To optimize this pipeline, we introduce an age-aware forgetting policy for
memory compression, as aggressively compressing historical frames can yield substantial performance gains. For compute routing, we propose TB-GRPO, a target-balanced reinforcement learning objective that routes hard queries to a stronger model while preventing mode collapse. Extensive evaluations demonstrate that R3-Streaming achieves state-of-the-art results among streaming MLLMs, reaching 57.92 on OVO-Bench and 76.36 on StreamingBench, while reducing visual
token usage by 95 to 96 percent.
Read the full abstract ▾
All authors Jinming Liu, Jianguo Huang, Zhaoyang Jia, Jiahao Li, Xiaoyi Zhang, Zongyu Guo, Bin Li, Wenjun Zeng, Yan Lu, Xin Jin
Computer Vision · Cross-listed from cs.LG
Domain Transfer Becomes Identifiable via a Single Alignment
Sagar Shrestha, Subash Timilsina, Hoang-Son Nguyen et al. (+1 more)
Domain transfer (DT) maps source to target distributions and supports tasks such as unsupervised image-to-image translation, single-
cell analysis, and cross-platform medical imaging. However, DT is fundamentally ill-posed: push-forward mappings are generally non-identifiable, as measure-preserving
automorphisms (MPAs) preserve marginals while altering cross-domain correspondences, leading to content-misaligned translation. Recent work shows that MPAs can be eliminated by jointly transferring multiple corresponding source/target conditional distributions, but supervision signals labeling such conditionals are not always available in practice. We develop an alternative route to DT identifiability. Under a structural sparsity condition on the Jacobian support pattern, we show that distribution matching together with a single paired anchor sample suffices to identify the ground-truth transfer – requiring substantially less supervision than prior approaches. To enable practical high-dimensional learning, we further propose an efficient Jacobian sparsity regularizer based on randomized masked finite differences, yielding a scalable surrogate without explicit Jacobian evaluation. Empirical results on synthetic and real-world DT tasks validate the theory.
Read the full abstract ▾
All authors Sagar Shrestha, Subash Timilsina, Hoang-Son Nguyen, Xiao Fu
Computer Vision
PanoWorld: A Generative Spatial World Model for Consistent Whole-House Panorama Synthesis
Jinrang Jia, Zhenjia Li, Yijiang Hu et al. (+1 more)
Generating a consistent whole-house VR tour from a floorplan and style reference requires both photorealistic panoramas and cross-view spatial coherence. Pure 2D generators produce appealing single panoramas but re-imagine geometry and materials when the viewpoint changes, whereas monolithic 3D generation becomes expensive and loses fine texture at multi-room scale. We introduce PanoWorld, a generative spatial world model that treats whole-house synthesis as autoregressive generation of node-based 360-degree panoramas, matching the discrete
navigation used by real VR tour products. PanoWorld uses a floorplan-derived 3D shell as a global
geometric proxy and a dynamic 3D Gaussian Splatting cache as renderable spatial
memory. A feed-forward panoramic LRM designed for metric-scale multi-room 360-degree inputs lifts generated panoramas into local 3DGS updates, while Room-aware Group
Attention suppresses cross-room feature interference. A
topology-aware progressive caching strategy fuses these local updates without repeatedly reconstructing the full history. By decoupling shell-based geometry guidance from cache-rendered visual
memory, PanoWorld preserves high-frequency 2D synthesis quality while improving cross-node layout and material consistency. The project link is https://jjrcn.github.io/PanoWorld-project-home/
Read the full abstract ▾
All authors Jinrang Jia, Zhenjia Li, Yijiang Hu, Yifeng Shi
Computer Vision
SurgLQA: Scalable Long-Horizon Surgical Video Question Answering
Diandian Guo, Xikai Yang, Ruiyang Li et al. (+2 more)
Surgical Video Question Answering (VideoQA) provides a promising paradigm for dynamic intraoperative interpretation, enabling real-time decision support and context-aware
retrieval in clinical environments. Nevertheless, existing approaches are predominantly restricted to images or short clips, limiting their ability to model long-range procedural dynamics and causal dependencies across extended
surgical workflows. To address this challenge, we propose SurgLQA, a unified long-horizon VideoQA framework for scalable
surgical reasoning. This framework incorporates Faithful Temporal Consolidation (FTC), which leverages intrinsic temporal cues to construct compact long-range representations while preserving fine-grained temporal fidelity. Further, we develop Temporally-Grounded Multi-Policy Scaling (TMS), an adaptive test-time inference paradigm that strategically adjusts policy-level
reasoning capacity within temporally grounded contexts. To facilitate systematic evaluation, we restructured a long-duration colonoscopy VideoQA
benchmark, Colon-LQA, and conducted extensive experiments on Colon-LQA and REAL-Colon-VQA. Experimental results demonstrate that our approach achieves consistent performance gains in long-range
reasoning with temporally grounded inference. Code link: https://github.com/RascalGdd/SurgLQA.
Read the full abstract ▾
All authors Diandian Guo, Xikai Yang, Ruiyang Li, Jialun Pei, Pheng-Ann Heng
Computer Vision · Cross-listed from cs.RO
WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
Zhuohang Li, Yiding Ma, Yu Shang et al. (+26 more)
World models have emerged as a central paradigm for
embodied intelligence, enabling
agents to predict action-conditioned future and reason about environmental dynamics. However, existing
embodied world model
benchmarks are still largely confined to
vision-only prediction, offline
embodied applications, and simulator-based evaluation, making them insufficient for assessing increasingly comprehensive world models. In this work, we introduce WorldArena 2.0, an expanded
benchmark that systematically broadens
embodied world model evaluation along three dimensions: modality, functionality, and platform. Along the modality dimension, WorldArena 2.0 extends evaluation from
vision-only to visuotactile modalities, enabling assessment of
multimodal perception and prediction. Along the functionality dimension, it extends beyond policy evaluation and planning to assess world models as interactive RL environments for policy optimization. Along the platform dimension, it moves beyond simulator-only evaluation to a diverse suite of simulated and real-world robotic settings across multiple embodiments. Under a standardized protocol, WorldArena 2.0 comprehensively evaluates perceptual quality, interactive utility, and cross-platform performance, providing a comprehensive testbed for tracking progress toward
embodied world models. The project page including codes and demos is available at: https://world-arena.ai/.
Read the full abstract ▾
All authors Zhuohang Li, Yiding Ma, Yu Shang, Lei Jin, Shengyuan Wang, Jianjie Fang, Zhaolu Wang, Xvyuan Liu, Yinzhou Tang, Xin Jin, Ziyou Wang, Weikang Su, Xin Zhang, Haisheng Su, Weizhen He, Wei Wu, Haoyi Duan, Gordon Wetzstein, Xihui Liu, Dhruv Shah, Zhaoxiang Zhang, Zhibo Chen, Jun Zhu, Yonghong Tian, Tat-Seng Chua, Xin Wang, Wenwu Zhu, Chen Gao, Yong Li
Computer Vision
One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perception
Yang Li, Weize Li, Quan Yuan et al. (+7 more)
By sharing intermediate features, collaborative perception extends each
agent's sensing beyond standalone limits, but real-world feature modality heterogeneity remains a key barrier to effective fusion. Most existing methods, including direct adaption and protocol-based transformation, typically rely on training adapters for newly emerging feature modalities and often require additional retraining or fine-tuning. Such repeated training is costly and is often infeasible across manufacturers due to model and data
privacy constraints, limiting real-world scalability. To address this issue, we propose UniTrans, a universal any-to-any feature modality translation model that instantiates translators on the fly for arbitrary modalities. UniTrans pretrains a bank of translator expert parameters and learns their combination coefficients as a function of source-to-target modality mapping. The mapping is measured in a modality-intrinsic latent space, where an intrinsic encoder extracts modality-specific yet scene-invariant codes from single-frame intermediate features, enabling UniTrans to instantiate translators in a zero-shot manner. Experiments on OPV2V-H and DAIR-V2X demonstrate that UniTrans consistently outperforms state-of-the-art methods in both simulated and real-world settings, enabling efficient any-to-any translation through a universal model. The code is available at https://github.com/CheeryLeeyy/UniTrans.
Read the full abstract ▾
All authors Yang Li, Weize Li, Quan Yuan, Congzhang Shao, Guiyang Luo, Yunqi Ba, Xuanhan Zhu, Xinyuan Ding, Xiaoyuan Fu, Jinglin Li
Computer Vision
Beyond Euclidean Prototypes: Spectral Disentanglement and Geodesic Matching for Few-Shot Medical Image Segmentation
Penghao Jia, Zhiyong Huang, Mingyang Hou et al. (+4 more)
Few-Shot Medical Image
Segmentation (FSMIS) aims to delineate novel anatomical targets from one or a few annotated support images, addressing the annotation scarcity in medical imaging. Notwithstanding recent advancements, current prototype-based methods are bottlenecked by two coupled limitations: 1) cue
entanglement, where a single spatial-domain prototype is forced to summarise organ silhouette, parenchymal texture and boundary appearance simultaneously, so any support-query mismatch on one cue propagates indiscriminately to the others; and 2)
topology-blind matching, where cosine similarity measures distance in the ambient Euclidean space and ignores the connectivity of the underlying feature
manifold, causing fragmented activations inside low-contrast organs and leakage into neighbouring tissues. To this end, we propose Spectral-Geodesic Prototype Network (SGP-Net), built around a Spectral-Geodesic Prototype Module with two coupled components. A Spectral Prototype Bank (SPB) decomposes support and query features into low-, mid- and high-frequency bands via learnable radial
Fourier filters, yielding three disentangled prototypes per class that separately encode shape, texture and boundary cues. A Geodesic Matcher (GM) then replaces cosine similarity with a differentiable heat-
diffusion approximation of geodesic distance, propagating matching signals along a feature affinity graph so that on-
manifold pixels accumulate consistent responses while off-
manifold look-alikes are suppressed. Experiments on three public FSMIS
benchmarks demonstrate that SGP-Net achieves competitive performance against recent state-of-the-art methods.
Read the full abstract ▾
All authors Penghao Jia, Zhiyong Huang, Mingyang Hou, Zhi Yu, Shuai Miao, Jiahong Wang, Yan Yan
Computer Vision
HexagonalWarriorMamba: Superior Threshold-Dependent Multi-label Classification of 12-Lead ECG Cardiac Abnormalities
Huawei Jiang, Husna Mutahira, Shibo Wei et al. (+6 more)
The accurate automated diagnosis of cardiac abnormalities from 12-lead electrocardiograms (ECGs) is critical for managing cardiovascular disease. However, detecting concurrent conditions remains a challenge for traditional deep learning models, which often have limited ability to model the long-range dependencies inherent in ECG signals. This manuscript proposes HexagonalWarriorMamba (HWMamba), a framework built on the Mamba architecture that processes 12-lead ECGs as single-channel 2D images rather than conventional 1D time series. By integrating a hierarchical architecture with a 2D Selective Scan mechanism, HWMamba is designed to model global context and complex spatial relationships within the data. The model is evaluated on the PhysioNet/Computing in Cardiology Challenge 2021
dataset, which includes 26 diagnostic labels and comprises recordings collected from seven institutions across four countries and three continents. Results demonstrate that HWMamba outperforms current state-of-the-art (SOTA) methods across five key threshold-dependent metrics, including Challenge Score and Subset Accuracy. These improvements provide a balance between strong discriminative capability and effective threshold selection derived from the training data, while maintaining near-SOTA performance in Macro AUROC. This Hexagonal Warrior performance, reflecting consistent performance across multiple evaluation dimensions, positions HWMamba as a robust and versatile approach for multi-label ECG classification.
Read the full abstract ▾
All authors Huawei Jiang, Husna Mutahira, Shibo Wei, Jiahang Li, Vladimir Shin, Juneho Yi, Dongryeol Ryu, Wonyoung Park, Mannan Saeed Muhammad
Computer Vision
PySIFT: GPU-Resident Deterministic SIFT for Deep Learning Vision Pipelines
Sivakumar K.S., Mohammad Daniyalur Rahman and Gopi Raju Matta
A widespread assumption in local feature research holds that classical handcrafted descriptors are accuracy-limited relics best replaced by learned alternatives. We show this is wrong. Through an 8-configuration ablation spanning four
benchmarks (HPatches, ROxford5K, IMC Phototourism, MegaDepth), we demonstrate that classical SIFT with DSP multi-scale pooling outperforms neural descriptor and orientation replacements (HardNet, OriNet) on every accuracy metric–while running 2–18$\times$ faster–and that learned matchers (LightGlue) complement rather than supersede classical features. The conclusion reframes a decade of work: not "replace SIFT" but "compose with SIFT," classical extraction paired with learned matching only where
geometric context demands it. This finding was invisible because no prior GPU SIFT kept the complete pipeline in VRAM or offered modularity for controlled classical-vs-learned ablations. We present PySIFT, the first fully GPU-resident SIFT, implemented in CuPy/Numba CUDA kernels with DLPack zero-copy handoff to downstream DL frameworks–submillisecond O(1) metadata swap regardless of keypoint count. On a laptop-grade NVIDIA RTX 3050 (4 GB VRAM), PySIFT achieves: (i) higher Mean Matching Accuracy (MMA) than OpenCV SIFT on HPatches, (ii) 383 ms faster per pair on high-resolution MegaDepth, (iii) higher
geometric accuracy on cross-
dataset benchmarks (+5.6 pp AUC@10${}^\circ$ on MegaDepth, more inliers on IMC Phototourism), and (iv) bitwise deterministic output–identical keypoints and descriptors across runs, with detection reproducing identically even across GPU architectures: a guarantee that learned extractors cannot match without significant performance sacrifice, and cannot achieve at all across GPU architectures due to cuDNN's architecture-dependent
algorithm selection. PySIFT is open-source, requiring no C++ compilation.
Read the full abstract ▾
Computer Vision
Imaging Hidden Objects with Consumer LiDAR via Motion Induced Sampling
Siddharth Somasundaram, Aaron Young, Akshat Dave et al. (+2 more)
LiDARs are being increasingly deployed for consumer imaging in handheld, wearable, and robotic applications. These sensors can capture the time-of-flight of light at picosecond resolution, which in principle, enables them to capture information about objects hidden from their field of view. While such non-line-of-sight (NLOS) imaging capabilities have been shown on research-grade
LiDARs, they are challenging to achieve on consumer devices due to poor signal quality resulting from low
laser power, low spatial resolution, and object and camera motion. Inspired by burst photography and synthetic aperture radar, we propose a multi-frame fusion strategy to overcome these challenges and demonstrate NLOS imaging on consumer
LiDAR. We first introduce the motion-induced aperture sampling model to unify the effects of object shape, object motion, and camera motion under a single measurement model. Using this model, we demonstrate several NLOS capabilities on a smartphone-grade
LiDAR: (1) 3D reconstruction, (2) single and multi-object tracking, and (3) camera localization using hidden objects. Previously, NLOS imaging capabilities were largely restricted to bulky and expensive research-grade hardware that requires extensive setup and calibration. Our results represent a shift towards plug-and-play NLOS imaging, where anyone can image hidden objects with off-the-shelf hardware ($<100) and no additional setup. We believe that democratization of such capabilities will advance consumer applications of NLOS imaging.
Read the full abstract ▾
All authors Siddharth Somasundaram, Aaron Young, Akshat Dave, Adithya Pediredla, Ramesh Raskar
Computer Vision · Cross-listed from stat.ML
Simple Approximation and Derivative Free Inference-Time Scaling for Diffusion Models via Sequential Monte Carlo on Path Measures
Chenyang Wang, Weizhong Wang, Yinuo Ren et al. (+2 more)
iffusion-based generative models increasingly rely on inference-time guidance, adding a drift term or reweighting mixture of experts, to improve sample quality on task-specific objectives. However, most existing techniques require repeated score or gradient evaluations, introducing bias, high computational overhead, or both. We introduce URGE, Unbiased Resampling via Girsanov Estimation, a derivative-free inference-time scaling
algorithm that performs path-wise importance reweighting via a Girsanov change of measure. Instead of computing gradient-based particle weights in previous work, URGE attaches a simple multiplicative weight to each simulated trajectory and periodically resamples. No score, no Hessian, and no PDE evaluation is required. We establish an equivalence between path-wise and particle-wise SMC: the Girsanov path weight admits a backward conditional expectation that recovers the previous particle-level weights, guaranteeing that both schemes produce the same unbiased terminal law. Empirically, URGE outperforms existing inference-time guidance baselines on synthetic tests and
diffusion-model
benchmarks, achieving better generation quality, while being significantly simpler to implement and fully gradient-free.
Read the full abstract ▾
All authors Chenyang Wang, Weizhong Wang, Yinuo Ren, Jose Blanchet, Yiping Lu
Computer Vision
Temporal Aware Pruning for Efficient Diffusion-based Video Generation
Sheng Li, Yang Sui, Junhao Ran et al. (+3 more)
Video diffusion models have recently enabled high-quality
video generation with ViT-based architectures, but remain computationally intensive because generation requires
attention computation over long spatiotemporal sequences.
Token pruning has proven effective for ViTs and VLMs. However, most prior
pruning methods are
attention-based and operate per frame, failing to ensure the vital temporal coherence across frames in
video generation tasks. In practice, naively adopting
attention-only
pruning causes noticeable degradation due to worsened background consistency, flickering, and reduced image quality. To address this, we propose TAPE, a training-free Temporal Aware
Pruning for Efficient
diffusion-based
video generation. TAPE (i) applies temporal smoothing to align
token-importance across adjacent frames and suppress selection jitter; and (ii) performs
token reselection in selected layers to align
token pruning with layers' diverse semantic focus and avoid error accumulation in specific areas; it also (iii) adopt a timestep-level budget scheduling that prunes aggressively at early noisy steps and relaxes
pruning during fidelity-critical refinement. The experimental results show that TAPE delivers significant speedups while preserving high visual fidelity, outperforming prior
token reduction approaches.
Read the full abstract ▾
All authors Sheng Li, Yang Sui, Junhao Ran, Bo Yuan, Yue Dai, Xulong Tang
Computer Vision
Stabilizing, Scaling & Enhancing MeanFlow for Large-scale Diffusion Distillation
Xiao He, Yang Li, Peizhen Zhang et al. (+3 more)
Diffusion models exhibit remarkable generative capability, but their high latency limits practical deployment. Many studies have attempted to reduce sampling steps to accelerate inference. Among them, MeanFlow has attracted considerable
attention due to its concise formulation and remarkable performance. Nevertheless, the instability of its optimization objective and the ”mean-seeking bias” have limited its applicability to distill large-scale industrial models. To stabilize MeanFlow for distilling large-scale models, we first introduce a warm-up technique, in which the original differential solution of MeanFlow is replaced by a discrete solution. This design avoids training collapse caused by the MeanFlow target containing a stop-gradient term from an undertrained model. Once the model acquires a preliminary ability to fit the average velocity field, we switch the optimization objective back to the differential solution, enabling further refinement. Meanwhile, to alleviate the ”mean-seeking bias” of MeanFlow under extremely few-step inference with complex target distributions, we incorporate trajectory distribution
alignment as an auxiliary objective, encouraging the student model's trajectory distribution to align more closely with that of the teacher model. Our proposed
distillation framework achieves superior performance compared to existing
distillation approaches when applied to the text-to-image (T2I) model FLUX.1-dev (up to 12B parameters). Furthermore, when extended to the 80B-parameter state-of-the-art (SOTA) T2I model HunyuanImage 3.0, our method continues to demonstrate robust generalization and strong performance.
Read the full abstract ▾
All authors Xiao He, Yang Li, Peizhen Zhang, Songtao Liu, Zhao Zhong, Nannan Wang