Computer Vision · Cross-listed from cs.GR
CADReasoner: Iterative Program Editing for CAD Reverse Engineering
Soslan Kabisov, Vsevolod Kirichuk, Andrey Volkov et al. (+5 more)
Computer-Aided Design (CAD) powers modern engineering, yet producing high-quality parts still demands substantial expert effort. Many AI systems tackle CAD reverse engineering, but most are single-pass and miss fine
geometric details. In contrast, human engineers compare the input shape with the reconstruction and iteratively modify the design based on remaining discrepancies.
Agent-based methods mimic this loop with frozen VLMs, but weak 3D grounding of current foundation models limits reliability and efficiency. We introduce CADReasoner, a model trained to iteratively refine its prediction using
geometric discrepancy between the input and the predicted shape. The model outputs a runnable CadQuery Python program whose rendered mesh is fed back at the next step. CADReasoner fuses multi-view renders and point clouds as complementary modalities. To bridge the realism gap, we propose a scan-simulation protocol applied during both training and evaluation. Across DeepCAD, Fusion 360, and MCB
benchmarks, CADReasoner attains state-of-the-art results on clean and scan-sim tracks.
Read the full abstract ▾
All authors Soslan Kabisov, Vsevolod Kirichuk, Andrey Volkov, Gennadii Savrasov, Marina Barannikov, Anton Konushin, Andrey Kuznetsov, Dmitrii Zhemchuzhnikov
Computer Vision
Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field
Ozge Mercanoglu Sincan, Jian He Low, Sobhan Asasi et al. (+1 more)
Sign
Language Translation (SLT) aims to automatically convert visual sign
language videos into spoken
language text and vice versa. While recent years have seen rapid progress, the true sources of performance improvements often remain unclear. Do reported performance gains come from methodological novelty, or from the choice of a different backbone, training optimizations, hyperparameter tuning, or even differences in the calculation of evaluation metrics? This paper presents a comprehensive study of recent gloss-free SLT models by re-implementing key contributions in a unified codebase. We ensure fair comparison by standardizing preprocessing,
video encoders, and training setups across all methods. Our analysis shows that many of the performance gains reported in the literature often diminish when models are evaluated under consistent conditions, suggesting that implementation details and evaluation setups play a significant role in determining results. We make the codebase publicly available here (https://github.com/ozgemercanoglu/sltbaselines) to support transparency and reproducibility in SLT research.
Read the full abstract ▾
All authors Ozge Mercanoglu Sincan, Jian He Low, Sobhan Asasi, Richard Bowden
Computer Vision · Cross-listed from cs.AR
Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review
Luigi Capogrosso, Pietro Bonazzi and Michele Magno
This review examines the rapidly evolving landscape of ultra-low-power edge processors, covering heterogeneous Systems-on-Chips (SoCs), neural accelerators, near-sensor and in-sensor architectures, and emerging dataflow and
memory-centric designs. We categorize commercially available and research-grade platforms according to their compute paradigms, power envelopes, and
memory hierarchies, and analyze their suitability for always-on and latency-critical Artificial Intelligence (AI) workloads. To complement the architectural overview with empirical evidence, we
benchmark a 336 million Multiply-Accumulate (MAC)
segmentation model (PicoSAM2) on three representative processors: GAP9, leveraging a multi-core RISC-V architecture augmented with hardware accelerators; the STM32N6, which pairs an advanced ARM Cortex-M55 core with a dedicated neural architecture accelerator; and the Sony IMX500, representing in-sensor stacked-Complementary Metal-
Oxide-
Semiconductor (CMOS) compute. Collectively, these platforms span MCU-class, embedded neural accelerator, and in-sensor paradigms. The evaluation reports latency, inference efficiency, energy efficiency, and energy-delay product. The results show a clear divergence in hardware behavior, with the IMX500 achieving the highest utilization (86.2 MAC/cycle) and the lowest energy-delay product, highlighting the growing significance and technological maturity of in-sensor processing. GAP9 offers the best energy efficiency within microcontroller-class power budgets, and the STM32N6 provides the lowest raw latency at a significantly higher energy cost. Together, the review and
benchmarks provide a unified view of the current design directions and practical trade-offs that are shaping the next generation of ultra-low-power and in-sensor AI processors.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.AI
RoboLayout: Differentiable 3D Scene Generation for Embodied Agents
Ali Shamsaddinlou
Recent advances in
vision language models (VLMs) have shown strong potential for spatial
reasoning and 3D scene layout generation from open-ended
language instructions. However, generating layouts that are not only semantically coherent but also feasible for interaction by
embodied agents remains challenging, particularly in physically constrained indoor environments. In this paper, RoboLayout is introduced as an extension of LayoutVLM that augments the original framework with
agent-aware
reasoning and improved optimization stability. RoboLayout integrates explicit reachability constraints into a differentiable layout optimization process, enabling the generation of layouts that are navigable and actionable by
embodied agents. Importantly, the
agent abstraction is not limited to a specific
robot platform and can represent diverse entities with distinct physical capabilities, such as service
robots, warehouse
robots, humans of different age groups, or animals, allowing environment design to be tailored to the intended
agent. In addition, a local refinement stage is proposed that selectively reoptimizes problematic object placements while keeping the remainder of the scene fixed, improving convergence efficiency without increasing global optimization iterations. Overall, RoboLayout preserves the strong semantic
alignment and physical plausibility of LayoutVLM while enhancing applicability to
agent-centric indoor scene generation, as demonstrated by experimental results across diverse scene configurations.
Read the full abstract ▾
Computer Vision
Depth from Defocus via Direct Optimization
Holly Jackson, Caleb Adams, Ignacio Lopez-Francos et al. (+1 more)
Though there exists a reasonable forward model for blur based on optical physics, recovering depth from a collection of defocused images remains a computationally challenging optimization problem. In this paper, we show that with contemporary optimization methods and reasonable computing resources, a global optimization approach to depth from defocus is feasible. Our approach rests on alternating minimization. When holding the depth map fixed, the forward model is linear with respect to the all-in-focus image. When holding the all-in-focus image fixed, the depth at each pixel can be computed independently, enabling embarrassingly parallel computation. We show that alternating between convex optimization and parallel grid search can effectively solve the depth-from-defocus problem at higher resolutions than current deep learning methods. We demonstrate our approach on
benchmark datasets with synthetic and real defocus blur and show promising results compared to prior approaches. Our code is available at github.com/hollyjackson/dfd.
Read the full abstract ▾
All authors Holly Jackson, Caleb Adams, Ignacio Lopez-Francos, Benjamin Recht
Computer Vision
Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning
Yurim Jang, Jaeung Lee, Dohyun Kim et al. (+2 more)
As pretrained models are increasingly shared on the web, ensuring that models can forget or delete sensitive, copyrighted, or private information upon request has become crucial. Machine unlearning has been proposed to address this challenge. However, current evaluations for unlearning methods rely on output-based metrics, which cannot verify whether information is completely deleted or merely suppressed at the representation level, where suppression is insufficient for true unlearning. To address this gap, we propose a novel restoration-based analysis framework that uses Sparse Autoencoders to identify class-specific expert features in intermediate layers and applies inference-time steering to quantitatively distinguish between suppression and deletion. Applying our framework to 12 major unlearning methods in image classification tasks, we find that most methods achieve high restoration rates of unlearned information, indicating that they only suppress information at the decision-boundary level, while preserving semantic features in intermediate representations. Notably, even retraining from pretrained checkpoints shows high restoration, revealing that robust semantic features inherited from pretraining are not removed by retraining. These results demonstrate that representation-level retention poses significant risks overlooked by output-based metrics, highlighting the need for new unlearning evaluation criteria. We propose new evaluation guidelines that prioritize representation-level verification, especially for
privacy-critical applications in the era of pre-trained models.
Read the full abstract ▾
All authors Yurim Jang, Jaeung Lee, Dohyun Kim, Jaemin Jo, Simon S. Woo
Computer Vision
HS-3D-NeRF: 3D Surface and Hyperspectral Reconstruction From Stationary Hyperspectral Images Using Multi-Channel NeRFs
Kibon Ku, Talukder Z. Jubery, Adarsh Krishnamurthy et al. (+1 more)
Advances in hyperspectral imaging (HSI) and 3D reconstruction have enabled accurate, high-throughput characterization of agricultural produce quality and plant phenotypes, both essential for advancing agricultural sustainability and breeding programs. HSI captures detailed biochemical features of produce, while 3D
geometric data substantially improves morphological analysis. However, integrating these two modalities at scale remains challenging, as conventional approaches involve complex hardware setups incompatible with automated phenotyping systems. Recent advances in neural radiance fields (NeRF) offer computationally efficient 3D reconstruction but typically require moving-camera setups, limiting throughput and reproducibility in standard indoor agricultural environments. To address these challenges, we introduce HSI-SC-NeRF, a stationary-camera multi-channel NeRF framework for high-throughput hyperspectral 3D reconstruction targeting postharvest inspection of agricultural produce. Multi-view hyperspectral data is captured using a stationary camera while the object rotates within a custom-built Teflon imaging chamber providing diffuse, uniform illumination. Object poses are estimated via ArUco calibration markers and transformed to the camera frame of reference through simulated pose transformations, enabling standard NeRF training on stationary-camera data. A multi-channel NeRF formulation optimizes reconstruction across all hyperspectral bands jointly using a composite spectral loss, supported by a two-stage training protocol that decouples
geometric initialization from radiometric refinement. Experiments on three agricultural produce samples demonstrate high spatial reconstruction accuracy and strong spectral fidelity across the visible and near-infrared spectrum, confirming the suitability of HSI-SC-NeRF for integration into automated agricultural workflows.
Read the full abstract ▾
All authors Kibon Ku, Talukder Z. Jubery, Adarsh Krishnamurthy, Baskar Ganapathysubramanian
Computer Vision
Xray-Visual Models: Scaling Vision models on Industry Scale Data
Shlok Mishra, Tsung-Yu Lin, Linda Wang et al. (+24 more)
We present Xray-Visual, a unified
vision model architecture for large-scale image and
video understanding trained on industry-scale social media data. Our model leverages over 15 billion curated image-text pairs and 10 billion
video-hashtag pairs from Facebook and Instagram, employing robust data curation pipelines that incorporate balancing and noise suppression strategies to maximize semantic diversity while minimizing label noise. We introduce a three-stage training pipeline that combines self-supervised MAE, semi-supervised hashtag classification, and CLIP-style contrastive learning to jointly optimize image and
video modalities. Our architecture builds on a
Vision Transformer backbone enhanced with efficient
token reorganization (EViT) for improved computational efficiency. Extensive experiments demonstrate that Xray-Visual achieves state-of-the-art performance across diverse
benchmarks, including ImageNet for image classification, Kinetics and HMDB51 for
video understanding, and MSCOCO for cross-modal
retrieval. The model exhibits strong robustness to domain shift and
adversarial perturbations. We further demonstrate that integrating large
language models as text encoders (LLM2CLIP) significantly enhances
retrieval performance and generalization capabilities, particularly in real-world environments. Xray-Visual establishes new
benchmarks for scalable,
multimodal vision models, while maintaining superior accuracy and computational efficiency.
Read the full abstract ▾
All authors Shlok Mishra, Tsung-Yu Lin, Linda Wang, Hongli Xu, Yimin Liu, Michael Hsu, Chaitanya Ahuja, Hao Yuan, Jianpeng Cheng, Hong-You Chen, Haoyuan Xu, Chao Li, Sreya Dutta Roy, Abhijeet Awasthi, Jihye Moon, Don Husa, Michael Ge, Sumedha Singla, Arkabandhu Chowdhury, Phong Dingh, Satya Narayan Shukla, Yonghuan Yang, David Jacobs, Qi Guo, Jun Xiao, Xiangjun Fan, Aashu Singh
Computer Vision
Coverage-aware Semantic Representation Learning for Underrepresented Visual Concepts
Sakib Ahammed, Xia Cui, Wenqi Lu et al. (+4 more)
Modern
vision models increasingly rely on rich semantic representations that extend beyond class labels to include descriptive concepts, attributes, and contextual cues. However, semantic concepts are not uniformly represented across classes: a concept may be frequent globally yet remain underrepresented within a class, resulting in low class-concept coverage. We formalize this class-concept phenomenon as Semantic Coverage Imbalance (SCI) and study its relationship with learned semantic representations. We find that lower class-concept coverage is consistently associated with weaker semantic representations. To address this, we introduce Coverage-aware Semantic Representation Learning, which shares more semantic structure for low coverage or uncertain relations while preserving more pair-specific structure for high coverage relations. We instantiate this principle in SemCovNet with Coverage-Calibrated Semantic Sharing (CCSS), a coverage- and uncertainty-aware partial-pooling mechanism. Using a range of
vision datasets spanning facial attributes and fine-grained recognition, as well as real-world medical
datasets, we show that SCI is widespread, degrades representation quality, and can be mitigated through coverage-aware semantic representation learning.
Read the full abstract ▾
All authors Sakib Ahammed, Xia Cui, Wenqi Lu, Xinqi Fan, Bill Cassidy, Xueli Chen, Moi Hoon Yap
Computer Vision
StereoAdapter-2: Globally Structure-Consistent Underwater Stereo Depth Estimation
Zeyu Ren, Xiang Li, Yiran Wang et al. (+2 more)
Stereo depth estimation is fundamental to
underwater robotic perception, yet suffers from severe domain shifts caused by wavelength-dependent light attenuation, scattering, and refraction. Recent approaches leverage
monocular foundation models with GRU-based iterative refinement for
underwater adaptation; however, the sequential gating and local convolutional kernels in GRUs necessitate multiple iterations for long-range disparity propagation, limiting performance in large-disparity and textureless
underwater regions. In this paper, we propose StereoAdapter-2, which replaces the conventional ConvGRU updater with a novel ConvSS2D operator based on selective state space models. The proposed operator employs a four-directional scanning strategy that naturally aligns with epipolar geometry while capturing vertical structural consistency, enabling efficient long-range spatial propagation within a single update step at linear computational complexity. Furthermore, we construct UW-StereoDepth-80K, a large-scale synthetic
underwater stereo
dataset featuring diverse baselines, attenuation coefficients, and scattering parameters through a two-stage generative pipeline combining semantic-aware style transfer and geometry-consistent novel view synthesis. Combined with dynamic LoRA adaptation inherited from StereoAdapter, our framework achieves state-of-the-art zero-shot performance on
underwater benchmarks with 17% improvement on TartanAir-UW and 7.2% improvment on SQUID, with real-world validation on the BlueROV2 platform demonstrates the robustness of our approach. Code: https://github.com/AIGeeksGroup/StereoAdapter-2. Website: https://aigeeksgroup.github.io/StereoAdapter-2.
Read the full abstract ▾
All authors Zeyu Ren, Xiang Li, Yiran Wang, Zeyu Zhang, Hao Tang
Computer Vision · Cross-listed from cs.RO
MALLVI: A Multi-Agent Framework for Integrated Generalized Robotics Manipulation
Mehrshad Taji, Arad Mahdinezhad Kashani, Iman Ahmadi et al. (+3 more)
Task planning for robotic manipulation with large
language models (LLMs) is an emerging area. Prior approaches rely on specialized models, fine tuning, or
prompt tuning, and often operate in an open loop manner without robust environmental feedback, making them fragile in dynamic settings. MALLVI presents a Multi
Agent Large
Language and
Vision framework that enables closed-loop feedback driven robotic manipulation. Given a natural
language instruction and an image of the environment, MALLVI generates executable atomic actions for a
robot manipulator. After action execution, a
Vision Language Model (VLM) evaluates environmental feedback and decides whether to repeat the process or proceed to the next step. Rather than using a single model, MALLVI coordinates specialized
agents, Decomposer, Localizer, Thinker, and Reflector, to manage perception, localization,
reasoning, and high level planning. An optional Descriptor
agent provides visual
memory of the initial state. The Reflector supports targeted error detection and recovery by reactivating only relevant
agents, avoiding full replanning. Experiments in simulation and real-world settings show that iterative closed loop multi
agent coordination improves generalization and increases success rates in zero shot manipulation tasks. Code available at https://github.com/iman1234ahmadi/MALLVI .
Read the full abstract ▾
All authors Mehrshad Taji, Arad Mahdinezhad Kashani, Iman Ahmadi, AmirHossein Jadidi, Saina Kashani, Babak Khalaj
Computer Vision
DODO: Discrete OCR Diffusion Models
Sean Man, Gilad Deutch, Roy Ganz et al. (+4 more)
Optical Character Recognition (OCR) is a fundamental task for digitizing information, serving as a critical bridge between visual data and textual understanding. While modern
Vision-
Language Models (VLM) have achieved high accuracy in this domain, they predominantly rely on autoregressive decoding, which becomes computationally expensive and slow for long documents as it requires a sequential forward pass for every generated
token. We identify a key opportunity to overcome this bottleneck: unlike open-ended generation, OCR is a highly deterministic task where the visual input strictly dictates a unique output sequence, theoretically enabling efficient, parallel decoding via
diffusion models. However, we show that existing masked
diffusion models fail to harness this potential; those introduce structural instabilities that are benign in flexible tasks, like captioning, but catastrophic for the rigid, exact-match requirements of OCR. To bridge this gap, we introduce DODO, the first VLM to utilize block discrete
diffusion and unlock its speedup potential for OCR. By decomposing generation into blocks, DODO mitigates the synchronization errors of global
diffusion. Empirically, our method achieves near state-of-the-art accuracy while enabling up to 5x faster inference compared to autoregressive baselines.
Read the full abstract ▾
All authors Sean Man, Gilad Deutch, Roy Ganz, Roi Ronen, Shahar Tsiper, Shai Mazor, Niv Nayman
Computer Vision
Analytic Score Optimization for Multi Dimension Video Quality Assessment
Boda Lin, Yongjie Zhu, Wenyu Qin et al. (+2 more)
Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of
video content. In this paper, we present a large-scale multi-dimensional VQA
dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each
video in our
dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring
alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based
alignment in advancing
video quality assessment.
Read the full abstract ▾
All authors Boda Lin, Yongjie Zhu, Wenyu Qin, Meng Wang, Pengfei Wan
Computer Vision
TeCoNeRV: Leveraging Temporal Coherence for Compressible Neural Representations for Videos
Namitha Padmanabhan, Matthew Gwilliam and Abhinav Shrivastava
Implicit Neural Representations (INRs) have recently demonstrated impressive performance for
video compression. However, since a separate INR must be overfit for each
video, scaling to high-resolution
videos while maintaining encoding efficiency remains a significant challenge. Hypernetwork-based approaches predict INR weights (hyponetworks) for unseen
videos at high speeds, but with low quality, large compressed size, and prohibitive
memory needs at higher resolutions. We address these fundamental limitations through three key contributions: (1) an approach that decomposes the weight prediction task spatially and temporally, by breaking short
video segments into patch tubelets, to reduce pretraining
memory overhead across resolutions; (2) a residual-based storage scheme that captures only differences between consecutive segment representations, significantly reducing bitstream size; and (3) a temporal coherence regularization framework that encourages changes in the weight space to be correlated with changes in
video, making residuals even smaller. Our proposed method, TeCoNeRV, achieves substantial improvements of 2.47dB and 4.94dB PSNR over the baseline at 480p and 720p on UVG, with 36% lower bitrates and 1.5-3${\times}$ faster encoding speeds. With our low
memory usage, we are the first hypernetwork approach to demonstrate results at $480$p, $720$p and $1080$p on UVG, HEVC and MCL-JCV. Our project page is available at https://namithap10.github.io/teconerv/.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
HERO: Learning Humanoid End-Effector Control for Visual Whole-Body Open-Vocabulary Object Grasping
Runpei Dong, Ziyan Li, Arjun Gupta et al. (+2 more)
Visual loco-manipulation of arbitrary in-the-wild objects requires accurate end-effector (EE) control and a generalizable understanding of the scene from visual inputs (eg, RGB-D images). Existing imitation and sim2real methods jointly learn both these aspects via monolithic end-to-end learning and are thus hard to scale. In this work, we bring to bear the best tools for each of these problems – large
vision models for generalizable scene understanding and simulated training for accurate EE control – leading to an overall modular loco-manipulation system that exhibits strong generalization. Our core technical innovation is HERO, an accurate residual-aware EE tracking policy made possible by combining classical robotics with machine learning. It uses a) inverse kinematics to convert residual end-effector targets into reference trajectories, b) a learned neural forward model for accurate forward kinematics, and c) goal adjustment and replanning. Together, these innovations reduce the end-effector tracking error to 2.44cm, outperforming the strongest prior method by 5.5x. Our overall system operates in diverse real-world environments, from offices to coffee shops, where the
robot reliably grasps various everyday objects (eg, mugs, apples, toys) on surfaces ranging from 43cm to 92cm in height. Systematic modular and end-to-end tests demonstrate the effectiveness of our proposed design. We believe our advances open up new ways of training
humanoids to interact with daily objects. Project page: https://hero-
humanoid.github.io/
Read the full abstract ▾
All authors Runpei Dong, Ziyan Li, Arjun Gupta, Xialin He, Saurabh Gupta
Computer Vision
Saliency-Aware Multi-Route Thinking: Revisiting Vision-Language Reasoning
Mingjia Shi, Yinhan He, Yaochen Zhu et al. (+1 more)
Vision-
language models (VLMs) aim to reason by jointly leveraging visual and textual modalities. While allocating additional inference-time computation has proven effective for large
language models (LLMs), achieving similar scaling in VLMs remains challenging. A key obstacle is that visual inputs are typically provided only once at the start of generation, while textual
reasoning (e.g., early visual summaries) is generated autoregressively, causing
reasoning to become increasingly text-dominated and allowing early visual grounding errors to accumulate. Moreover, vanilla guidance for visual grounding during inference is often coarse and noisy, making it difficult to steer
reasoning over long texts. To address these challenges, we propose Saliency-Aware Principle (SAP) selection. SAP operates on high-level
reasoning principles rather than
token-level trajectories, which enable stable control over discrete generation under noisy feedback while allowing later
reasoning steps to re-consult visual evidence when renewed grounding is required. In addition, SAP supports multi-route inference, enabling parallel exploration of diverse
reasoning behaviors. SAP is model-agnostic and data-free, requiring no additional training. Empirical results show that SAP achieves competitive performance, especially in reducing object
hallucination, under comparable
token-generation budgets while yielding more stable
reasoning and lower response latency than CoT-style long sequential
reasoning.
Read the full abstract ▾
All authors Mingjia Shi, Yinhan He, Yaochen Zhu, Jundong Li
Computer Vision
Are Object-Centric Representations Better At Compositional Generalization?
Ferdinand Kapl, Amir Mohammad Karimi Mamaghan, Maximilian Seitzer et al. (+4 more)
Compositional generalization, the ability to reason about novel combinations of familiar concepts, is fundamental to human cognition and a critical challenge for machine learning. Object-centric (OC) representations, which encode a scene as a set of objects, are often argued to support such generalization, but systematic evidence in visually rich settings is limited. We introduce a Visual Question Answering
benchmark across three controlled visual worlds (CLEVRTex, Super-CLEVR, and MOVi-C) to measure how well
vision encoders, with and without object-centric biases, generalize to unseen combinations of object properties. To ensure a fair and comprehensive comparison, we carefully account for training data diversity, sample size, representation size, downstream model capacity, and compute. We use DINOv2 and SigLIP2, two widely used
vision encoders, as the foundation models and their OC counterparts. Our key findings reveal that (1) OC approaches are superior in harder compositional generalization settings; (2) original dense representations surpass OC only on easier settings and typically require substantially more downstream compute; and (3) OC models are more sample efficient, achieving stronger generalization with fewer images, whereas dense encoders catch up or surpass them only with sufficient data and diversity. Overall, object-centric representations offer stronger compositional generalization when any one of
dataset size, training data diversity, or downstream compute is constrained.
Read the full abstract ▾
All authors Ferdinand Kapl, Amir Mohammad Karimi Mamaghan, Maximilian Seitzer, Karl Henrik Johansson, Carsten Marr, Stefan Bauer, Andrea Dittadi
Computer Vision
SAW-Bench: Learning Situated Awareness in the Real World
Chuhan Li, Rilyn Han, Joy Hsu et al. (+5 more)
A core aspect of human perception is situated awareness, the ability to relate ourselves to the surrounding physical environment and reason over possible actions in context. However, most existing
benchmarks for
multimodal foundation models (MFMs) emphasize environment-centric spatial relations (relations among objects in a scene), while largely overlooking observer-centric relationships that require
reasoning relative to
agent's viewpoint, pose, and motion. To bridge this gap, we introduce SAW-Bench (Situated Awareness in the Real World), a novel
benchmark for evaluating
egocentric situated awareness using real-world
videos. SAW-Bench comprises 786 self-recorded
videos captured with Ray-Ban Meta (Gen 2) smart glasses spanning diverse indoor and outdoor environments, and over 2,071 human-annotated question-answer pairs. It probes a model's observer-centric understanding with six different awareness tasks. Our comprehensive evaluation reveals a human-model performance gap of 37.66%, even with the best-performing MFM, Gemini 3 Flash. Beyond this gap, our in-depth analysis uncovers several notable findings; for example, while models can exploit partial
geometric cues in
egocentric videos, they often fail to infer a coherent camera geometry, leading to systematic spatial
reasoning errors. We position SAW-Bench as a
benchmark for situated spatial intelligence, moving beyond passive observation to understanding physically grounded, observer-centric dynamics.
Read the full abstract ▾
All authors Chuhan Li, Rilyn Han, Joy Hsu, Yongyuan Liang, Rajiv Dhawan, Jiajun Wu, Ming-Hsuan Yang, Xin Eric Wang
Computer Vision
VETime: Vision Enhanced Zero-Shot Time Series Anomaly Detection
Yingyuan Yang, Tian Lan, Yifei Gao et al. (+6 more)
Time-series anomaly detection (TSAD) requires identifying both immediate Point Anomalies and long-range Context Anomalies. However, existing foundation models face a fundamental trade-off: 1D temporal models provide fine-grained pointwise localization but lack a global contextual perspective, while 2D
vision-based models capture global patterns but suffer from information bottlenecks due to a lack of temporal
alignment and coarse-grained pointwise detection. To resolve this dilemma, we propose VETime, the first TSAD framework that unifies temporal and visual modalities through fine-grained visual-temporal
alignment and dynamic fusion. VETime introduces a Reversible Image Conversion and a Patch-Level Temporal
Alignment module to establish a shared visual-temporal timeline, preserving discriminative details while maintaining temporal sensitivity. Furthermore, we design an Anomaly Window Contrastive Learning mechanism and a Task-Adaptive Multi-Modal Fusion to adaptively integrate the complementary perceptual strengths of both modalities. Extensive experiments demonstrate that VETime significantly outperforms state-of-the-art models in zero-shot scenarios, achieving superior localization precision with lower computational overhead than current
vision-based approaches. Code available at: https://github.com/yyyangcoder/VETime .
Read the full abstract ▾
All authors Yingyuan Yang, Tian Lan, Yifei Gao, Yimeng Lu, Xuming An, Wenjun He, Meng Wang, Chenghao Liu, Chen Zhang
Computer Vision
PredMapNet: Future and Historical Reasoning for Consistent Online HD Vectorized Map Construction
Bo Lang, Nirav Savaliya, Zhihao Zheng et al. (+3 more)
High-definition (HD) maps are crucial to autonomous driving, providing structured representations of road elements to support
navigation and planning. However, existing query-based methods often employ random query initialization and depend on implicit temporal modeling, which lead to temporal inconsistencies and instabilities during the construction of a global map. To overcome these challenges, we introduce a novel end-to-end framework for consistent online HD vectorized map construction, which jointly performs map instance tracking and short-term prediction. First, we propose a Semantic-Aware Query Generator that initializes queries with spatially aligned semantic masks to capture scene-level context globally. Next, we design a History Rasterized Map
Memory to store fine-grained instance-level maps for each tracked instance, enabling explicit historical priors. A History-Map Guidance Module then integrates rasterized map information into track queries, improving temporal continuity. Finally, we propose a Short-Term Future Guidance module to forecast the immediate motion of map instances based on the stored history trajectories. These predicted future locations serve as hints for tracked instances to further avoid implausible predictions and keep temporal consistency. Extensive experiments on the nuScenes and Argoverse2
datasets demonstrate that our proposed method outperforms state-of-the-art (SOTA) methods with good efficiency.
Read the full abstract ▾
All authors Bo Lang, Nirav Savaliya, Zhihao Zheng, Jinglun Feng, Zheng-Hang Yeh, Mooi Choo Chuah
Computer Vision
Unpaired Image-to-Image Translation via a Self-Supervised Semantic Bridge
Jiaming Liu, Felix Petersen, Yunhe Gao et al. (+6 more)
Adversarial diffusion and
diffusion-inversion methods have advanced unpaired image-to-image translation, but each faces key limitations.
Adversarial approaches require target-domain
adversarial loss during training, which can limit generalization to unseen data, while
diffusion-inversion methods often produce low-fidelity translations due to imperfect inversion into noise-latent representations. In this work, we propose the Self-Supervised Semantic Bridge (SSB), a versatile framework that integrates external semantic priors into
diffusion bridge models to enable spatially faithful translation without cross-domain supervision. Our key idea is to leverage self-supervised visual encoders to learn representations that are invariant to appearance changes but capture
geometric structure, forming a shared latent space that conditions the
diffusion bridges. Extensive experiments show that SSB outperforms strong prior methods for challenging medical image synthesis in both in-domain and out-of-domain settings, and extends easily to high-quality text-guided editing.
Read the full abstract ▾
All authors Jiaming Liu, Felix Petersen, Yunhe Gao, Yabin Zhang, Hyojin Kim, Akshay S. Chaudhari, Yu Sun, Stefano Ermon, Sergios Gatidis
Computer Vision · Cross-listed from cs.GR
Style-Aware Gloss Control for Generative Non-Photorealistic Rendering
Santiago Jimenez-Navarro, Belen Masia and Ana Serrano
Humans can infer material characteristics of objects from their visual appearance, and this ability extends to artistic depictions, where similar perceptual strategies guide the interpretation of paintings or drawings. Among the factors that define material appearance, gloss, along with color, is widely regarded as one of the most important, and recent studies indicate that humans can perceive gloss independently of the artistic style used to depict an object. To investigate how gloss and artistic style are represented in learned models, we train an unsupervised generative model on a newly curated
dataset of painterly objects designed to systematically vary such factors. Our analysis reveals a hierarchical latent space in which gloss is disentangled from other appearance factors, allowing for a detailed study of how gloss is represented and varies across artistic styles. Building on this representation, we introduce a lightweight adapter that connects our style- and gloss-aware latent space to a latent-
diffusion model, enabling the synthesis of non-photorealistic images with fine-grained control of these factors. We compare our approach with previous models and observe improved disentanglement and controllability of the learned factors.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models
Melkamu Abay Mersha and Jugal Kalita
Transformer models achieve state-of-the-art performance across domains and tasks, yet their deeply layered representations make their predictions difficult to interpret. Existing explainability methods rely on final-layer attributions, capture either local
token-level attributions or global
attention patterns without unification, and lack context-awareness of inter-
token dependencies and structural components. They also fail to capture how relevance evolves across layers and how structural components shape decision-making. To address these limitations, we proposed the Context-Aware Layer-wise Integrated Gradients (CA-LIG) Framework, a unified hierarchical attribution framework that computes layer-wise Integrated Gradients within each
Transformer block and fuses these
token-level attributions with class-specific
attention gradients. This integration yields signed, context-sensitive attribution maps that capture supportive and opposing evidence while tracing the hierarchical flow of relevance through the
Transformer layers. We evaluate the CA-LIG Framework across diverse tasks, domains, and
transformer model families, including sentiment analysis and long and multi-class document classification with BERT, hate
speech detection in a low-resource
language setting with XLM-R and AfroLM, and image classification with Masked Autoencoder
vision Transformer model. Across all tasks and architectures, CA-LIG provides more faithful attributions, shows stronger sensitivity to contextual dependencies, and produces clearer, more semantically coherent visualizations than established explainability methods. These results indicate that CA-LIG provides a more comprehensive, context-aware, and reliable explanation of
Transformer decision-making, advancing both the practical interpretability and conceptual understanding of deep neural models.
Read the full abstract ▾
Computer Vision
A Contrastive Learning Framework Empowered by Attention-based Feature Adaptation for Street-View Image Classification
Qi You, Yitai Cheng, Zichao Zeng et al. (+1 more)
Street-view image attribute classification is a vital downstream task of image classification, enabling applications such as autonomous driving, urban analytics, and high-definition map construction. It remains computationally demanding whether training from scratch, initialising from pre-trained weights, or fine-tuning large models. Although pre-trained
vision-
language models such as CLIP offer rich image representations, existing adaptation or fine-tuning methods often rely on their global image
embeddings, limiting their ability to capture fine-grained, localised attributes essential in complex, cluttered street scenes. To address this, we propose CLIP-MHAdapter, a variant of the current lightweight CLIP adaptation paradigm that appends a bottleneck MLP equipped with multi-head self-
attention operating on patch
tokens to model inter-patch dependencies. With approximately 1.4 million trainable parameters, CLIP-MHAdapter achieves superior or competitive accuracy across eight attribute classification tasks on the Global StreetScapes
dataset, attaining new state-of-the-art results while maintaining low computational cost. The code is available at https://github.com/SpaceTimeLab/CLIP-MHAdapter.
Read the full abstract ▾
All authors Qi You, Yitai Cheng, Zichao Zeng, James Haworth