Computer Vision · Cross-listed from cs.IR
RE-TRIANGLE: Does TRIANGLE Enable Multimodal Alignment Beyond Cosine Similarity in Retrieval?
Arijit Ghosh, Aritra Bandyopadhyay, Chiranjeev Bindra et al. (+1 more)
Multimodal alignment is critical for bridging the semantic gap in information
retrieval. However, traditional pairwise strategies introduce a
geometric blind spot: while they align anchor modalities (e.g., text) with others, they lack constraints to enforce mutual consistency between peripheral modalities (e.g.,
video and audio). The TRIANGLE framework addresses this by minimizing the area of modality triplets on a hypersphere to enforce holistic
alignment. In this reproducibility study, we verify the robustness of this
geometric objective for
retrieval tasks. We confirm that TRIANGLE outperforms pairwise baselines in zero-shot settings, achieving Recall@1 gains of up to +8.7 points, though benefits are domain-dependent. However, we fail to reproduce the reported learning-from-scratch results. Analysis using a synthetic toy
dataset attributes this to instability when jointly optimizing
geometric alignment with Data-Text Matching (DTM) loss. Furthermore, we find that cosine regularization primarily stabilizes text-to-
video retrieval, and fine-tuning with domain supervision amplifies
geometric benefits but reduces cross-
dataset generalization. Our findings support the efficacy of
geometric alignment while highlighting critical optimization sensitivities. Code available at https://github.com/ARIJIT00171/RE-TRIANGLE.
Read the full abstract ▾
All authors Arijit Ghosh, Aritra Bandyopadhyay, Chiranjeev Bindra, Jingfen Qiao
Computer Vision · Cross-listed from cs.SE
VISTA: An End-to-End Benchmark for Visual Spec-to-Web-App Coding Agents
JunJia Guo, Yuhang Yao, Chang Li et al. (+3 more)
Coding
agents can now build applications from design mockups, but a screen that looks right is not an application that works. Existing
benchmarks either score visual reconstruction without a backend or test full-stack functionality from textual requirements, and few tie scores to the individual components a design requires. We introduce VISTA (VIsual Spec-To-App), an end-to-end
benchmark in which coding
agents turn multi-page design handoffs (Figma renders, structure, and textual requirements) into runnable full-stack Web and Mobile (Android) applications. VISTA provides 8,487 human-annotated interactive components across 18 applications (126 Web pages and 576 Android screens) and an executable evaluator that locates annotated components in the running application and probes their interactions, yielding localization, behavior, and joint scores traceable to individual design requirements. Across 14 deployed coding-
agent systems, the best joint score is 0.553 on Web and 0.378 on Mobile, and 13 of 14 systems score lower on behavior than on localization on Web. Component probes detect unresponsive controls in 90.1% of audited Web deliveries. Development traces show where the process departs from the task: 44.5% of trajectories never open the design screenshots, 73.0% omit the prescribed self-audit, and 8.3% report success after a failed check. VISTA makes the gap between rendering a design and delivering a working application measurable at the level of individual components. Code is available at \href{https://github.com/kaboider/VISTA_Bench}.
Read the full abstract ▾
All authors JunJia Guo, Yuhang Yao, Chang Li, Zimeng Pan, Jiawei Zhou, Jingdi Chen
Computer Vision · Cross-listed from cs.GR
AssetGen: Deployable 3D Asset Generation at Interactive Speed
Dilin Wang, Xiaoyu Xiang, Kihyuk Sohn et al. (+14 more)
While 3D generation is progressing rapidly, recent work has often focused on obtaining high-resolution assets, leaving user experience and deployability as afterthoughts. We present AssetGen, a 3D generator that focuses instead on these two aspects. Given one reference image, in 30 seconds it produces a high-quality mesh with baked normals, a color texture, and a controlled polygon budget suitable for real-time rendering, including mobile use cases. The AssetGen Flash variant further reduces latency to 14 seconds for interactive and agentic creation loops. Our model generates the object geometry with a coarse-to-refine VecSet framework, which implements mesh simplification, cleaning, and normal baking on the GPU, and a fast parallel UV unwrapping. It then generates textures in a multi-view fashion, followed by backprojection and 3D inpainting. Model
distillation, kernel optimization, and pipeline parallelization are co-designed to accelerate the system end-to-end. We introduce numerous automated and blind human evaluations and demonstrate competitive visual quality against leading commercial solutions in 30 seconds and preview-quality results in less than 15 seconds. The final result is a system that supports AI-assisted, deployable 3D content creation in interactive workflows.
Read the full abstract ▾
All authors Dilin Wang, Xiaoyu Xiang, Kihyuk Sohn, Tom Monnier, Yu-Ying Yeh, Thu Nguyen-Phuoc, Jiawen Zhang, Yuchen Fan, Antoine Toisoul, Hyunyoung Jung, Prithviraj Dhar, Michael Bunnell, Nikolaos Sarafianos, Chuhang Zou, Roman Shapovalov, Andrea Vedaldi, Rakesh Ranjan
Computer Vision
Plume Segmentation from MethaneSAT with Cross-Sensor Transfer Learning and Physics-Informed Postprocessing
Manuel Pérez-Carrasco, Maya Nasr, Zhan Zhang et al. (+16 more)
Automated detection and masking of individual methane plumes from satellite imagery is important for operational emission attribution and quantification. We present a machine learning framework for plume detection from MethaneSAT retrieved column-averaged dry-air mole fractions of methane. We address two core challenges: the scarcity of labeled MethaneSAT data and the need for inference reliability across diverse atmospheric and surface conditions. We first demonstrate that Mask R-CNN with a ResNet-50 backbone outperforms a U-Net semantic
segmentation baseline on both MethaneAIR (an airborne version of MethaneSAT) and MethaneSAT data, with pixel-level F1 score gains of 10.49 and 5.48 percentage points respectively. To address MethaneSAT data scarcity, we evaluate three cross-sensor transfer strategies leveraging MethaneAIR flights and synthetic plumes. Mask R-CNN with ResNet-50 fine-tuned from MethaneAIR pre-trained weights is the most effective strategy, achieving instance-level precision of 0.60 and recall of 0.98 at the baseline operating point. A physics-informed post-processing pipeline converts detections into two operationally distinct modes. The first is a high-sensitivity mode that applies morphological filtering and proximity-based merging for comprehensive emission screening, achieving precision of 0.71 and recall of 0.94. The second is a high-precision mode that additionally applies a distribution-based classifier for confident source attribution, achieving precision of 0.92 and recall of 0.70. Manual review of detections classified as false positives against our wavelet-based reference plume mask labels shows that some correspond to real methane enhancements... Our data and code used in this work are publicly available at: https://doi.org/10.7910/DVN/FR959H
Read the full abstract ▾
All authors Manuel Pérez-Carrasco, Maya Nasr, Zhan Zhang, Apisada Chulakadabba, Javier Roger, Raia Ottenheimer, Sébastien Roche, Maryann Sargent, Chris Chan Miller, Daniel Varon, Jack Warren, Luis Guanter, Kang Sun, Jonathan Franklin, Jia Chen, Cecilia Garraffo, Xiong Liu, Ritesh Gautam, Steven Wofsy
Computer Vision
CRISP – Clustering-Based Redundancy-Reduced Instance Sampling for Pathology Case Representation and Retrieval
Zahra Rahimi Afzal, Wataru Uegami, Saghir Alfasly et al. (+6 more)
Digital pathology archives increasingly contain multiple whole-slide images (WSIs) per case, capturing spatially distinct tumor regions and reflecting intrinsic morphological heterogeneity. However, most existing approaches rely on a single pathologist-selected slide, thereby discarding potentially informative evidence distributed across the remaining WSIs. To date, no autonomous framework has been proposed for comprehensive multi-WSI case processing. Here, we present an unsupervised framework for case-level analysis that integrates information from all available slides within a case. Rather than relying on a single designated slide, the proposed approach constructs case-level representations by selectively distilling informative patches across WSIs. We introduce Clustering-Based Redundancy-Reduced Instance Sampling for Pathology (CRISP), a two-stage framework that first reduces redundancy within individual WSIs and subsequently applies clustering-based sampling to select a compact yet representative set of patches for the entire case. The resulting patch set captures case-level heterogeneity while avoiding exhaustive processing of gigapixel images, and directly serves as a
retrieval index. Using two Mayo Clinic breast cancer
datasets for diagnosis and treatment planning, we demonstrate that CRISP consistently matches or surpasses the current standard practice of combined model and pathologist slide selection for patient/case search and
retrieval. By automating case-level processing and eliminating subjective WSI selection, CRISP potentially enables the exploitation of clinically relevant information distributed across multiple WSIs that is currently overlooked.
Read the full abstract ▾
All authors Zahra Rahimi Afzal, Wataru Uegami, Saghir Alfasly, Wenchao Han, Saba Yasir, Judy C. Boughey, Matthew P. Goetz, Krishna R. Kalari, H.R. Tizhoosh
Computer Vision · Cross-listed from cs.LG
Rethinking Continual Anomaly Detection on the Edge: Benchmarking Under Realistic Industrial Conditions
Chad Weatherly and Sen Lin
Continual anomaly detection (CAD) addresses the need for industrial inspection systems to adapt to evolving production conditions, yet existing methods share three critical gaps: unrealistic evaluation, no systematic comparison, and no consideration of edge deployment constraints. We introduce a unified
benchmark combining discrete-task evaluation on structural and logical anomalies, a novel continuous drift protocol, the first head-to-head comparison of all published CAD methods, and computational efficiency profiling on edge hardware. Our results reveal that existing CAD methods do not consistently outperform traditional approaches with simple experience replay. Thus motivated, we propose DINOSaur, a training-free method combining a frozen DINOv3 backbone with spatially-indexed coreset
memory and neighborhood-restricted anomaly scoring. DINOSaur achieves zero forgetting by construction, outperforms all evaluated methods across all five protocols, and runs at sub-100\,ms inference on an NVIDIA Jetson Orin Nano, with on-device adaptation to new tasks in under 30 seconds.
Read the full abstract ▾
Computer Vision
GIBLy: Improving 3D Semantic Segmentation through an Architecture-Agnostic Lightweight Geometric Inductive Bias Layer
Diogo Lavado, Alessandra Micheletti and Clàudia Soares
In 3D scene understanding, deep learning models rely on large models and extensive training to capture basic
geometric structures that are present in the 3D data. However, existing methods lack explicit mechanisms to incorporate
geometric information, such as learnable primitive shapes, often necessitating large models and more training data which in turn increases cost and can limit generalization. We introduce GIBLy, a lightweight
geometric inductive bias layer that integrates learnable
geometric priors into 3D
segmentation pipelines. GIBLy enhances existing architectures – whether MLP-based, convolution-based, or
transformer-based – by providing features aligned with simple
geometric shapes (and thus human-interpretable) that improve
segmentation performance with minimal computational overhead. We validate our approach across multiple 3D semantic
segmentation benchmarks, demonstrating consistent performance gains, including up to +11.5% mIoU on TS40K with PTV3, while adding only 58K extra parameters. Our results highlight the benefit of explicitly encoding
geometric structure to support accurate and efficient 3D scene understanding, with a lightweight add-on layer
Read the full abstract ▾
Computer Vision
Radiuma: A Unified Zero-Code Executable Graphical Workflow Generator for Reproducible and Shareable Medical Image Analysis and Machine Learning
Mohammad Salmanpour, Mehrdad Oveisi, Isaac Shiri et al. (+1 more)
Medical image computing software is essential for identifying imaging biomarkers that can support diagnosis, prognosis, treatment planning, and clinical research. However, the lack of standardized, user-friendly, and reproducible software environments has limited the broader adoption of advanced medical image analysis workflows. We present Radiuma, a freely available modular platform designed to support reliable and reproducible medical image analysis across multiple modalities and file formats. Radiuma integrates image reading, visualization, registration, fusion, processing,
segmentation, radiomics feature extraction, and machine learning modules for classification, regression, and clustering. Its modular design allows users to execute each component independently or connect modules through a visual workflow system, where the output of one step can be graphically passed to the next. This enables the creation of custom, executable, and reproducible multi-step pipelines without requiring extensive programming expertise. Results from each module can be inspected directly in the visualization window, providing immediate feedback on processing quality and workflow accuracy. Radiuma also supports saving and sharing customized workflows, promoting transparency, reusability, and consistency across collaborative studies. By combining flexibility, usability, and standardized analysis tools, Radiuma provides a practical environment for radiomics and machine learning research in clinical and translational settings. The platform is designed to be accessible to users with diverse expertise, including radiologists, physicists, clinicians, and data scientists.
Read the full abstract ▾
All authors Mohammad Salmanpour, Mehrdad Oveisi, Isaac Shiri, Arman Rahmim
Computer Vision
Single View Seafloor Recovery from Imaging Sonar via Differentiable Rendering
Sevan Brodjian, Michael Hobley and Pietro Perona
Sonar is often the only modality suitable for high-resolution imaging
underwater due to light attenuation and turbidity. Forward-looking imaging sonar provides measurements over range and horizontal angle but collapses vertical structure into a flat image, creating ambiguities that make 3D recovery challenging. A common use case for imaging sonar is
underwater terrain mapping (bathymetry), yet current methods require many views, expensive multi-sensor setups, or significant training data, which limits use and adaptability to new environments. We present a training-free method that recovers bathymetry from a single sonar image in under 30 seconds via differentiable rendering, conditioned on a known seafloor tilt. To our knowledge, this is the first differentiable rendering approach for single-view height recovery in sonar. Our method implements differentiable sonar ray tracing and optimizes an explicit height field to reproduce the target image. On synthetic
datasets, our approach outperforms a supervised CNN under distribution shift and remains close on rough terrain, while the CNN wins in-distribution. By modeling physically grounded priors of the sonar process, our method adapts across sensor configurations and environments without training data.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Filtered Posterior Mean Collections: A Unified Framework for Analytical Models of Diffusion Generalization
Matthew Niedoba, Berend Zwartsenberg and Frank Wood
The neural-network denoising functions which form the backbone of image
diffusion models are remarkably consistent in their generalization behaviour across a wide variety of network architectures and training procedure hyperparameters. A recent line of research has sought to model the outputs of these networks by aggregating posterior weighted averages of training
dataset patches. In this work, we consolidate these approaches into a unified model class which we call Filtered Posterior Mean Collections (FPMCs). We define this model class using query precision vectors, response weights, and source distributions, and illustrate that existing methods are recoverable with specific choices of these design axes. Investigating each axis in turn, we find that FPMC performance can be improved with soft relaxations of prior patch-based methods, and through augmentations of source distributions. Applying these findings to an existing FPMC, we demonstrate consistent sample improvement across three natural image
datasets.
Read the full abstract ▾
Computer Vision
Loki: Representation over Architecture for Diffusion-Based Portrait Animation
Pouyan Navard and Sernam Lim
Portrait animation transfers a driver clip's facial expression and head pose onto a single reference image while preserving the reference's identity. State-of-the-art
diffusion systems address this by stacking trained modules for expression, pose, and identity in turn, paying for it in trainable parameters, proprietary corpora, and residual
entanglement between the very axes the system is meant to control independently. This complexity compensates for an upstream choice – learning facial expression and head pose from RGB, a representation in which identity, pose, and expression are inseparable without being learned apart. Loki steps out of RGB on the conditioning path. Driver expression and head pose are encoded by a face model whose parameter axes are identity-orthogonal by construction, then rasterised into a spatial map that the
diffusion backbone consumes natively. Identity is routed separately through the
diffusion backbone's own pretrained features via lightweight key-value injection. Because the parametric representation factorises identity from expression and pose, cross ID reenactment reduces to a coefficient substitution at inference, requiring no cross ID training data. Loki requires ~43% fewer inference parameters than leading
diffusion baselines and trained on 1496x less
video samples. We define two metrics that directly measure whether the generated head pose trajectory and facial expression followed the driver's – the questions portrait animation actually asks; Loki leads or co-leads on both.
Read the full abstract ▾
Computer Vision
EchoVQA: Enabling Conversational Assistance for Point-of-Care Cardiac Ultrasound
Filippos Bellos, Yutong Li, Jessie N Dong et al. (+6 more)
Point-of-care transthoracic echocardiography (TTE) enables cardiac assessment in virtually any clinical setting, yet its diagnostic utility remains constrained by the expertise required for image acquisition and interpretation. Visual question answering (VQA) offers a promising paradigm for bridging this expertise gap through interactive clinical assistance, but existing echocardiography VQA
datasets are limited in scale, restricted to high-quality images, and only cover a few views. We introduce EchoVQA, the first large-scale VQA
dataset for echocardiography, comprising 14,299 images and 74,819 question-answer pairs. The
dataset integrates public sources (EchoNet-Dynamic, CAMUS) with our own point-of-care acquisitions from two handheld probes (Lumify, Clarius), spanning diverse views and including both high-quality and suboptimal images. Uniquely, EchoVQA includes acquisition guidance questions to help users optimize transducer positioning toward a diagnostic apical 4-chamber view for left ventricular ejection fraction estimation – a challenging task for novice operators in point-of-care settings. We further develop a parameter-efficient method based on
multimodal learnable
prompts achieving state-of-the-art performance on most
benchmarks, including EchoVQA, with significantly less trainable parameters than existing state-of-the-art approaches.
Read the full abstract ▾
All authors Filippos Bellos, Yutong Li, Jessie N Dong, Zaiyang Guo, Emily Mackay, Yayuan Li, Yannis Avrithis, Alison Pouch, Jason J. Corso
Computer Vision
ImPartial: Multi-channel Whole-Cell Segmentation using Partial Annotations
Gunjan Shrivastava and Saad Nadeem
Accurate
cell segmentation in pathology images typically requires dense pixel-wise annotations, which are costly and time-consuming to obtain. This challenge is especially important for emerging biological imaging modalities and multiplexed
datasets with variable channel configurations, where expert-labeled data are scarce. In this work, we introduce ImPartial, a deep learning framework designed to achieve state-of-the-art
segmentation performance in low-annotation regimes using sparse scribbles and limited supervision. ImPartial augments the
segmentation objective via self-supervised multi-channel quantized imputation. This approach leverages the observation that perfect pixel-wise reconstruction or denoising of the image is not needed for accurate
segmentation, and thus, introduces a self-supervised classification objective that better aligns with the overall
segmentation goal. We demonstrate that ImPartial achieves performance at par with fully supervised models while requiring substantially fewer annotations. Extensive experiments on
benchmark multiplexed cellular imaging and single-plex clinical brightfield immunohistochemistry
datasets show consistent improvements over strong baselines with only partial annotations. All
benchmark datasets and code are available via our Github: https://github.com/nadeemlab/ImPartial.
Read the full abstract ▾
Computer Vision
COSY: Compositional 3DGS Synthesis for Disentangled Human Head Editing
Florian Barthel, Shalini De Mello, Koki Nagano et al. (+3 more)
Recent 3D Gaussian Splatting (3DGS) GANs for human heads synthesize and render photorealistic 3D models in real-time and offer a vast variety in identity and appearance. However, controlling specific semantic attributes such as hair color or glasses remains challenging, as edits in the entangled latent space often induce unintended changes in identity or appearance. Although there are several methods that aim to disentangle the latent space post training by estimating directions that only modify certain features, these methods cannot guarantee complete disentanglement and often require pre-trained classifiers. In our approach, we propose a new generator architecture that synthesizes components, such as hair, skin, glasses, and torso, completely independently. This allows for changing the latent vector for one region while keeping the remaining parts fixed. Further, we achieve this separation using only sparse information such as the hair or skin color, eliminating the requirement of
segmentation masks or
geometric priors, often seen in prior work. To ensure matching shape and lighting conditions during editing, we allow minimal shared information via context
tokens between the independent generators. These
tokens even allow us to control the shape and light, without any prior annotation. Compared to existing works on GAN-based generation and editing, our method shows better disentanglement, more precise editing control, and competitive visual quality.
Read the full abstract ▾
All authors Florian Barthel, Shalini De Mello, Koki Nagano, Wieland Morgenstern, Anna Hilsmann, Peter Eisert
Computer Vision
D2-V2X: Depth-Driven Cooperative V2X Reasoning for Autonomous Driving
Kevin Richard, Alphin Varghese, Colin Pham et al. (+2 more)
Single-vehicle
Vision-
Language Models (VLMs) are fundamentally constrained by sensor occlusions. While Vehicle-to-Everything (V2X) systems mitigate this, current
benchmarks lack the cooperative
reasoning required for resolving ambiguities in complex environments. We introduce D2-V2X, a spatially-aware Question-Rationale-Answer (QRA)
benchmark featuring 8,500 triplets derived from
multimodal vehicle and infrastructure sensors. We additionally establish a baseline that aligns 3D
LiDAR features with the VLM's latent space. By enforcing natural
language Chain-of-Thought rationales prior to structured JSON outputs, our model is forced to explicitly articulate spatial relations. Our experiments demonstrate that grounding VLMs in cooperative
LiDAR achieves 24.4% recall in identifying occluded hazards compared to near-zero in zero-shot models and reduces spatial estimation error for visible objects by 77% compared to the zero-shot baseline. While the model achieves a functional decision-making F1-score of 53.5, we identify 3D-to-2D projection as a fundamental bottleneck in current VLM architectures, establishing a new baseline for future innovation. Data, code, and trained models available at https://github.com/KevinRichard1/D2-V2X
Read the full abstract ▾
All authors Kevin Richard, Alphin Varghese, Colin Pham, David Oh, Srijan Das
Computer Vision
WideDepth: Millimeter-Accurate Benchmark for Fisheye Depth Estimation
Ilia Indyk, Ignat Penshin, Ivan Sosin et al. (+3 more)
Fisheye cameras are increasingly adopted in robotics for near-field manipulation,
navigation, and immersive perception, yet indoor depth
benchmarks with accurate ground truth are still missing. To address this, we introduce WideDepth - the first indoor
dataset for fisheye depth estimation, featuring 101 scenes containing 5K high-resolution stereo pairs labeled with millimeter-level ground truth depth and disparity. Our
dataset also includes paired pinhole and fisheye samples across varying fields of view and baselines in both horizontal and vertical stereo setups. We further propose a method to adapt pinhole-trained stereo models to fisheye images and introduce a novel stereo fisheye image generation pipeline based on high-resolution
LiDAR scans. Leveraging these methods, we thoroughly evaluate state-of-the-art
monocular depth, stereo matching, and depth completion models on our
benchmark. Additionally, we provide 18K
LiDAR-derived sparse depth training samples, achieving up to a 62% performance boost on fisheye data when fine-tuning pinhole-based stereo models. In summary, the high precision and versatility of our
benchmark set a strong foundation for advancing research in fisheye depth estimation and robotics perception. Project page: https://ilyaind.github.io/WideDepth
Read the full abstract ▾
All authors Ilia Indyk, Ignat Penshin, Ivan Sosin, Maxim Monastyrny, Aleksei Valenkov, Ilya Makarov
Computer Vision
Distance-Aware Joint Spatio-Temporal Graph Contrastive Learning for Major Depressive Disorder Diagnosis
Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin et al. (+1 more)
Major depressive disorder (MDD) is a common neuropsychiatric condition whose accurate diagnosis from resting-state functional
magnetic resonance imaging (rs-fMRI) remains difficult. Dynamic functional connectivity (DFC) captures time-varying interactions among
brain regions and provides rich spatio-temporal information, yet current DFC-based methods face three limitations: sliding-window Pearson correlation yields noisy estimates sensitive to window length and motion artifacts; correlation-derived node features do not fully exploit frequency-domain properties of blood-oxygen-level-dependent (BOLD) signals; and most spatio-temporal graph models handle spatial structure and temporal dynamics in separate stages, restricting their ability to represent coupled
brain network
evolution. To overcome these issues, we reformulate DFC learning as joint spatio-temporal graph representation learning under a Hawkes-process-inspired temporal dependency prior and propose HWSTCL, a two-stage framework built on a reliability-refined joint spatio-temporal graph with a kernel-weighted pretraining objective. Within each temporal window, BOLD signals are encoded as spectral node descriptors and functional edges are refined by an exponential distance-decay prior that down-weights less reliable long-range connections. The joint graph is then formed by linking each region to itself across future windows through a Hawkes-inspired exponential kernel, allowing spatial and temporal information to be propagated together during message passing. A kernel-weighted contrastive objective further promotes temporal consistency for each region across windows while reducing redundant similarity between different regions. Experiments on a
benchmark rs-fMRI
dataset show that HWSTCL outperforms recent baselines and yields coherent spatio-temporal representations for MDD diagnosis.
Read the full abstract ▾
All authors Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin, Alan Wee-Chung Liew
Computer Vision
fMRI-Diffusion: Generating fMRI Time Series Via a Temporal Transformer Diffusion Model for Major Depressive Disorder Diagnosis
Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin et al. (+1 more)
Diagnosing Major Depressive Disorder (MDD) from functional
magnetic resonance imaging (fMRI) using functional connectivity (FC) analysis requires large amounts of labeled data that are scarce in clinical settings. Existing augmentation methods synthesize FC
matrices, which compress fMRI recordings into static pairwise summaries and discard temporal information. We propose fMRI-
Diffusion, a framework that synthesizes region-of-interest (ROI)-level fMRI time series rather than FC
matrices. A Temporal
Transformer serves as the denoising network within a denoising
diffusion probabilistic model, treating each time point as a
token to capture temporal dependencies through self-
attention. A supervised pretraining strategy initializes the
Transformer with task-relevant representations before
diffusion training, and FC
matrices are derived from the synthesized time series for classification. Experiments on the REST-meta-MDD
dataset show that augmenting training data with synthetic time series consistently improves diagnostic accuracy across ten classifiers, six parcellation atlases, and three acquisition sites. The method outperforms five recent FC-based synthesis approaches, with accuracy gains of up to 3.7 percentage points over the strongest baseline. Ablation studies confirm the contributions of both the
Transformer-based denoiser and the pretraining strategy. Distributional fidelity metrics remain below 0.06 across all conditions, indicating close agreement between real and synthetic distributions. These findings suggest that synthesizing fMRI time series before FC computation preserves temporal information lost in
matrix-level augmentation and provides a practical strategy for MDD diagnosis under limited data.
Read the full abstract ▾
All authors Muhammad Asif Hasan, Yanming Zhu, Xuefei Yin, Alan Wee-Chung Liew
Computer Vision
Geo-Align: Video Generation Alignment via Metric Geometry Reward
Zizun Li, Haoyu Guo, Runzhe Teng et al. (+2 more)
Camera-controlled
video generation has achieved remarkable progress in recent years. However, existing
video-to-
video re-rendering methods primarily rely on Supervised Fine-Tuning using synthetic
datasets. At present, there is an extreme scarcity of synchronized, multi-view real-world
video data. Consequently, the prevailing paradigm often exhibits limited generalization when processing out-of-distribution real-world
videos, with models struggling to accurately adhere to physical scales and camera trajectories. To bridge this gap, we propose Geo-Align, the first Reinforcement Learning framework specifically designed for camera-controlled
video re-rendering. Built upon a pretrained model, we optimize the model through a scale-aware perceptual reward mechanism. Specifically, we introduce a metric 3D
estimator to extract precise camera trajectories from generated
videos, explicitly penalizing deviations in rotation and translation. Furthermore, we meticulously designed a data pipeline strategy based on real-world conditioning
videos and target camera trajectories derived from synthetic data, eliminating the reliance on paired data. Extensive experiments demonstrate that Geo-Align consistently outperforms existing supervised learning baselines in both precise camera controllability and visual fidelity, indicating the effectiveness of our method.
Read the full abstract ▾
All authors Zizun Li, Haoyu Guo, Runzhe Teng, Chunhua Shen, Tong He
Computer Vision
PiD: Fast and High-Resolution Latent Decoding with Pixel Diffusion
Yifan Lu, Qi Wu, Jay Zhangjie Wu et al. (+4 more)
Most practical high-resolution text-to-image systems, including latent
diffusion and autoregressive models, perform generation in a compact latent space, and a decoder maps the generated latents back to pixels. Yet the latent-to-pixel decoder is reconstruction-oriented, optimized to invert the encoder rather than synthesize more details, and becomes increasingly costly at megapixel scale. This drawback calls for a more expressive and efficient decoding paradigm. Motivated by recent progress in scalable pixel-space
diffusion, we introduce PiD, a Pixel
diffusion Decoder that reformulates latent decoding as conditional pixel
diffusion, unifying decoding and upsampling into one generative module. By denoising directly in high-resolution pixel space, PiD synthesizes $4\times$ and even $8\times$ upscaled images with low latency. For latent conditioning, a lightweight sigma-aware adapter injects noise-corrupted latents into the pixel
diffusion backbone, enabling PiD to decode partially denoised latents and terminate the latent
diffusion process early. To further improve efficiency, we distill the model using DMD2, reducing inference to just 4 steps. PiD applies to both conventional VAE latents and semantic latents (e.g., SigLIP, DINOv2) used in recent RAE-based models. PiD decodes latents of $512 \times 512$ images into $2048 \times 2048$ pixels in under 1 second with 13 GB peak
memory on a consumer RTX 5090, and as fast as 210 ms on a GB200 GPU, about $6\times$ faster than cascaded
diffusion-based super-resolution pipelines with better visual fidelity.
Read the full abstract ▾
All authors Yifan Lu, Qi Wu, Jay Zhangjie Wu, Zian Wang, Huan Ling, Sanja Fidler, Xuanchi Ren
Computer Vision
ETCHR: Editing To Clarify and Harness Reasoning
Beichen Zhang, Yuhong Liu, Jinsong Li et al. (+3 more)
Multimodal Large
Language Models have advanced visual
reasoning, yet a purely textual chain of thought remains a bottleneck for questions that require fine-grained focus or view transformations. The ”think with images” paradigm narrows this gap, but existing approaches are either constrained by fixed predefined toolkits or produce noisy intermediate images from unified
multimodal methods. We pursue a third option: using a dedicated image editing model and decouple it with an understanding model. However, off-the-shelf image editors fail as
reasoning assistants with two complementary gaps: a
language-side gap, where editors trained as passive instruction-followers cannot map an abstract question to an appropriate visual transformation, and a generation-side gap, where edit correctness degrades as
reasoning depth grows. Guided by this analysis, we introduce ETCHR (Editing To Clarify and Harness
Reasoning), a question-conditioned,
reasoning-aware image editor decoupled from the downstream understanding model and trained with a two-stage recipe targeted at the two gaps:
Reasoning Imitation via supervised fine-tuning on edit trajectories, followed by
Reasoning Enhancement with VLM-derived rewards for edit correctness and downstream
reasoning accuracy. Since the editor is decoupled, ETCHR plugs into different open- and closed-source MLLMs in a training-free manner. Across five task families (fine-grained perception, chart understanding, logic
reasoning, jigsaw restoration, and 3D understanding), ETCHR raises average Pass@1 from 55.95 to 60.77 (+4.82) with Qwen3-VL-8B, from 65.08 to 70.55 (+5.47) with Gemini-3.1-Flash-Lite, and from 76.55 to 81.16 (+4.61) with the 1T-parameter MoE model Kimi K2.5.
Read the full abstract ▾
All authors Beichen Zhang, Yuhong Liu, Jinsong Li, Yuhang Zang, Jiaqi Wang, Dahua Lin
Computer Vision
From Activation to Specificity: Automating Counterfactual Testing of Visual Representations in the Human Brain
Yuval Golbari, Navve Wasserman, Matias Cosarinsky et al. (+5 more)
Identifying which
brain regions represent a visual concept in the human
brain is a central challenge in neuroscience. Existing approaches have localized coarse functional regions (e.g., faces, places) through activation maximization, identifying regions that activate strongly for a target concept relative to other concepts. Yet strong activation alone does not establish that a region represents the concept itself, as responses may instead be driven by correlated visual or semantic cues. We introduce BrainTRACE (Testing Representations through Counterfactual Evidence), an automated framework that combines generative and
brain models to synthesize controlled stimuli and validate neural representations through targeted counterfactual-specificity testing. Given a query specifying a concept of interest, our framework constructs targeted stimulus sets comprising concept images, counterfactual edits that remove the target concept while preserving other image content, and images with candidate correlated distractors. It then uses an image-to-fMRI encoding model to predict
brain responses and searches for representations that respond specifically to the target concept over correlated alternatives. BrainTRACE returns validated candidate representations and proposes follow-up fMRI experiments to further test or extend its discoveries. Our approach successfully recovers known functional localizations and identifies new candidate representations across dozens of concepts, validated on both predicted and measured fMRI data. Critically, we show that without counterfactual evaluation, a large fraction of localizations would be false positives, confirming that activation alone is insufficient evidence of representation.
Read the full abstract ▾
All authors Yuval Golbari, Navve Wasserman, Matias Cosarinsky, Roman Beliy, Aude Oliva, Antonio Torralba, Michal Irani, Tamar Rott Shaham
Computer Vision
Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers
Shuhong Zheng, Michael Oechsle, Erik Sandström et al. (+3 more)
Visual geometry
transformers have become powerful architectures for multi-view 3D reconstruction, enabling joint prediction of multiple 3D attributes in a feed-forward manner. However, their computational cost grows quadratically with the input sequence length due to the global
attention layers inside these models. This limits both their scalability and efficiency. In this work, we address this challenge with a simple yet general strategy: restricting the number of key/value
tokens that each query interacts with during global
attention. To achieve effective
token selection, we introduce a two-stage framework. First, an inter-frame selection step operates at the frame level to identify frames that should be preserved. Second, an intra-frame selection step further discards more redundant
tokens within the selected frames. Our analysis highlights the advantage of a diversity-based strategy for inter-frame selection, which ensures broad coverage of the scene. For intra-frame selection, we show that layer-aware sparsification is necessary, with the selection process guided by the
entropy of the global
attention pattern. Our approach offers a superior speed-accuracy trade-off compared to existing solutions. Extensive experiments show that it accelerates visual geometry
transformers by over 85% for scenes with 500 images while maintaining, or even improving, baseline performance, which hints that how our
token selection strategy can play a crucial role in future applications of visual geometry
transformers. Our project website is available at https://zsh2000.github.io/good-
token-hunting.github.io.
Read the full abstract ▾
All authors Shuhong Zheng, Michael Oechsle, Erik Sandström, Marie-Julie Rakotosaona, Federico Tombari, Igor Gilitschenski
Computer Vision
Smart-Insertion-V: Photorealistic Video Insertion via a Closed-Loop Feedback Dual-Stream Framework
Xiao Cao, Yansong Qu, Xiangzhen Chang et al. (+6 more)
Mask-free
video object insertion has emerged as a challenging task, requiring harmonious integration of reference objects into source
videos. However, existing methods struggle when references exhibit severe stylistic domain gaps with the source scene. To overcome this, we propose \textit{Smart-Insertion-V}, an end-to-end Dual-Stream framework that concurrently conducts
video insertion and image style transfer. Within this framework, the image stream synchronously guides the
video generation process, while a Closed-loop Feedback mechanism is further incorporated to ensure robust insertion. Inevitably, integrating these diverse conditioning signals results in feature
entanglement and style leakage. To tackle this issue, we design Dual-World-View RoPE to distinguish different signals via spatial-temporal offsets without incurring heavy training overhead. Furthermore, to facilitate spatial grounding and stylistic adaptation, we introduce a Decoupled Guidance Module that leverages a
Vision-
Language Model for semantic
reasoning while preserving original temporal guidance with native text encoder. To bridge data gap for harmonious reference insertion task, we propose a data curation pipeline and will release an open-source
dataset. Experiments demonstrate that our method can insert objects into plausible positions while achieving the most harmonious results.
Read the full abstract ▾
All authors Xiao Cao, Yansong Qu, Xiangzhen Chang, Wen Xiao, Jiakui Hu, Heyuan Li, Jialun Liu, Zhiyong Huang, Xuelong Li
Computer Vision
HorizonStream: Long-Horizon Attention for Streaming 3D Reconstruction
Chong Cheng, Peilin Tao, Nanjie Yao et al. (+9 more)
Online 3D reconstruction requires estimating camera pose and scene geometry under strict causal and bounded-
memory constraints. Existing methods often suffer from drift, jitter, or collapse on long sequences. We trace these failures to a fundamental mismatch. Streaming geometry is inherently temporally heterogeneous, with evidence ranging from short-lived correspondences to persistent global scale. However, current architectures impose uniform and pathological influence patterns. For example, sliding windows enforce hard cutoffs, while ungated recurrence and causal
attention cause cache saturation and spike-like
attention sinks. To resolve this, we formalize
geometric propagation as an evidence influence kernel and propose HorizonStream, a long-horizon
Transformer that explicitly factorizes this kernel. For the long-range temporal factor,
Geometric Linear
Attention learns channel-wise decay rates to enable bounded, multi-timescale propagation of
geometric evidence. For the short-range spatial factor,
Geometric Local
Attention with Spatiotemporal RoPE performs reliable 3D matching while suppressing
attention sinks. Finally, Metric Readout
Tokens recover stable scale and rigid pose directly from the persistent
geometric state. Extensive experiments show that HorizonStream, trained on only 48-frame clips, generalizes stably to sequences exceeding 10,000\ frames with constant
memory and linear time, achieving state-of-the-art streaming 3D reconstruction performance. Project Page: https://3dagentworld.github.io/horizonstream/
Read the full abstract ▾
All authors Chong Cheng, Peilin Tao, Nanjie Yao, Guanzhi Ding, Xianda Chen, Yuansen Du, Xiaoyang Guo, Wei Yin, Weiqiang Ren, Qian Zhang, Zhengqing Chen, Hao Wang
Computer Vision
GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstruction
Katharina Schmid, Nicolas von Lützow, Jozef Hladký et al. (+2 more)
We introduce a new approach to high-fidelity 3D scene reconstruction from multi-view RGB images that tightly couples reconstruction with a strong generative 3D prior. We cast scene reconstruction as conditional 3D generation over a set of spatially-localized, overlapping chunks that together tile the scene, scaling generation to large scene extents. Crucially, we inherit the fidelity and completeness of state-of-the-art generative shape models – we use Trellis.2 as an example – which we generalize to the scene level. To this end, we propose a projection-based conditioning mechanism that lifts posed multi-view image features into a coherent 3D representation aligned with the generative model, independent of view ordering and spatially anchored to the scene, yielding high-fidelity, multi-view consistent generated geometry. This enables lifting the strong object-level prior of Trellis.2 to multi-view, scene-scale generation, producing faithful, editable PBR mesh reconstructions of indoor environments. As a result, we obtain high-fidelity results that outperform cutting-edge reconstruction methods by 16%.
Read the full abstract ▾
All authors Katharina Schmid, Nicolas von Lützow, Jozef Hladký, Angela Dai, Matthias Nießner
Computer Vision
PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs
Rim Assouel, Amir Bar, Michal Drozdzal et al. (+1 more)
Despite remarkable progress in
Multimodal Large
Language Models (MLLMs), these models still struggle with fine-grained understanding tasks. In this work, we propose Procedurally Generated Tasks (PGT), a simple data-driven framework that serves a dual purpose: inducing fine-grained visual understanding and acting as a low-cost diagnostic tool to identify the source of perception failures. By overlaying unambiguous
geometric primitives on images, PGT generate additional dense supervision that disentangles visual grounding capability from semantic priors. Extensive experiments on relational, quantitative, and 3D/depth understanding
benchmarks show that PGT yields remarkable gains across diverse architectures. Instruction tuning MLLMs on LLaVA-v1.5-Instruct augmented with PGT data results in improvements of up to +20% on the What'sUp
benchmark and +13.3% on CV-Bench-2D, while maintaining general perception capabilities. Moreover, finetuning state-of-the-art MLLMs on PGT data leads to boosts of up to +5.5% on What'sUp and +8.3% on CV-Bench-2D. These findings demonstrate that PGT effectively address the bottleneck of fine-grained perception, revealing that many spatial
reasoning deficits stem from inadequate supervision signals rather than inherent architectural or resolution limitations.
Read the full abstract ▾
All authors Rim Assouel, Amir Bar, Michal Drozdzal, Adriana Romero-Soriano
Computer Vision
LaMo: Self-Supervised Latent Motion Priors for Physical Realism in Video Generation
Bo Jiang, Depu Meng, Yihan Hu et al. (+3 more)
Modern
video generators produce visually compelling clips but still struggle with physical and motion consistency, limiting their use as reliable world simulators. Existing remedies often rely on external simulators, teacher models, or curated physics-focused data. We explore a complementary self-supervised direction: extracting motion cues from the unlabeled
videos already used to train
video diffusion models. We propose LaMo, which formulates a latent motion prior over frame-to-frame latent changes conditioned on the current latent and
prompt. This prior is exposed through two lightweight readouts: a macro motion drift used during training as a Motion Drift Loss, and a learned micro motion field used during sampling as Motion Prior Guidance. Both components are plug-and-play with existing
video diffusion backbones, requiring no architectural or I/O changes. On VideoPhy and VideoPhy2, LaMo improves CogVideoX backbones and outperforms recent physics-aware baselines that use external supervision. On VBench, it preserves overall generation quality while improving motion-related dimensions. These results suggest that unlabeled
video contains useful motion supervision for improving physical fidelity in modern
video diffusion models.
Read the full abstract ▾
All authors Bo Jiang, Depu Meng, Yihan Hu, Yichen Xie, Tianshuo Xu, Wei Zhan
Computer Vision
Linxiang Su
Vision Transformers (ViTs) can learn strong image-level representations while their patch representations become less effective for dense prediction during prolonged training. We revisit this dense degradation phenomenon and argue that it is not fully explained by high-norm artifacts alone. Instead, we characterize semantic
diffusion: an optimization shortcut in which global semantic information spreads through patch
tokens beyond what is locally justified. Our analysis shows that dense representation quality is not captured by locality alone: shallow features can remain better aligned with foreground regions yet underperform deeper features, and [CLS] features remain complementary for dense prediction. These observations suggest that the goal should not be to remove global context, but to make
token interactions more selective. We therefore study sparse
attention as a minimal intervention, replacing softmax
attention with entmax-1.5 while preserving global
token connectivity. On DINOv1 ViT-S/16 trained for 200 epochs on ImageNet-1K, this change preserves ImageNet linear probing accuracy and substantially improves semantic
segmentation performance: VOC mIoU increases from 42.80 to 48.78, ADE20K from 19.85 to 21.97, and Cityscapes from 36.79 to 37.87. These results suggest that selective
token mixing is a simple and effective bias for improving dense ViT representations.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Leveraging Foundation Models for Causal Generative Modeling
Aneesh Komanduri and Xintao Wu
Causal generative modeling is essential for developing reliable and transparent AI systems capable of counterfactual
reasoning. While existing approaches focus on integrating causal constraints during the training of generative models, they often lack a unified framework to leverage the zero-shot
reasoning capabilities of pretrained foundation models. We introduce FM-CGM, a modular framework for end-to-end visual causal
reasoning using pretrained foundation models. FM-CGM formalizes the causal pipeline through three core components: a concept extractor, a concept manipulator, and a counterfactual generator. By leveraging a large
reasoning model for causal inference and a text-to-image
diffusion model for generation, our approach enables zero-shot causal discovery, intervention, and counterfactual generation. We then develop Causal Semantic Guidance (CSG), a cross-
attention-based mechanism that ensures semantic interventions propagate to descendant concepts while preserving invariant regions. We empirically show that our approach can identify plausible causal structures and is suitable for faithful counterfactual image generation.
Read the full abstract ▾