Computer Vision
Semantic Saliency from Multi-Modal Large Language Model Scene Understanding Maps
Ziqi Wen, Shravan Murlidaran, Jonathan Skaza et al. (+2 more)
Image-computable low-level image saliency has shaped perceptual psychology and computer science, but is limited by its inability to capture human high-level cognitive factors that influence
attention and eye movements. Here, we propose a new fully automated method to compute semantic saliency by quantifying each object's contribution to scene comprehension without fitting any parameters to human data: the Automated Scene Understanding Map (AUTO-SUM). AUTO-SUM automatically segments and removes individual objects from a scene, uses
multimodal large
language models (MLLMs) to describe the original and modified scenes, and measures the semantic similarity between these descriptions using text
embeddings. We show that AUTO-SUM can predict which object is most important in a human-measured scene understanding map (H-SUM) estimated using human scene descriptions. We also show that AUTO-SUM can predict the object most fixated by human observers during free viewing and scene description tasks better than a low-level saliency model (GBVS, Itti-Koch), a saliency model trained based on human meaning ratings (Meaning Map), and comparable to DeepGaze, which is trained on human fixation data. We contend that AUTO-SUM can be used as a semantic saliency model that complements lower-level saliency models. Code are available at https://github.com/starsky77/AUTO-SUM-Source-Code .
Read the full abstract ▾
All authors Ziqi Wen, Shravan Murlidaran, Jonathan Skaza, William Wang, Miguel P. Eckstein
Computer Vision
EmoTrace-Bench: Diagnosing Emotional Expression in Text-to-Video Generation through Controlled Cues and Intensity
Yeheng Wang, Liudan Shi, Yi Zheng et al. (+5 more)
Text-to-
video (T2V) models are increasingly used to generate
videos with targeted emotional expressions, yet existing evaluation methods mainly rely on holistic
video-text
alignment and provide limited insight into whether failures arise from target-emotion expression, intensity control, or specific affective cues. We introduce EmoTrace-Bench, a fine-grained diagnostic
benchmark that controls emotion intensity and selected affective cues in
prompts and evaluates generated
videos at multiple levels. EmoTrace-Bench contains 448 controlled
prompts covering eight emotions, two intensity levels, and diverse cue conditions. Using 12 T2V models, we generate 5,376
videos and evaluate target-emotion presence, perceived intensity, and ten audiovisual dimensions of emotional expression with an automated evaluator calibrated and validated against human annotations on a subset of the data. Results reveal substantial differences across models and
prompt designs.
Prompts that explicitly describe a character's emotional reaction are followed more reliably than
prompts that describe only events or scenes, whereas adding more affective cues or using more explicit cinematic audiovisual
language does not consistently improve emotional generation. These results illustrate how aggregate
alignment scores may overlook localized differences in target-emotion expression, intensity control, and cue realization. EmoTrace-Bench provides a traceable and interpretable framework for diagnosing emotional
video generation without modifying existing generation pipelines, and we release the
prompts and the automated evaluation rules.
Read the full abstract ▾
All authors Yeheng Wang, Liudan Shi, Yi Zheng, Jingke Li, Shiyuan Liu, Xin Tao, Yuanxing Zhang, Guoyu Sun
Computer Vision
Rethinking Supervision in Document Unwarping: A Self-consistent Flow-free Approach
Shaokai Liu, Hao Feng and Wengang Zhou
In recent years, the proliferation of smartphones has led to an upsurge in the digitization of document files via these portable devices. However, images captured by smartphones often suffer from distortions, thereby negatively affecting digital preservation and downstream applications. To address this issue, we introduce DRNet, a novel deep network for document image rectification. Our approach is based on three key designs. Firstly, we exploit the intrinsic
geometric consistency inherent in document images to guide the learning process of distortion rectification. Secondly, we design a coarse-to-fine rectification network to leverage the representations derived from the distorted document image, thereby enhancing the rectification result. Thirdly, we propose a unique perspective for supervising the learning of rectification networks, where undistorted document images are employed for supervision, which is free of warping mesh as ground truth in existing methods. Technically, both low-level pixel
alignment and high-level semantic
alignment jointly contribute to the learning of the mapping relationship between deformed document images and distortion-free ones. We evaluate our method on the challenging DocUNet
Benchmark dataset, where it sets a series of state-of-the-art records, demonstrating the superiority of our approach compared to existing learning-based solutions. Additionally, we conduct a comprehensive series of ablation experiments to further validate the effectiveness and merits of our method.
Read the full abstract ▾
Computer Vision
Harnessing SPAD Noise: Test-Time Gated Attention for Indoor Depth Estimation
Yiquan Gao
Monocular depth estimation (MDE) has emerged as a cost-effective alternative to hardware-dependent 3D sensing modalities, yet predicting precise metric depth remains a persistent challenge due to intrinsic scale ambiguity, edge bleeding, and limited zero-shot generalization in unseen environments. Recent multi-modal approaches incorporate 1D/3D Single-
Photon Avalanche Diode (SPAD) data to resolve these ambiguities, but they mainly treat SPAD noise as a harmful artifact required for denoising or leave the noise ignored. In this paper, we present an essentially different insight: native SPAD noise is an informative prior that implicitly encodes deterministic environmental characteristics. Driven by this paradigm shift, we introduce a novel, lightweight Test-Time Gated
Attention framework by Harnessing SPAD Noise, therefore dubbed HSN. Tailored for indoor scenarios, HSN leverages the synergistic combination of 1D SPAD histograms and Gate Adapters (GAs). Through three complementary optimization phases, it generates channel-wise gated
attention vectors to dynamically adjust encoder feature maps at inference time, effectively resolving ambiguous scale and boundary challenges without the heavy computational overhead or dense 3D modalities that other TTA methods suffer from. Extensive experiments on challenging real-world indoor
benchmarks show that HSN achieves state-of-the-art metric depth accuracy in in-domain settings, while exhibiting even greater potential under zero-shot generalization to unfamiliar environments. Furthermore, interpretability analysis reveals that the proposed gated
attention mechanism adaptively shifts
attention toward deeper semantic layers under intensifying SPAD noise. Code will be made publicly available.
Read the full abstract ▾
Computer Vision
OmniRSI: Taming Latent Diffusion for Unified Multimodal Remote Sensing Image Generation
Hebaixu Wang, Jing Zhang, Haonan Guo et al. (+3 more)
Multimodal remote sensing image generation emerges as a promising paradigm for recovering complete and complementary observations from heterogeneous and partially observed remote sensing data. However, existing methods predominantly depend on sensor-specific fine-tuning of natural image generation models, limiting unified
multimodal generation capacity due to the absence of shared representations and robust conditioning mechanisms. To address these challenges, we propose OmniRSI, a unified
multimodal remote sensing image generation model. Specifically, we develop a remote sensing-specific variational autoencoder that captures
multimodal priors and promotes a locally smooth latent space for unified representation and robust reconstruction across diverse observed data. Building upon that, we fully adapt Stable
Diffusion 3.5 with a ControlNet branch to support unified
multimodal generation and translation conditioned on textual and visual cues. Besides, we devise a training-free cyclic latent
alignment mechanism that iteratively enriches incomplete visual conditions with generated
multimodal information, progressively improving guidance reliability while preserving observed measurements. We also construct RSI-MM, a large-scale
dataset comprising 11.5M multi-resolution images with text annotations and partially paired data for training and evaluation. Extensive experiments verify that OmniRSI achieves superior performance in
multimodal reconstruction, generation and translation.
Read the full abstract ▾
All authors Hebaixu Wang, Jing Zhang, Haonan Guo, Di Wang, Jiayi Ma, Bo Du
Computer Vision · Cross-listed from cs.AI
A Decade of PhotoMappers: A Longitudinal Study of Crowdsourced Disaster Photos Geolocalization with Vision-Language Models and Geospatial Reasoning
Hao Li, Wenping Yin, Fabian Deuser et al. (+5 more)
Crowdsourced disaster imagery provides timely, ground-level observations when conventional disaster mapping is delayed or incomplete, yet its geographic value depends on where observations are produced, how they are interpreted, and whether they can be reliably located as actionable Geospatial Intelligence (GEOINT). Using more than a decade of PhotoMappers, a crowdsourced disaster-mapping initiative, this study examines the changing geography of disaster information and asks how emerging GeoAI models compare with human geographic interpretation. We combine longitudinal spatial analysis with cross-view geolocalization using street view imagery (SVI) and remote sensing imagery (RSI), and evaluate
vision-
language models (VLMs)
reasoning through 30 indicators spanning geographic cues and
reasoning evidence. The results reveal a geographically and temporally uneven archive shaped by changing disaster types, Community Lifelines, and operational priorities. Integrating VGI, SVI, and RSI with Sinkhorn re-ranking achieves 40.90% R@1, 61.10% R@5, and 69.45% R@10, while yearly R@1 varies markedly from 25.00% to 65.07%. Geographic
reasoning is similarly uneven: Qwen3.6-27B achieves an overall agreement of 67% compared with 51% for InternVL3-8B, with both models showing greater difficulty with localized and place-specific evidence than broad spatial configurations. We argue that the value of GeoAI lies less in replacing volunteered geographic interpretation than in complementing it, scaling candidate
retrieval while retaining human judgment for ambiguous and consequential cases. This provides a pathway toward more human-centered and responsible GeoAI for disaster mapping. All data and code are openly available at https://github.com/bobleegogogo/PhotoMappers-VLM.
Read the full abstract ▾
All authors Hao Li, Wenping Yin, Fabian Deuser, Jia Jia, Ziqi Liu, Levente Juhász, Fan Zhang, Filip Biljecki
Computer Vision · Cross-listed from cs.CR
From Acoustic Sensing to Physical Actions: A Comprehensive Survey of Security Risks in Edge-Deployed Audio LLM-Based Agents
Binhong Wang, Haidong Kang, Yanfeng Zhang et al. (+3 more)
Audio Large
Language Models (Audio LLMs) are embedded in
agents for smart homes, vehicles, wearables, and embodied systems, where acoustic inputs can propagate from perception into persistent state and consequential execution. This setting extends security analysis beyond model behavior to the system conditions that determine how acoustic inputs can influence
agent behavior and external actions. Existing surveys examine acoustic security, LLM-based
agents, and edge deployment in isolation, leaving their security interactions in deployed Audio LLM-based
agents insufficiently characterized. This survey introduces an Acoustic Sensing-to-Action (S2A) system model that traces these security implications along the path from acoustic sensing to external execution, while accounting for edge-cloud runtime and lifecycle transformation. We use the model to synthesize attacks, defenses, and evaluation methods across the acoustic,
agent, and edge-system literature. The synthesis shows that security gaps can emerge within components and at the transitions between stages, where protections developed for one stage do not necessarily provide end-to-end assurance in the deployed system. These findings motivate security evaluation that follows acoustic influence from sensing to action across persistent state, system boundaries, and changing deployment conditions in real-world
agents.
Read the full abstract ▾
All authors Binhong Wang, Haidong Kang, Yanfeng Zhang, Hong-Ning Dai, Guo Yu, Shangce Gao
Computer Vision
VideoEvoHarness: Self Evolving Agent Harness from Unlabeled Videos for Video Understanding
Weixin Xu, Yifan Liu, Yi Wang et al. (+5 more)
Video understanding
agents rely on a harness to acquire evidence, use tools, and reason, but improving it requires substantial effort. Recent self
evolution methods seek to optimize harnesses through execution feedback to reduce human intervention. However, existing methods often rely on annotated
video questions and answers or focus on updating model parameters, leaving harness
evolution from unlabeled
videos underexplored. VideoEvoHarness jointly evolves a Questioner and a Solver harness from unlabeled
videos with frozen model parameters. The Questioner constructs questions and computes reference answers directly from
video memory records. Question quality assessments and Solver execution trajectories guide question generation policy revisions. The Solver supports evolving skills, strategies, hyperparameters, and tools using execution feedback. Executable strategies define workflows in code, reducing repeated interpretation of natural
language instructions. Each round, candidate harnesses are compared on identical validation questions, with validated improvements retained for later use. Experiments on
benchmarks and a dedicated
benchmark question set, using
videos disjoint from
evolution videos, show higher overall accuracy than initially and transfer gains across backbones. Overall accuracy on this set also improves progressively across rounds. This supports our foundational framework for harness
evolution from unlabeled
videos, opening a promising direction for self evolving
video agents. Code is available at https://github.com/Wei-Xin-Xu/VideoEvoHarness.
Read the full abstract ▾
All authors Weixin Xu, Yifan Liu, Yi Wang, Mingfei Rong, Zhenyu Yang, Zhisheng Zhong, Shengsheng Qian, Changsheng Xu
Computer Vision
UniForge3D: Joint 3D Reconstruction and Generation in a Shared Anchor-Guided Space
Wenjue Chen, Wenyue Chen, Heliang Zheng et al. (+3 more)
Generating complete 3D assets from sparse, unposed images requires preserving observed geometry while completing unseen regions. Joint reconstruction and generation offers a way to address both goals: reconstruction recovers the geometry observed in the input images, while generation contributes learned shape priors. However, combining camera-relative reconstruction with canonical-space generation introduces a coordinate mismatch that makes direct
geometric interaction more difficult. Moreover, conditioning each 3D location on all views without accounting for their relevance may mix useful
geometric evidence with unrelated image content, hindering local detail recovery. We present UniForge3D, which jointly models reconstruction and generation in a shared anchor-guided space, resolving their coordinate mismatch. This space follows the anchor view's azimuth while keeping the object upright. The two branches exchange features bidirectionally to predict per-view point maps and a complete sparse voxel structure. We further condition point-map prediction on the generated structure to establish pixel-to-voxel correspondences and select the most informative input view for each voxel. Features from the selected views then guide detailed shape generation, helping preserve observed
geometric details. Our method achieves state-of-the-art appearance and geometry results on Toys4K and GSO, generating complete 3D assets with improved fidelity. Project webpage: https://xishuxishu.github.io/uniforge3d-project/
Read the full abstract ▾
All authors Wenjue Chen, Wenyue Chen, Heliang Zheng, Rongfei Jia, Yuan Liu, Meng Li
Computer Vision
Video-Based Sexual Harassment Detection Using Fine-Tuned Video Transformers
Jubayer- Al- Mahmud, Md Shamimul Islam and Md Nur Hossain
Sexual harassment represents a pervasive societal challenge, with a substantial proportion of incidents remaining unreported due to insufficient evidence and delayed intervention mechanisms. Smart surveillance systems capable of detecting such incidents in real-time could assist law enforcement and serve as a deterrent. However,
video-based harassment detection poses significant challenges, including variations in clothing, illumina- tion, partial occlusions, low resolution, and diverse camera view- points. Despite its social importance, this problem has received limited research
attention, primarily due to the scarcity of large- scale annotated
video datasets. To address this gap, we extend the Sexual Harassment
Video (SHV)
dataset to 600 short
video clips sourced from YouTube and annotated into harassment and non-harassment categories using consistent labeling guidelines. We fine-tune three state-of-the-art
video transformer models, VideoMAE, ViViT, and TimeSformer, pretrained on Kinetics- 400, for binary harassment classification. Experimental results show that all three models achieve competitive classification performance, with TimeSformer attaining the highest test accu- racy of 92.27%, outperforming VideoMAE 91.71% and ViViT 91.16% on the extended SHV test set. Our analysis reveals that performance is notably influenced by
dataset size,
video quality, frame sampling strategy, and model architecture. This work presents a comprehensive evaluation of modern
video transformers for this sensitive real-world task.
Read the full abstract ▾
Computer Vision
Across Viewpoints and Time: Trajectory-Generalizable Physical Attacks on World Models and Video Understanding Models
Song Xia, Meiwen Ding, Chenqi Kong et al. (+2 more)
World models and
video understanding models are increasingly deployed in dynamic visual environments, yet their robustness to physical
adversarial threats remains largely underexplored. We study whether a single physical
adversarial texture can mislead the victim model across different real-world scenarios where observing viewpoints vary along uncontrolled object–camera trajectories. We identify that heterogeneous
video semantics and trajectories induce poorly aligned gradients for optimizing the
adversarial texture. To address that, we propose the Trajectory-aware
adversarial Alignment and Refinement Attack (TARA), which incorporates trajectory-aware optimization to capture the evolving
geometric variations of a printed texture under real-world scenarios. To improve cross-instance optimization compatibility, TARA incorporates trajectory-aware contrastive
alignment to encourage attacked representations from diverse
video–trajectory instances to approach a shared
adversarial region and then performs trajectory-aware task refinement to translate the aligned representation into task-level prediction errors. For reproducible evaluation, we develop a geometry-grounded tracking pipeline that provides physically plausible
adversarial regions with temporally consistent trajectories on EPIC-KITCHENS-100 and Something-Something V2. We further collect real-world print-and-recapture
videos to validate attack effectiveness under physical deployment conditions. Together, the annotated
benchmarks and real-world captured data form a reproducible testbed for future evaluation. Notably, our results reveal that a single printable
adversarial texture can persistently disrupt world models and
video understanding models under physical conditions, highlighting the fundamental security challenge.
Read the full abstract ▾
All authors Song Xia, Meiwen Ding, Chenqi Kong, Wenhan Yang, Xudong Jiang
Computer Vision
StimFlow: Synthesizing Brain Activity from Images via Stimulus-conditioned Flow Matching
Bui Van Hung, Dang Thanh Vu, Hoang-Son Vo et al. (+3 more)
The inherent variability of fMRI responses across trials encourages a generative view of the stimulus-to-response mapping. Prior generative encoders often operate in compressed latent spaces and condition predominantly on semantic features, which provide limited low-level cues for early visual cortex. We propose StimFlow, which synthesizes fMRI directly in native voxel space using conditional flow matching with a linear path. This design aligns training and inference distributions, enabling competitive performance with a single sample and reducing reliance on averaging. To match the visual hierarchy, StimFlow combines structural DINOv2 features with semantic CLIP features, which prove complementary along the cortical processing stream. With a single sample, StimFlow achieves a per-voxel Pearson correlation of 0.371, matching baselines that average five draws. Generation in voxel space allows direct ROI analysis and concept localization without a decoder. Furthermore, it recovers over half of full-data performance from just one hour of scans, supporting efficient cross-subject adaptation.
Read the full abstract ▾
All authors Bui Van Hung, Dang Thanh Vu, Hoang-Son Vo, Mai Duc Minh Huy, Mai Tien Dung, Soo-Hyung Kim
Computer Vision
BaT-SAM: Introducing Inductive Bias into SAM via a Mixture of Convolutional Experts for Brain Tumor Segmentation
Sumiaya Jannat, Md Mahedi Hasan, Md Shamimul Islam et al. (+1 more)
Accurate
brain tumor segmentation from
magnetic resonance imaging (MRI) is essential for diagnosis and treatment planning; however, manual delineation is labor-intensive and time-consuming. Existing approaches are either task-specific with limited generalization or rely on user-provided
prompts during inference. We present BaT-SAM, a parameter-efficient adaptation framework that enables fully automatic
brain tumor segmentation with the Segment Anything Model (SAM). BaT-SAM injects local inductive bias into SAM's frozen
Vision Transformer (ViT) encoder via Conv-LoRA, where image features are processed by a sparsely gated mixture of multi-scale convolutional experts. This design compensates for the limited translation-equivariant inductive bias of vanilla ViTs while fine-tuning fewer than 5M parameters (less than 1% of the full model). To remove the need for user
prompts, we further introduce a lightweight self-prompting box regressor that is jointly optimized with the adapter using scheduled sampling and predicts tumor locations directly from image
embeddings. On the BRISC2025
benchmark, BaT-SAM achieves a weighted mean IoU of 0.832 while outperforming state-of-the-art
benchmarks. It further demonstrates strong cross-
dataset generalization, achieving Dice scores of 0.702 on Figshare and 0.784 on the LGG MRI Segmentation
dataset.
Read the full abstract ▾
All authors Sumiaya Jannat, Md Mahedi Hasan, Md Shamimul Islam, Md Nur Hossain
Computer Vision · Cross-listed from cs.LG
CIViC: Evidence Gates for Audio-Conditioned Visual Prediction on Proxy Tasks
Mengzhe Geng, Max W. Y. Lam, Yuewen Cao et al. (+4 more)
Removing audio can shift a visual predictor's output without showing whether the task uses the intended audio information. We introduce Conditional Input and Visual Interpretation Checks (CIViC), a four-gate protocol that links an interpretation to an intervention, held-out task comparison, or endpoint control. We demonstrate CIViC on actor-disjoint face-raster tests, a synthetic audio-feature mixture, and source-
video-disjoint SpeakerVid proxy tasks. In the static face-raster test, cross-actor audio replacement meets the registered detection rule with a small effect; on a separate temporal endpoint, constant-audio fits score better than audio-trained fits in four matched seeds. In the synthetic mixture, the condition-augmented fit has higher 1,000-step RMSE than the audio-only fit (0.1807 vs. 0.1707), with the gap shrinking during training. In the SpeakerVid proxy, the five-feature-zero fit (five low-level audio summaries zeroed; identity, dialogue, and condition inputs retained) has lower whole-frame grayscale RMSE than aligned audio across four held-out conditions and five paired seeds, while aligned audio has lower segment-drift error. The results show that output activation, held-out error, and endpoint controls support different interpretations across these proxy tasks.
Read the full abstract ▾
All authors Mengzhe Geng, Max W. Y. Lam, Yuewen Cao, Shoukang Hu, Jing Xu, Marc Tessier, Xu Li
Computer Vision
TempoKV: Predictive KV Cache Quantization for Autoregressive Video Generation
Jiahui Hu, Weile Mo, Peiliang Cai et al. (+5 more)
Autoregressive (AR)
video diffusion models enable streaming generation of long
videos but suffer from significant
memory overhead due to the rapidly growing KV cache. Existing KV cache quantization methods are primarily designed for LLMs and transfer poorly to
video models, where activations exhibit different statistics and strong spatiotemporal redundancy. We observe that the
video KV cache is largely predictable over time, with a clear asymmetry. Keys are highly predictable from their temporal history, while values depend primarily on the current key and need a short history of preceding values. Based on these observations, we propose \method, a training-free predictive KV cache quantization framework for AR
video generation. \method predicts keys and values according to their distinct dependencies and stores only the residuals at low precision. Both predictors are lightweight, calibrated once offline by linear regression on a small set of BF16 samples, and remain fixed across
prompts and generation runs. Experiments on Self-Forcing, Causal Forcing, and LongCat-
Video-13B show that \method consistently achieves higher generation quality than existing KV cache quantization methods at comparable or higher compression ratios, reaching up to $6.71\times$ KV cache compression with the best overall quality–efficiency tradeoff. Our code will be released on GitHub.
Read the full abstract ▾
All authors Jiahui Hu, Weile Mo, Peiliang Cai, Jiacheng Liu, Jinkui Ren, Xiantao Zhang, Xiaobing Tu, Linfeng Zhang
Computer Vision
Channel Match: Calibrating Shape Latents for Few-Step 3D Generation
Di Yang, Zhihao Li, Yanhai Xiong et al. (+1 more)
Recent native 3D generators achieve high-quality asset creation by modeling structured 3D latents with flow-based
transformers, but their high-resolution generation remains computationally expensive due to costly iterative sampling. In this work, we introduce a few-step
distillation framework for native 3D generation, enabled by continuous self-consistency training with a custom JVP FlashAttention kernel and an analytic velocity regression objective for sparse structured 3D latents. To stabilize joint optimization, we further introduce an adaptive loss weighting strategy that automatically balances them without manual tuning. Beyond acceleration, we reveal a previously overlooked failure mode of few-step native 3D generation: sampling-induced latent distribution drift preserves
geometric correctness but disrupts downstream PBR texture synthesis, causing texture artifacts on geometrically valid assets. To address this issue, we propose Channel Match, a training-free adaptive per-channel latent calibration method that restores the latent distribution without modifying either generator. We validate our approach on TRELLIS.2 at the $1024^3$ resolution setting. Our two-step model achieves $4.5\times$ faster geometry sampling while maintaining geometry quality comparable to the many-step teacher, and Channel Match further improves rendering quality by recovering the latent interface between geometry and appearance generation.
Read the full abstract ▾
All authors Di Yang, Zhihao Li, Yanhai Xiong, Yufei Wang
Computer Vision
HGA-Quant: Heterogeneity-Aware Quantization for AutoRegressive Visual Generation Models
Xinmeng Liu, Xianglong Yan, Kaicheng Yang et al. (+2 more)
Recently, AutoRegressive Visual Generation (ARVG) models have emerged as a competitive paradigm for high-quality image synthesis. However, their substantial computational and
memory demands make them difficult to deploy on resource-constrained edge devices. Post-Training Quantization (PTQ) provides a promising solution to reduce the computational and
memory costs of ARVG models, yet existing methods often suffer from substantial image quality degradation under low-bit quantization. In this paper, we propose a training-free PTQ framework for ARVG, consisting of Tail-Aware Bounded-init Grid Refinement (T-BGR) and Coarse-to-Fine Activation Precision Scheduling (CAPS). T-BGR introduces a tail-aware reconstruction objective for weight quantizer calibration, assigning greater importance to distribution-tail coefficients and preserving fine-grained information without introducing additional parameters or high-precision branches. CAPS performs coarse-to-fine activation precision scheduling by jointly considering the importance and computational cost of different generation stages, enabling efficient mixed-precision allocation under a given bit budget. Extensive experiments on multiple ARVG backbones demonstrate that our framework consistently improves robustness under aggressive low-bit quantization. Notably, on Infinity, our framework achieves near full-precision image generation quality at an effective W4A5.6 activation precision, while existing methods at W4A6 still exhibit noticeable quality degradation or inconsistent performance. Code is available at https://github.com/lxm-lc/HGA-Quant.
Read the full abstract ▾
All authors Xinmeng Liu, Xianglong Yan, Kaicheng Yang, Weisheng Dong, Yulun Zhang
Computer Vision · Cross-listed from cs.GR
Co-DSL: Human-LLM Co-design of Reusable Spatial Concepts for Layout Generation
R. Kenny Jones, Jiajun Wu and Maneesh Agrawala
We present Co-DSL, a framework in which users and LLMs co-design reusable spatial concepts for layout generation. In the first stage, our system allows users to create a library of personalized operations. To add a concept, a user provides a description, which the LLM converts into an initial implementation. The user can then observe the concept's execution behavior over example scenarios and provide lightweight feedback to the LLM, which revises the concept's implementation logic. In the second stage, given a library, an LLM can author a layout program that meets a new layout request from a user. Executing a layout program instantiates a concept-graph, a declarative composition of library concepts. In this way, we require the LLM only to specify which concepts should hold, while we use a solver to search for a layout that satisfies them. Across five spatial layout domains, we find that Co-DSL libraries let LLMs capture layout patterns that are challenging for prior fixed-operation systems, and demonstrate that our concept-graph representation is needed for LLMs to reliably compose concepts together. We further find that Co-DSL lets LLMs author generative layout programs that expose high-level parametric controls users can manipulate to explore a distribution of related layouts without re-prompting.
Read the full abstract ▾
Computer Vision
ASM-6D: Fast Certified Pose-Shape Inference with Complete Correspondence Flow Maps
Ho Jin Choi, Minku Kim and Nadia Figueroa
Category-level 6D pose and shape estimation requires recovering an object's pose and geometry from partial observations without an exact instance model. We present ASM-6D, which couples generative complete-geometry prediction with explicit category-level pose-shape inference through shared correspondences. Given an RGB-D image, ASM-6D generates alternative complete 3D correspondence hypotheses whose point identities are shared with a category-level Active Shape Model (ASM), including occluded points. Each hypothesis is fitted directly for rotation, translation, metric scale, and instance shape without a separate correspondence-assignment stage. We analytically eliminate translation and shape coefficients, reducing fitting to a nine-variable scaled-rotation optimization problem that enables efficient batched evaluation of multiple hypotheses in real time. ASM-6D achieves state-of-the-art performance on ROPE and HouseCat6D
benchmarks. Ablations demonstrate the importance of generative correspondence prediction and active shape variation. The representation also supports shape completion, shape-dependent pose ambiguity, tracking, and robotic grasping at 6-38 Hz, depending on the inference budget.
Read the full abstract ▾
Computer Vision
Protein Crystallization Segmentation and Classification Using Subordinate Color Channel in Fluorescence Microscopy Images
Truong X. Tran, Marc L. Pusey and Ramazan S. Aygun
The accuracy of detecting
protein crystals for fluorescence microscopy images is very critical for high throughput and automated systems. Although the trace fluorescent labeling method could highlight
protein crystals, reflection and emission from the fluorescence dye is not always due to
crystal regions. Therefore, the analysis of the peak wavelength in the emission spectra of a fluorophore may not always yield effective results. In this paper, we show that using the subordinate color intensity corresponding to longer wavelengths than the peak wavelength of the emission spectra could improve the accuracy of
protein crystal detection. Hence, we have built a segmentation method based on the percentile intensity of the subordinate color for trace fluorescently labeled (TFL'd)
protein crystallization trial images. Compared to using the dominant color channel, our segmentation method on subordinate color channel was able to reduce the misclassification rate of likely-leads or
crystals as non-
crystals by the percentage of from 9.71% to 2.02% depending on the classifier. Similarly, the accuracy of classifiers were increased by the percentage of from 1.77% to 5.53%. Our method reached around 94% accuracy while keeping misclassification of likely-leads and
crystals as non-
crystals below 1%. Moreover, to evaluate the generalizability of our method, we have conducted new wet lab experiments on two
proteins, Concanavalin A (Con A) and Ab inorganic pyrophosphate (AbIPPase), and the misclassification rate was below 1%. Our experiments show that using the subordinate channel may be more helpful for TFL'd
protein trial image classification.
Read the full abstract ▾
Computer Vision
Rui Wang, Haoyu Zhang and Ran Cheng
Depth estimation and Gaussian reconstruction share a need for multi-scale feature processing, yet use different prediction interfaces and training objectives. Whether a decoder's internal computation can be optimized once and reused across these systems remains a structural design question. Search over primitive configurations alone leaves the organization of transformations fixed. To address this gap, we propose GeoBridge, an evolutionary framework for transferable decoder
topologies. First, GeoBridge makes stage-internal
topology searchable through residual, sequential, and parallel composition of configurable primitives. Shape-preserving composition retains the stage interfaces of a fixed multi-scale scaffold. Second, a functional-tree encoding supports stage-aware crossover and hierarchical mutation, while a gradient sensitivity-
alignment proxy evaluates candidates without training. Multi-objective
evolution balances this proxy with computation and parameter cost. Third, selected
topologies are instantiated and trained in target systems with task-specific prediction interfaces. A single source search takes approximately 12 GPU hours. GeoBridgeNet-A3 achieves the best depth accuracy among four adapted decoder baselines under a common encoder and training protocol on NYU Depth V2 and KITTI. The same
topology reaches 27.11 dB PSNR in Gaussian novel-view synthesis on RealEstate10K, and its refined representations support open-vocabulary segmentation on LERF. These results support decoder
topology as a reusable design object across the evaluated
geometric vision systems. Code and pretrained weights are available at https://github.com/EMI-Group/GeoBridge-NAS.
Read the full abstract ▾
Computer Vision · Cross-listed from physics.med-ph
Automated carotid artery atherosclerotic inflammation segmentation in PET using MSR-Net
Ran Li, Abhinav K. Jha, Jie Zheng et al. (+8 more)
Objective. To develop an automated method for segmentation and uptake quantification of carotid atherosclerotic inflammation on 64Cu-CANF-comb PET while accounting for partial-volume and tissue-fraction effects. Approach. Synthetic carotid PET images with known fractional-volume ground truth were generated by combining MR-derived high-resolution activity maps with GATE Monte Carlo simulation incorporating major PET physical processes and matched to the Biograph mMR acquisition and reconstruction. A two-stage network first localized the carotid vessel wall from T1-weighted MR and then estimated voxel-wise inflammation fraction from masked PET. Both stages used a multiscale residual (MSR) convolutional backbone. Performance was compared with U-Net,
Transformer U-Net, DUCK-Net, and iterative Yang partial-volume correction. Main results. In a two-alternative forced-choice study, five nuclear-medicine physicians identified real images with 54% accuracy (54/100; Wilson 95% CI 0.44-0.63; exact binomial p = 0.48). On 184 simulated test slices from five held-out patients, the proposed method achieved a weighted DSC of 0.79, compared with 0.73 for U-Net, 0.69 for
Transformer U-Net, and 0.75 for DUCK-Net, with corresponding HD values of 1.85, 2.29, 2.32, and 2.23 voxels. Patient-cluster bootstrap comparisons favored the proposed method for DSC and HD against all three baselines (Holm-corrected p <= 0.031). Overall ENRMSE was 0.24 versus 0.59, 0.68, and 0.59 for the three DL baselines and 0.57 for iterative Yang. In seven patients, corrected SUV showed a positive association with MR-derived lipid-rich/necrotic core with hemorrhage. Significance. The proposed framework improved carotid PET segmentation and uptake quantification under simulated conditions. The patient analysis is preliminary and requires external validation with biological reference standards before clinical deployment.
Read the full abstract ▾
All authors Ran Li, Abhinav K. Jha, Jie Zheng, Richard Laforest, Yongjian Liu, Mohamed Zayed, Tyler J. Fraum, Barry A. Siegel, Malak Itani, Joyce C. Mhlanga, Pamela K. Woodard