Computer Vision
VisExMEM: Visual Evidence-Based Explainable Multimodal Evaluation Metric
Subhadeep Roy, Christoph Leiter and Steffen Eger
Current image-text evaluation often relies on a global score, which gives limited insight into which semantic details are supported by the image. Explainable metrics provide rationales, but the visual evidence supporting those rationales can still be difficult to verify. We introduce VisExMEM, a reference-free evaluator that decomposes captions into atomic claims, localizes visual evidence, verifies each claim, and averages claim-level support into a global score. Explanations use the same evidence and decisions. We train Qwen3.5-based 4B and 9B models on 100K samples with multi-task SFT, counterfactual refinement, and evidence-aware GDPO. Across eight
benchmarks, VisExMEM-9B achieves the highest descriptive composite among open-source evaluators in our comparison, exceeding VQAScore by 11.7 points while remaining competitive with GPT-5.6 Sol. We also introduce AGITA-Bench, containing 600 human-annotated image-text pairs and 8.7K atomic claims for global and claim-level evaluation. For untuned Qwen3.5 and GPT-5.6 Sol controls, decomposed prompting raises the same composite by 3.6-5.1 points relative to holistic prompting. Human evaluation shows that VisExMEM explanations outperform prior explainable metrics and approach GPT-5.6 Sol.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.DC
Federated Coreset Subsampling for Distributed Memory Banks
Jason Witzemann, Daniel Pröll and Sebastian Hegenbart
In real-world scenarios, industrial processes can span multiple factories and goods are often produced at multiple sites, leading to a shared yet non-identical distribution of data across locations. To address this, we propose two
federated approaches for i) combining existing coresets (Double Subsampling) and ii) creating new coresets (
Federated Subsampling) in a distributed anomaly detection setting. In particular, we adapt the
memory bank based PatchCore
algorithm to a
federated setting, resulting in increased cold-start performance, resilience to domain- and sensor-shift, and resilience to unreliable actor participation, a common problem in classical, network based
federated learning strategies.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Learning $\omega$-Powered Conditional Distributions with Importance-Weighted Flow Matching
Maciej Falkiewicz, Frantzeska Lavda, Jason Ramapuram et al. (+1 more)
Classifier-Free Guidance (CFG) with guidance scale $\gamma$ is commonly motivated by the $\omega$-powered conditional distribution obtained by setting $\omega=\gamma$, but CFG does not, in general, sample from this distribution exactly. We study whether the powered target can instead be learned explicitly. We introduce Importance-Weighted Conditional Flow Matching (IW-CFM), which reweights conditional flow-matching examples using density ratios obtained from frozen conditional and unconditional flow models. With exact normalized weights, the resulting objective has the same gradient as flow matching on the powered target; in practice, we use self-normalized estimation and a regularized empirical approximation for high-dimensional data. On synthetic distributions, IW-CFM recovers analytical powered targets for which numerically matched CFG, $\gamma=\omega$, exhibits a visible mismatch. On ImageNet, an amortized lightweight corrector achieves better
alignment than fixed- and matched-guidance CFG with the corresponding weighted and filtered reference distributions. We additionally present an exploratory CLIP analysis of powered-density rankings on real ImageNet data. Aggregate shifts across selected
prompt-specific contrasts coexist with heterogeneous classwise responses, including nonmonotonic trajectories. These results provide a practical route for studying and approximating model-defined powered conditional distributions.
Read the full abstract ▾
All authors Maciej Falkiewicz, Frantzeska Lavda, Jason Ramapuram, Alexandros Kalousis
Computer Vision
Not All Challenges Are Equal: Evaluating Interactive Liveness Detection Against Real-Time Face Swaps
Anton Firc, David Drtil, Martin Perešíni et al. (+3 more)
Remote identity verification assumes that a camera stream shows a live person, yet public real-time face-swap tools can replace that person before the stream reaches the verifier. Passive deepfake detectors generalize poorly to new generators, while interactive challenges are often selected without evidence of which gestures expose current attacks. We evaluate randomized gesture challenges in 108 live sessions with six subjects, 24 impersonated identities, and two face-swap tools, DeepFaceLive and FaceFusion, injected through a virtual camera. The static calibration segment reaches a session-level AUC of 0.878 with a frozen UCF spatial detector, while the mean across the active-gesture windows reaches 0.988. Pose and combined gestures have the highest point estimates, whereas expressions show no clear improvement. Hand occlusion against FaceFusion, configured with an occlusion mask, scores below the static baseline. The end-to-end time required to satisfy the validator also distinguishes attacks from genuine sessions, achieving a session-level AUC of 0.732, although this signal includes synthesis and transport delays. These results show that challenge sets should be tested against current attack configurations and that response time can provide an additional, deployment-specific signal.
Read the full abstract ▾
All authors Anton Firc, David Drtil, Martin Perešíni, Milan Šalko, Vojtěch Staněk, Kamil Malinka
Computer Vision
Recovering from Pose-Selection Failures in Repeated Visual Localization
Jaehyun Kim, Wonseok Kang and Tae-Wan Kim
Repetitive structures can give an incorrect pose candidate more inlier support than an accurate one, causing visual localization to select it. History-Guided Pose Reselection (HPR) uses coarse position forecasts from up to five past localization outputs to distinguish well-separated current-query candidates. The best-fitting forecast proposes an alternative; HPR accepts it only if every well-fitting forecast is much closer to it than to the localizer's pose. HPR needs no odometry and returns the selected candidate unchanged, using prediction for selection rather than pose refinement. With one development-set threshold, HPR improves recall within 5 meters and 10 degrees in all 12 main configurations of Hierarchical Localization (HLoc), MASt3R-based pose generation, and JIST-based
retrieval, losing successes on fewer than 0.4 percent of queries. On NAVER Gangnam B1, HPR corrects 97 HLoc failures and loses six successes; gains persist within 0.25 meters and 2 degrees on all three HLoc test sets. Median HPR runtime on HLoc candidates is below one millisecond, excluding pose generation. Recovery weakens under less frequent localization.
Read the full abstract ▾
Computer Vision
Throwing4: Phase-Aware Pose Modeling for Throw-Distance Prediction in Elite Athletics
Bikash Kumar Badatya, Kartike Tiwari, Jyotirmoy Amin et al. (+1 more)
Elite athletics throwing provides a structured biomechanical action sequence together with an objective continuous outcome, namely throw distance, making it a suitable setting for performance-oriented sports analytics. In this work, we investigate whether explicit phase
alignment of pose dynamics improves throw-distance prediction under realistic broadcast conditions. To support this study, we curate Throwing4, a
dataset of 500 elite throwing clips spanning javelin, discus, hammer, and shot put, with corresponding two-dimensional pose sequences, frame-level phase annotations, and official distance labels. We formulate throw-distance prediction from broadcast-derived pose as a regression problem and compare full-sequence modeling against a phase-conditioned approach that partitions each attempt into four technique stages and fuses phase-specific motion representations. Experiments across multiple graph-based pose encoders show a consistent reduction in prediction error when phase structure is incorporated, indicating that phase-aligned modeling provides a more stable representation than treating the entire motion as an unstructured sequence. These findings suggest that explicit biomechanical phase information is a useful inductive prior for pose-based performance modeling in elite athletics throwing. Beyond score prediction, the
dataset and protocol provide a basis for future work on fine-grained technique analysis, interpretable sports analytics, and robust modeling from broadcast
video.
Read the full abstract ▾
All authors Bikash Kumar Badatya, Kartike Tiwari, Jyotirmoy Amin, Ravi Hegde
Computer Vision
From Sound to Sense: An Anatomy of Semantic Supervision for Speech-Driven Visual Grounding in Remote Sensing Imagery
Zhongyu Pang
Speech-driven visual grounding in remote sensing (RSVG) removes the typing bottleneck of text-based interaction and is intended to preserve prosodic intent, enabling hands-free target acquisition in time-critical missions. The direct-audio paradigm, mapping raw
speech straight to a bounding box and bypassing automatic
speech recognition (ASR), was recently adopted by RSTalker, but what such grounders actually learn about
language remains unmeasured. SemEcho grafts a frozen Whisper
speech pathway onto a VLTVG grounding host so that
speech and text share one decoder, making the paired text, free by construction since
speech corpora in this task are synthesized from DIOR-RSVG expressions, a training-time teacher and a fair, same-host text upper bound. On this instrument we dissect three semantic injection channels (feature-mimicry
distillation, word-level CTC supervision, and two-stage ASR pretraining) with linear semantic probes, and release Audio-DIOR-R (80,973 training clips; 10 timbres x K=3; Qwen3-TTS synthesis with online sox noise lambda in [0, 0.4]) and Audio-DIOR-Comp, a re-split holding out 17 attribute-category combinations under a 2x2 acoustic-semantic protocol. Our anatomy yields four findings. (i) Whether direct
alignment beats a domain-trained ASR cascade is decided by recognizer quality: against a weak in-domain cascade (CTC 0.53) direct wins by 9.0 Acc@0.5 points with transcription errors concentrated on spatial-relation words, but a strong in-domain cascade (CTC 0.029) reaches the text upper bound (80.8 vs. 81.0) and beats even the strongest direct model by 8.8 points; the anti-ASR premise holds only against weak recognizers.
Read the full abstract ▾
Computer Vision
VideoSlate: Stateful Commit Control for Temporally Grounded Agentic Video Reasoning
Shuo Zheng, Yuanjie Dang, Peng Chen et al. (+4 more)
Agentic
video reasoning systems gather and assess
video evidence across stages, selecting relevant spans as visual evidence for answering questions. New candidates and observations
prompt reassessment of the current span, but revisions can correct localization errors or discard useful evidence already present in the adopted span. Systems therefore need to connect evidence organization across stages with temporal state control, allowing accumulated information to jointly inform revision assessment while explicit adoption decisions govern how assessment outcomes affect downstream visual evidence. We introduce VideoSlate, a stateful commit-control framework built around shared evidence. A typed evidence blackboard links candidate spans, provenance, construction, and verifier observations, enabling an incumbent-relative certifier to assess revision support from the candidate, current span, and related proposals. Based on certification, the controller authorizes a revision or retains the current span, separating evidence accumulation from state updates and recording each decision and its evidence on the blackboard. Extensive evaluations across
video grounding and grounded question-answering
benchmarks show that VideoSlate can improve temporal localization through selective revision.
Read the full abstract ▾
All authors Shuo Zheng, Yuanjie Dang, Peng Chen, Jianjun Xiang, Ronghua Liang, Mohammed Bennamoun, Qiuhong Ke
Computer Vision
SyncReward: Human-Aligned Synchronization Reward for Audio-Visual Generation
Kaisi Guan, Ying Ba, Wenjing Wang et al. (+4 more)
Recent years have witnessed rapid progress in joint audio–visual generation. However, audio–visual synchronization remains a challenge, as generated
speech and sound often temporally misalign with visual events. A key bottleneck is the lack of human-annotated data, reliable evaluation
benchmarks, and synchronization-aware evaluation models for generated audio–visual outputs. To address these limitations, we conduct a comprehensive study spanning data, benchmarking, and reward modeling. We first introduce SyncReward-Data-80K, a
dataset containing 80K professional synchronization ratings for clips from eight audio–visual generators. A taxonomy-guided construction pipeline covers four sound-event families:
Speech, Onset, Instrument, and Ambient. We further construct SyncReward-Bench, a held-out
benchmark of 1K clips, each independently rated by multiple professionals. Using this
benchmark, we systematically evaluate specialized synchronization metrics and omni-modal models, revealing their substantial misalignment with human judgments. Finally, we develop SyncReward, a reward model that combines semantic–temporal contrastive pretraining on real clips with human supervision on generated clips. As an evaluation metric, SyncReward achieves the strongest agreement with human ratings among the evaluated methods. As a reward, it improves synchronization through inference-time best-of-$N$ selection and reinforcement learning based post-training. We will release the data,
benchmark, model, and code at https://github.com/WeChatCV/SyncReward.
Read the full abstract ▾
All authors Kaisi Guan, Ying Ba, Wenjing Wang, Xihua Wang, Chen Li, Jing Lyu, Ruihau Song
Computer Vision
WorldRoPE: Probabilistic World-Centric Rotary Embedding for Multi-View Attention
Mijeong Kim, Kyujin Lim and Bohyung Han
Rotary Positional
Embedding (RoPE) is widely used to encode relative positions in
Transformers. In multi-view settings, however, patch coordinates are defined independently in each image: the same 3D point can therefore have different coordinates across views, making them insufficient for establishing cross-view correspondences. We introduce WorldRoPE, a probabilistic world-centric RoPE for multi-view
attention. At each layer, WorldRoPE back-projects each patch
token into a set of volumetric Gaussian primitives in a shared world frame. These primitives represent multiple plausible 3D supports for a
token, jointly capturing alternative depth hypotheses, depth uncertainty, and the finite spatial extent of its patch frustum. When a key
token's support is projected into a query view, it induces a distribution over where that key may appear in the query image. WorldRoPE integrates the rotary operator over this distribution in closed form, combining rotations at the projected primitive centers according to their mixture weights with covariance-dependent attenuation. This makes
attention less sensitive to uncertain key locations in the query view, without rasterizing or warping
token features. The formulation reduces to deterministic RoPE when the support collapses to a point. Across layers, the predicted geometry guides cross-view
attention, while the updated features refine the geometry used at the next layer. Experiments on novel view synthesis, depth estimation, and 3D geometry reconstruction demonstrate the effectiveness of WorldRoPE for multi-view
Transformers.
Read the full abstract ▾
Computer Vision
BAT3R: Robust Online 3D Reconstruction with Bayesian Adaptive State Updates
Lijian Li, Yuanpeng He, Yudian Zheng et al. (+6 more)
Feed-forward
transformer models have driven remarkable progress in 3D
vision,yet their quadratic complexity renders them impractical for long
video sequences. Streaming 3D reconstruction addresses this bottleneck by processing frames sequen?tially with constant
memory. However, existing recurrent architectures inevitably suffer from progressive degradation over long sequences, as they fail to account for the intrinsic reliability of observations and the accumulated confidence of the historical state, leading to severe error accumulation. To address these issues, we introduce BAT3R, a training-free
Bayesian adaptation framework comprising two synergistic modules: State Innovation Gating (SIG) and
Bayesian-guided Adaptive State Updating (BASU). SIG acts as a distributional regularizer that anchors state
evolution to its
geometric initialization, effectively arresting representational drift during long-term inference. Building on this, BASU reformulates state update as a recursive
Bayesian inference process within a Kalman filtering framework. By jointly modeling the temporal reliability of observations via information-theoretic gain and spatial relevance via cross-
attention maps, BASU achieves an asymmetric spatio-temporal state fusion: well-consolidated historical regions are protected from redundant updates, while genuinely novel scene content is aggressively incor?porated. Extensive experiments on standard
benchmarks (7-Scenes, TUM, ScanNet, KITTI, etc.) demonstrate that BAT3R consistently outperforms state-of-the-art online 3D reconstruction methods, with the most significant gains observed on long sequences and complex scenes.
Read the full abstract ▾
All authors Lijian Li, Yuanpeng He, Yudian Zheng, Chung-ju Huang, Linyu Li, Dongming Jin, Wenpin Jiao, Zhi Jin, Chi-Man Pun
Computer Vision
Explicitly Aligned Multi-View Breast Cancer Risk Prediction from Variable-Length Screening Histories
Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin et al. (+5 more)
Accurate breast cancer risk prediction from screening mammography is essential for improved risk stratification and personalized screening. It can benefit from both temporal breast tissue changes across longitudinal examinations and complementary anatomical information from craniocaudal (CC) and mediolateral oblique (MLO) views. Leveraging longitudinal information requires identifying corresponding tissue regions across examinations, yet most approaches rely on implicit feature
alignment without explicitly enforcing spatial consistency. Recent work has demonstrated the benefits of explicit temporal
alignment; however, existing methods model each view independently, preventing them from exploiting complementary CC and MLO information. Furthermore, explicit approaches have been limited to a single prior examination. To fill these gaps, we propose (github.com/sot176/FlexLMV-Net), a breast-wise longitudinal multi-view framework that jointly models CC and MLO views across screening examinations while accommodating a variable number of prior examinations. FlexLMV-Net explicitly aligns longitudinal feature maps and integrates complementary views through cross-view temporal fusion and a dual-stream
attention block. Evaluated on EMBED and CSAW-CC, FlexLMV-Net consistently outperforms existing approaches across breast density, cancer subtypes, and demographic subgroups, highlighting the benefits of explicit longitudinal
alignment, multi-view modeling, and flexible use of available screening history. We further introduce (github.com/sot176/BCRBench), an open-source
benchmark that unifies state-of-the-art single- and multi-timepoint breast cancer risk prediction models under standardized preprocessing, training, inference, and evaluation protocols, enabling fair and reproducible comparison of existing methods.
Read the full abstract ▾
All authors Solveig Thrun, Zijun Sun, Suaiba A. Salahuddin, Kristoffer Wickstrøm, Elisabeth Wetzer, Stine Hansen, Robert Jenssen, Michael Kampffmeyer
Computer Vision
Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis
Hangil Park, Seungheon Kim and Tae-Kyun Kim
Training-free
language-based
video anomaly detection (VAD) uses frozen
vision–
language models (VLMs) and large
language models (LLMs) to localize abnormal events without task-specific model-weight optimization. Existing approaches increasingly exploit broader temporal and semantic context, yet generic context does not necessarily foreground abnormal events or specify how
video-level semantics should modify local anomaly scores. We introduce a training-free semantic feedback framework that uses VAD-guided active observation for
video anomaly understanding (VAU) and returns the resulting anomaly-focused description to refine temporal detection. Starting from an upstream anomaly-score curve, we adapt
reasoning-based iterative frame selection: high-score regions seed the observations, and anomaly scores together with VLM relevance assessments guide subsequent temporal exploration. A frozen
video-
language model generates a single open-ended anomaly description from the selected frames. The description is returned through two complementary feedback levels:
video-level feedback determines whether the score distribution should be suppressed, preserved, or sharpened, with selective visual rechecking, while local feedback retrieves event-related windows and promotes scores only after target-local visual verification. All model weights remain frozen, and feedback is applied once without regenerating the description. On the HIVAU
video-summary evaluations of UCF-Crime and XD-Violence, VAD-guided observation improves GPT-rated reasonability, detail, and consistency over the matched CLIP-guided variant. For VAD, semantic feedback improves the re-evaluated upstream detector from 84.36% to 86.25% AUC on UCF-Crime and from 68.07% to 72.58% AP on XD-Violence, where the final AUC is 91.74%.
Read the full abstract ▾
Computer Vision
Building age estimation based on widely available multi-source1 geospatial data
Jialu Sui, Hongsheng Zhang, Rui Liu et al. (+4 more)
Building age is fundamental for urban infrastructure safety, operation, and planning, as it directly affects structural safety, and sustainable development. Existing methods mostly rely on detailed building information such as street view, function, or materials, which are harder to obtain than geospatial data like satellite imagery and Digital Surface Models (DSMs). However, existing methods based on those geospatial data mainly rely on optical imagery and local texture features, limiting their ability to capture structural information and distinguish buildings from different periods. Moreover, the inherent temporal ordering among building age categories is often overlooked. To address these limitations, we propose the first deep-learning framework for multi-building age estimation using widely available multi-source geospatial data, including remote sensing data and their derived products, as well as building footprint information that estimates age at both pixel and instance levels within a unified pipeline. Specifically, a large pretrained backbone with fine-tuning adapters extracts representative features, and raster-derived and property-derived fusion modules support pixel-wise and instance-wise predictions. Pixel-level estimation employs a pyramid decoder with Cosine-guided Mixture-of-Experts (CMoE) to fuse multi-source raster data across scales, capturing fine-grained building age differences, while instance-level prediction uses
attention pooling and Property-Conditional Adaptive Instance Normalization (PCAdaIN) to extract instance-level representations and integrate property attributes. A Temporal Ordinal Constraint Loss (TOCL) enforces the sequential ordering of predicted ages. We also construct a building age
benchmark for Hong Kong to enable comprehensive evaluation. The source code and
datasets are available at https://github.com/littlebeen/RS-buildingAge.git.
Read the full abstract ▾
All authors Jialu Sui, Hongsheng Zhang, Rui Liu, Jing Ling, Ruisheng Wang, Hao Wu, Gang Chen
Computer Vision
RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models
Yaoyang Liu, Dongyu Yan, Yihua Du et al. (+5 more)
Video-based interactive world models aim to enable real-time, user-controllable environments. Yet most existing systems are trained and rolled out at a fixed temporal rate, limiting both runtime-rate adaptation and physical-horizon coverage – a limitation we term temporal-rate coupling. Specifically, when runtime generation throughput differs from the learned temporal rate, the generated world evolves unintentionally more slowly or quickly. Moreover, using a single temporal rate imposes an inherent trade-off between temporal granularity and physical horizon: higher rates capture finer motion but cover a shorter duration under a fixed context window. To address both limitations, we introduce Rate-Adaptive Rollout (RAR), a framework that enables interactive world models to roll out at controllable temporal rates. At its core, Stride-RoPE aligns latent frames sampled at different rates on a shared physical-time coordinate, enabling consistent multi-rate rollout, unseen-rate generalization, and online rate switching without discarding history. Meanwhile, RAR uses multi-stride
video sampling to enable joint learning of fine-grained dynamics and long-horizon
evolution. Across action- and camera-controlled environments, RAR achieves temporally consistent multi-rate rollout and online rate switching while preserving high-rate generation quality and extending physical horizons under matched training budgets. RAR also transfers to pretrained world models while preserving perceptual quality.
Read the full abstract ▾
All authors Yaoyang Liu, Dongyu Yan, Yihua Du, Shengju Qian, Zeke Xie, Xin Wang, Ying-Cong Chen, Long Chen
Computer Vision
Learning Temporal Evidence Selection for Video Anomaly Understanding
Hangil Park, Seungheon Kim and Tae-Kyun Kim
Video anomaly understanding (VAU) aims to generate
language-level interpretations of anomalous events in long
videos, requiring visual evidence that supports explanation beyond anomaly detection alone. Existing VAU pipelines commonly rely on anomaly-centric or predefined temporal sampling strategies, which efficiently emphasize suspicious moments but overlook contextual cues, object interactions, and temporally distributed evidence needed for
reasoning. Motivated by recent progress in learnable temporal selection for long-
video question answering, we revisit VAU temporal sampling as a downstream-aligned evidence selection problem. We introduce the semantic-conditioned Temporal Sampling Q-Former that learns to select temporal visual evidence through the VAU generation objective. Since generic VAU
prompts provide weak
video-specific guidance, we condition the sampler on CLIP-embedded object tags automatically extracted from the input
video. These tags are used only for temporal evidence selection and are not appended to the LLM generation
prompt, separating sampler conditioning from
prompt augmentation. The proposed sampler operates in visual-
token space, computes soft temporal selection weights from frame-level representations, and merges duplicated selections before response generation. Experiments on HIVAU-70K show that our model achieves the best performance among the compared fine-tuned models, while ablations demonstrate that learnable temporal sampling improves over fixed anomaly-focused sampling and that CLIP-embedded object-tag conditioning further improves over generic-
prompt conditioning. Our results suggest that temporal sampling in VAU should be aligned with explanation-oriented evidence utility rather than anomaly saliency alone.
Read the full abstract ▾
Computer Vision
ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusion
Jens Lundell, Alberta Longhini, Lei Shu et al. (+6 more)
Garment manipulation requires inferring both garment identity and complete configuration from partial, self-occluded observations. This creates a chicken-and-egg problem: completing a partial observation requires a garment-specific deformation prior, while identifying the garment requires a configuration that explains the observation. We introduce ClothAtlas, which jointly identifies garments and estimates their complete configurations from partial point clouds and a library of canonical meshes. A shared graph-conditioned flow-matching model samples configurations for each candidate mesh and ranks candidates by observation fit. The model accommodates different mesh shapes, vertex counts, and connectivities, allowing new garments to be added from canonical meshes alone without retraining. On held-out DexGarmentLab garments, ClothAtlas ranks the correct mesh first among 221 candidates for all 22 test garments from a single flat observation. From a single self-occluded observation, it identifies 15/22 garments, compared with 8/22 for a deterministic graph
estimator. Persistent vertex identities support grasp-target localization on real garments, reducing mean error from 15.4 to 6.4 cm overall and from 17.0 to 8.3 cm on occluded targets compared with dense descriptor matching. Project page: https://clothatlas.github.io/
Read the full abstract ▾
All authors Jens Lundell, Alberta Longhini, Lei Shu, Hemal Arora, Evan Robert, Carlota Parés-Morlans, Marcel Büsching, Mårten Björkman, Jeannette Bohg
Computer Vision
3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design
Yueling Liao, San Jiang, Xing Zhang et al. (+1 more)
3D Gaussian Splatting (3DGS) has been becoming a promising 3D reconstruction technique because of its advantage of high-fidelity rendering effect with high efficiency. However, it still suffers from coarse density control and semantically blind loss functions when reconstructing the fine-grained structures in complex urban scenes. Thus, this study proposes SG-GS, a semantic-guided 3DGS method for fine-grained structure reconstruction. It takes the semantic segmentation deduced by the Segment Anything Model (SAM) as a constraint and systematically integrates it into both Gaussian density control and loss function design. On the one hand, a semantic-constrained adaptive density control mechanism is constructed, which dynamically allocates gradient thresholds to different semantic regions of the scene and introduces an additional boundary-enhanced densification strategy, so that new Gaussians are directed to structurally critical regions. On the other hand, a multi-feature guided loss function is designed, and fine-grained supervision signals are gradually introduced through a phased progressive training strategy, alleviating the artifacts caused by the lack of semantic constraints in the basic loss. Experiments on three
datasets show that the proposed method consistently improves reconstruction accuracy. Compared with the original 3DGS, the PSNR increases by approximately 2.7% to 5.2%, while artifacts on linear structures and boundary blurring are effectively mitigated. The method can serve the fine-grained 3D reconstruction of urban scenes.
Read the full abstract ▾
All authors Yueling Liao, San Jiang, Xing Zhang, Zhongwen Hu
Computer Vision
ProGAN-Based Synthetic Image Generation for Industrial Visual Inspection of Metallic Parts
Hossein Omid Beiki, Sepehr Asgari, Oliver Petrovic et al. (+1 more)
Automated visual inspection is essential for efficient repair and recycling of metallic components. AI-based defect detection is a strong fit. However, it is often constrained by scarce, low-diversity training data. To address this gap, we study Generative AI-driven synthetic image generation using Progressive Growing GANs (ProGAN). We implement a ProGAN pipeline with Wasserstein loss and gradient penalty and evaluate key hyperparameters on two public
datasets, MVTec AD (metal nut) and MPDD (tube). As a result, training remained stable and the model yielded realistic images. These images can augment training data, improve robustness, and reduce the cost of collecting rare defects.
Read the full abstract ▾
All authors Hossein Omid Beiki, Sepehr Asgari, Oliver Petrovic, Christian Brecher
Computer Vision
DepthTile: Depth-Guided Existing-Image Self-Tiling with Input-Side Tileability Assessment
Rui Wang, Xi Chen, Hui Dai et al. (+4 more)
Existing-image self-tiling aims to transform a given image into a seamlessly repeatable tile while preserving its original semantics and local details. This task requires weak editing to retain the source but leave periodic seams, whereas strong editing improves tileability at the large risk of content drift.Existing
diffusion-based tiling pipelines alleviate boundary discontinuities, yet lack awareness of which regions are geometrically important to preserve. In this work, we propose DepthTile, a depth-guided self-tiling method whose generation module requires no retraining of the
diffusion backbone. It converts monocular depth into a spatial preservation prior and integrates it into the timestep-dependent masking mechanism of Tiled
Diffusion. DepthTile converts depth into a spatial editability map and fuses it with the differential
diffusion mask. This strengthens source preservation in near-field regions indicated by the monocular depth prior while retaining editing flexibility near compatible background boundaries. To access when self-tiling is feasible, we introduce the Self-Tiling Ability Score (STAS), a lightweight input-side predictor based on RGB, gradient, depth, and depth-gradient cues. On a 335-image
benchmark, relative to the default Tiled
Diffusion pipeline, DepthTile reduces MAE, LPIPS, and Tiling Score by 12.8%, 26.7%, and 8.4%, respectively. STAS reliably enables difficulty-aware analysis and further supports compatibility-aware pairwise image tiling.
Read the full abstract ▾
All authors Rui Wang, Xi Chen, Hui Dai, Piaopiao Yu, Yixue Hao, Long Hu, Min Chen
Computer Vision
FlowAvatar: Real-Time Full-Body Avatars from Sparse Egocentric Inputs on Consumer XR Devices
Chenfeng Gao, Taeyoung Yeon, Sungheon Park et al. (+4 more)
Prior avatar embodiment research has studied tracking completeness (how faithfully the body, hands, and face are sensed and reproduced) and personalized appearance in isolation, each requiring specialized lab equipment. How these factors interact on consumer hardware remains an open question, because no existing system unifies body pose, face, hands, and personalized appearance on a standalone headset. We present FlowAvatar, a system enabling real-time personalized look-alike avatars with full-body pose, hand articulation, facial expression, and eye gaze tracking from sparse egocentric inputs on consumer XR devices (Meta Quest Pro). Our pose model achieves competitive spatial accuracy and state-of-the-art temporal consistency on the AMASS
benchmark against prior works with comprehensive spatial and temporal losses validated through architecture and loss ablations, while running in real-time. Two user studies (N=12, N=10) provide converging evidence that tracking completeness and personalized appearance serve complementary roles: tracking completeness broadly improves perceived motion quality, while only their combination achieves positive total embodiment.
Read the full abstract ▾
All authors Chenfeng Gao, Taeyoung Yeon, Sungheon Park, Vasco Xu, Anish Prabhu, Mar Gonzalez-Franco, Karan Ahuja
Computer Vision
Graph3R: Unifying View Graph Initialization with Feed-Forward Reconstruction
Jingwen Yu, Hengli Wang, Jianhao Jiao et al. (+3 more)
Feed-forward reconstruction recovers camera poses and dense geometry from image sets whose views are assumed to come from a single scene. In practice, uncurated image collections often contain views from disjoint scenes, and reconstructing these views jointly can corrupt camera poses and geometry. Existing pipelines typically resolve view associations in separate stages before final reconstruction, leaving view graph initialization and reconstruction inference only loosely coupled. We introduce Graph3R, which unifies view graph initialization with feed-forward reconstruction through a shared
geometric representation. A lightweight verification head predicts pairwise covisibility from the multi-view features of $\pi^3$ to construct a view graph. This graph classifies the input views into scene groups. For each group, we mask the shared features before decoding camera poses and dense geometry. Reusing the multi-view features allows Graph3R to reconstruct disjoint scenes separately with a single backbone pass. Across three
benchmarks, Graph3R improves average maximum recall at 100% precision by upto 8.99 percentage points over the baseline, while improving reconstruction quality on mixed-scene inputs. Regarding the multi-session mapping task, Graph3R improves cross-session localization and achieves competitive pose estimation accuracy, while delivering a runtime speedup of approximately $35.9\times$ over the baseline when reconstructing the query group. By integrating learned view association with
geometric inference, our approach takes a step toward end-to-end reconstruction from uncurated image collections.
Read the full abstract ▾
All authors Jingwen Yu, Hengli Wang, Jianhao Jiao, Zhonghang Liu, Ping Tan, Hong Zhang
Computer Vision · Cross-listed from cs.AI
Hardware-Friendly Token-Group Activation Quantization for Low-Bit Mamba Super-Resolution
Jinwoo Chung and Jangho Kim
Mamba-based super-resolution models are efficient alternatives to
Transformer-based restoration backbones, but their low-bit post-training quantization remains challenging because selective scan, Mamba's input-dependent state-space operator, induces different activation ranges across spatial
tokens. Thus, a single layer-wise clipping range poorly matches all
tokens. We propose
Token-Group Quantization, a hardware-friendly activation PTQ framework that statically approximates a hardware-unfriendly post-scan grouping oracle, where
tokens with similar preferred clipping ranges are assigned to the same group. To avoid runtime post-scan grouping in static low-bit inference, a lightweight predictor infers the groups from pre-scan features available before the quantized scan path. During calibration, we form pseudo-labels by clustering post-scan activation descriptors that capture similar rounding–clipping behavior. At inference, the frozen predictor maps pre-scan features to groups and selects entries from offline-calibrated group-wise clipping-bound tables, without runtime statistics collection, dynamic clipping updates, or post-scan group assignment. To address low-bit restoration degradation, we further introduce a Frequency-Preserving Refinement (FPR) objective for clipping-bound calibration. FPR aligns
Fourier amplitude and phase between full-precision teacher and quantized student outputs to preserve high-frequency structures. Across PTQ protocols, bit-widths, backbones, and tasks, our method consistently outperforms prior Mamba and SR PTQ baselines, transfers to
Transformer SR and
Vision Mamba classification, composes with rotation-based quantization, and achieves 2.23x speedup on Jetson Orin Nano over FP16. Code is available here.
Read the full abstract ▾
Computer Vision
EgoWholeView: Camera-Only Head-Mounted Capture of Body and Bimanual Hand Motion for Embodied Demonstrations
Yaoqing Hu, Xuguo He, Daohan Cui et al. (+2 more)
Head-mounted body and bimanual motion capture supports self-avatars, teleoperation, and embodied demonstrations. However, a single egocentric camera is vulnerable to a limited field of view, scale ambiguity, and hand-object occlusion. We present EgoWholeView, a wearable capture system of six synchronized wide-angle RGB cameras with a role-specialized pipeline. The outward views localize the rig in a world frame, while an overlapping downward pair independently predicts SMPL-X body and MANO hand parameters. Both predictors are adapted to egocentric fisheye data. To combine the two, reliability cues measured from the predictions themselves weight the fusion of body and hand joint rotations, while shape is aggregated across views and frames. The pair's baseline fixes metric scale, and triangulation recovers root translation. Relative to monocular prediction, fusion reduces PA-MPJPE by 3.7% for the body and 11.2% for the hands, and temporal refinement lowers angular-acceleration magnitude by 32.2% and 31.0%, respectively. On our six-camera recording, the pipeline produces body and hand states in a shared world frame using
benchmark-fitted gates without retuning, and temporal refinement lowers hand acceleration magnitude by 41.08%. Using only RGB cameras and no third-person view, EgoWholeView offers a low-cost, portable route to egocentric interaction data.
Read the full abstract ▾
All authors Yaoqing Hu, Xuguo He, Daohan Cui, Yukun Shi, Kai Chen
Computer Vision
MFlowHOI: Whole-Body Human-Object Interaction Generation via Part-Aware MeanFlow
Qiang He, Yonghao Zhang, Yinda Zhang et al. (+7 more)
Whole-body human-object interaction (HOI) generation requires coordinating body and finger motion, object trajectories, and time-varying contacts. Existing methods typically rely on iterative
diffusion or test-time optimization, resulting in high inference costs. Direct one-step generation is highly desired in downstream applications due to efficient inference. However, it is challenging because these HOI components differ substantially in structure and scale. We present MFlowHOI, an efficient one-step framework for whole-body HOI generation. A Contact-Aware Multipart RVQVAE organizes body, hand, object, and contact signals into a structured latent space, with a shared decoder preserving their global coordination. An ST-DiT then models temporal and cross-part dependencies in this space and generates a complete interaction in a single MeanFlow evaluation. Predicted object-local contact correspondences further enable
geometric refinement and interaction-preserving transfer to photorealistic 3D Gaussian avatars with different body proportions. Experiments demonstrate that MFlowHOI outperforms existing base generators in motion quality and text
alignment while substantially reducing inference time.
Read the full abstract ▾
All authors Qiang He, Yonghao Zhang, Yinda Zhang, Chenyu Meng, Liufei Ouyang, Haoxiang Hu, Cangjun Gao, Xiaoming Deng, Cuixia Ma, Hongan Wang
Computer Vision
Seg2Click: Learning Pixel-Level Action Masks for GUI Visual Grounding
Jingchao Wang and Dingjiang Huang
Visual grounding is a central challenge for GUI
agents, requiring an instruction to be mapped to a target within a crowded screenshot. Coordinate generation with single-point supervision reduces a spatially extended target to one position, despite the presence of many valid click locations. Region-based patch scoring alleviates this ambiguity, but its spatial decisions remain tied to visual-
token footprints that may span the target, neighboring controls, and background. To address these limitations, we propose Seg2Click, which reformulates GUI grounding as executable-region prediction: identifying where a click can validly execute the given instruction. Using bounding-box supervision, Seg2Click predicts an executable-region mask and computes the final click point through a deterministic rule. This formulation captures the spatial extent of valid click locations and enables finer localization beyond the visual-
token grid, making it well suited to small, densely arranged controls without requiring a verifier or zoom-in pass. Experiments across five backbone configurations from two model families demonstrate the effectiveness of Seg2Click. Seg2Click-9B achieves 92.06%, 94.10%, and 55.22% on ScreenSpot, ScreenSpot-v2, and ScreenSpot-Pro, respectively, while matched-backbone comparisons show improvements over both coordinate generation and patch-based grounding. Code will be available at https://github.com/jcwang0602/Seg2Click.
Read the full abstract ▾
Computer Vision
Look Back to Move Forward: Learning to Revisit for Agentic Compositional Image Generation
Zixu Wang, Xiaoran Zhao, Yahui Liu et al. (+8 more)
Text-to-image models have made remarkable progress in visual fidelity and instruction following, yet complex compositional generation remains challenging. Existing agentic approaches address this limitation through iterative generation and editing, adapting their actions based on interaction history. However, they typically fix the continuation state (i.e., the image from which refinement proceeds) to the latest image. Since iterative editing is not necessarily monotonic, the
agent may continue refining from the latest image even when it has already degraded. In this work, we introduce GenRevisit, an agentic framework that extends action selection to joint continuation-state and action selection. The
agent can not only use interaction history to adjust editing instructions but also select an image candidate to edit, jointly deciding where to continue from and how to refine. We train GenRevisit with SFT followed by agentic reinforcement learning. To address the coarse credit assignment of final outcome reward in GRPO, which assigns the same trajectory-level advantage to all actions, we further propose Normalized Action Credit Assignment (NACA) to redistribute positive advantage according to each action's contribution to reducing the remaining objective. On GenEval2, GenRevisit achieves GM scores of 84.3 and 74.4 with Qwen-Image-2512 and Z-Image-Turbo, surpassing the strongest baselines using the same generators by 12.7 and 10.9 points respectively. GenRevisit achieves state-of-the-art performance on GenEval, GenEval++, and GenEval2 among baselines.
Read the full abstract ▾
All authors Zixu Wang, Xiaoran Zhao, Yahui Liu, Feng-ao Wang, Junhao Xiao, You Shu, Bing Liu, Yiting Liu, Songlei Jian, Zhiliang Tian, Dongsheng Li
Computer Vision
MP-MNCA: Morphology-Prior Masked Neighbor Cross-Attention for Efficient Spatial Domain Identification
Sidharthan Sivacoumar Chitra and Sanjeev Manivannan
Spatial transcriptomics jointly captures molecular state, tissue morphology, and spatial organization, yet spatial-domain methods are typically compared only at late training checkpoints, obscuring differences in convergence efficiency. We introduce MP-MNCA, a morphology-prior masked neighbor cross-
attention framework that preserves the full 3,000 dimensional highly-variable-gene representation while selectively routing molecular information across spatial neighbors, with histological similarity incorporated as a continuous
attention prior under a self-supervised masked neighbor-contrastive objective. Across 12 heterogeneous human dorsolateral prefrontal cortex (DLPFC) sections, MP-MNCA is evaluated against established spatial-domain baselines using both convergence trajectories and epoch-wise spatial-domain visualizations. Our results show that MP-MNCA (i) reaches coherent spatial-domain structure in substantially fewer training epochs, as reflected quantitatively by convergence curves and qualitatively by progressive spatial grids, and (ii) maintains strong clustering quality, achieving a mean peak refined ARI of 0.546 versus 0.549 for the strongest baseline (STAIG), while attaining the highest mean refined NMI of 0.657 versus 0.640. Moreover, MP-MNCA outperforms STAIG on 7 of 12 sections in ARI and 11 of 12 in NMI, demonstrating faster convergence without sacrificing spatial-domain quality.
Read the full abstract ▾
Computer Vision
VGGA: Geometry-Grounded Autoencoder for Video Generation
Chaofan Gan, Yian Zhao, Shijie Li et al. (+4 more)
Autoencoders are the core component of latent
video generative models, compressing high-dimensional
videos into compact latent spaces for downstream generation. However, existing
video autoencoders are primarily optimized for pixel-level reconstruction, providing limited supervision for learning structured latent representations. We argue that
videos should not be treated merely as signals to be reconstructed, but rather as 2D projections of dynamic 3D worlds that exhibit explicit and well-structured
geometric properties. Building on this insight, we propose
Video Geometry Grounded Autoencoder (VGGA), a geometry-aware
video autoencoder that explicitly incorporates
geometric constraints into the reconstruction process. Specifically, VGGA grounds the learned representations along three complementary
geometric dimensions: Structural Depth, Motion Correspondence, and Projective Warping. These objectives are unified within a
Video Geometry Decoder, which employs a shared spatio-temporal upsampling trunk to jointly optimize
geometric learning and RGB reconstruction. Through these geometry-grounded objectives, VGGA encourages the latent representations to capture the underlying 3D structure and motion of dynamic scenes. VGGA consistently improves
video generation across both geometry-centric and human-motion domains, reducing gFVD by 28.1% on RealEstate10K and 37.7% on Taichi-HD. Notably, these substantial generation improvements are achieved while maintaining comparable reconstruction quality, demonstrating that geometry-grounded representation learning yields a more effective latent space for
video generation.
Read the full abstract ▾
All authors Chaofan Gan, Yian Zhao, Shijie Li, Tieyuan Chen, Zicheng Zhao, Mehrtash Harandi, Weiyao Lin
Computer Vision
DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentation
Zhifan Zeng, Wenbo Xiao, Arcot Sowmya et al. (+1 more)
Open-vocabulary 3D Gaussian segmentation typically binds semantic features to the per-Gaussian state via per-scene gradient training (1-4 h) or the more recent closed-form
distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard decision that degrades
retrieval accuracy and yields unstable mask boundaries at render time. We present DIGS, a training-free framework with two stages, class-agnostic mapping and multi-modal encoding, that decouples instance discovery from linguistic naming. At the Gaussian level, our distribution-informed representation maintains an explicit top-$M$ label distribution per primitive, updated by a tempered Dirichlet rule; preservation of multi-peaked label distributions at boundary primitives allows a late-argmax renderer to recover clean instance boundaries. A bidirectional state machine aggregates these distributions into a persistent class-agnostic instance ontology. The instance-level encoding stage then fuses the CLIP visual
embedding of each instance's isolated 3D footprint with the CLIP-text
embedding of an LLM-generated attribute description, disambiguating visually similar instances. On a single A100, DIGS completes mapping and instance encoding on LERF-OVS in 1.0-3.2 min per scene, depending on whether LLM fusion is enabled, and serves native 3D queries in 5.4 ms. It reaches 71.95% mean mIoU on LERF-OVS, outperforming both the training-free SFS and training-based LaGa baselines. DIGS also achieves the highest mean mIoU on LERF-Mask and 3D-OVS and surpasses Gaga and ObjectGS on Replica under their respective evaluation protocols. Code is available at https://github.com/ZephyrZos/DIGS.
Read the full abstract ▾
All authors Zhifan Zeng, Wenbo Xiao, Arcot Sowmya, Changming Sun