Computer Vision
Monte Carlo Energy Aggregation for Mobile 3D Gaussian Splatting
Xiaobiao Du, YuAn Wang, Hao Li et al. (+3 more)
Recent advances in 3D Gaussian Splatting have demonstrated unprecedented success in novel view synthesis. However, the substantial inference and storage overhead driven by high-order Spherical Harmonics (SH) are primary bottlenecks for mobile platforms. In this paper, we present Flux-GS, a real-time Gaussian Splatting method designed to achieve high-fidelity rendering with significantly reduced overhead for resource-constrained mobile platforms. We first propose a Monte Carlo Specular Energy Aggregator, sampling third-order radiance residuals and aggregating specular energy into a compact latent space. In this way, our method effectively preserves visually salient lighting features in lower-order bands without expensive
distillation or pre-training. To mitigate the high-frequency details lost during compression, we introduce an Attribute-Conditioned SH Enhancement module. This module predicts Gaussian-aware offsets based on intrinsic Gaussian attributes, which enhance the first-order SH representation prior to inference, without extra inference costs. Furthermore, the original single-view gradient-based densification is prone to producing excessive Gaussians and overfitting to a certain view. We address these limitations by proposing a Multi-view Alpha-based Densification and
Pruning strategy. By leveraging multi-view guidance, we ensure multi-view structure consistency and the precise removal of redundant primitives. Extensive experiments demonstrate that Flux-GS achieves substantial parameter reduction while maintaining competitive visual quality, offering a robust and scalable solution for real-time mobile rendering. Code: \textcolor{magenta}{\href{https://xiaobiaodu.github.io/flux-gs-project/}{https://xiaobiaodu.github.io/flux-gs-project/}}.
Read the full abstract ▾
All authors Xiaobiao Du, YuAn Wang, Hao Li, Bosheng Wang, Xun Sun, Xin Yu
Computer Vision
Shell-Supervised Gaussian Splatting for Urban Real-to-Sim Reconstruction
Yuan Yang, Peijun Lu, Fangzhou Lu et al. (+5 more)
Real-to-sim reconstruction for
embodied AI requires geometry that is useful for collision
reasoning,
navigation, and
agent-environment interaction, not only photorealistic novel-view synthesis. However, close-range urban facades are difficult for
video-to-3D reconstruction: glass, reflections, repeated windows, and weak texture can produce visually plausible renderings with unstable surface geometry. We introduce shell-supervised Gaussian Splatting, a reconstruction-stage framework that uses an external facade structural shell as lightweight
geometric supervision for
video-driven Gaussian reconstruction. The method aligns an exterior shell to the
video reconstruction frame, renders per-view depth, camera-space normal, and valid-mask maps, and applies these cues through mask-gated losses during Gaussian optimization. This design preserves RGB-driven appearance while regularizing only visible shell-supported facade regions. Experiments on anonymized close-range urban facade scenes show improved facade orientation and visible-surface point-cloud consistency over photo-only,
monocular-cue, and surface-oriented Gaussian baselines, while maintaining comparable held-out rendering quality.
Read the full abstract ▾
All authors Yuan Yang, Peijun Lu, Fangzhou Lu, Sai Fan, Siqi Yan, Chenyuan Zhang, Haobo Liang, Yichen Wang
Computer Vision
SkelEM: Training-Signal Decoupling of Skeleton and Diffusion for Self-supervised Axial Super-Resolution in Volume Microscopy
Bohao Chen, Yanchao Zhang, Yanan Lv et al. (+3 more)
Volume microscopy, including electron and light microscopy, suffers from severe anisotropic resolution due to physical axial sectioning. Existing self-supervised axial super-resolution (ASR) methods face a trilemma bounded by overly smoothed regression textures, structural
hallucinations of pure
diffusion models, and prohibitive inference latency. In this paper, we propose Skeleton-refinE Microscopy (SkelEM), a self-supervised framework that decouples ASR at the training-signal level: a frozen topological network and a
diffusion refiner are optimized by disjoint objectives, separating low-frequency
topology formulation from high-frequency detail enhancement. Building on this deterministic skeleton, we exploit a unified cycle-consistent mechanism on input sparse slices to simultaneously extract a real-domain residual prior and bidirectionally align the
diffusion refiner, washing away cross-plane artifacts without synthetic bias. By truncating the reverse
diffusion process with this physical prior, SkelEM achieves high-fidelity detail restoration in merely $\le 5$ steps. To rigorously assess cross-instrument generalization, we further introduce BRAVE-ASR, a new
benchmark of co-aligned anisotropic and isotropic volumes acquired on a
Plasma-FIB instrument. Across public
benchmarks, SkelEM achieves the most favorable balance across the fidelity-perception trade-off among self-supervised methods, with state-of-the-art downstream membrane
segmentation performance and robust zero-shot generalization across distinct modalities.
Read the full abstract ▾
All authors Bohao Chen, Yanchao Zhang, Yanan Lv, Chenxun Deng, Hua Han, Xi Chen
Computer Vision
GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoising
Yi He, Jiangming Wang, Xinyu Wang et al. (+5 more)
Precisely manipulating objects in a single photograph (translation, rotation, scaling) while obeying 3D physical constraints remains unsolved for
diffusion-based editors. Current 2D methods lack spatial awareness and produce perspective violations. Forcing structural proxies into the latent space also disrupts variance homogeneity, and the resulting self-
attention leakage leads to ghosting and background blur. The core difficulty is asymmetric: the relocated object must follow a rigid geometry, yet the uncovered background needs freedom to synthesize plausible content. We present GeoEdit, a training-free Lift-Manipulate-Render-Denoise pipeline that satisfies both constraints. We decouple scene and object in 3D, align them through point correspondence, and render a geometry-aligned proxy with a structural depth map. A Dual-Branch Denoising stage then refines this proxy: a
video diffusion backbone preserves object identity, while 3D constraints are injected into the foreground within a narrow denoising window at matching noise variance (variance-homogeneous injection). The background denoises freely. Because the injected signal matches the native latent statistics, self-
attention stays undisturbed. We also introduce GeoEditBench, a pose-aware
benchmark covering object translation, object rotation, and camera movement with pose-aware evaluation metrics. Experiments confirm consistent gains in
geometric accuracy, identity fidelity, and background quality. Our codes are available at https://github.com/Heey731/GeoEdit.
Read the full abstract ▾
All authors Yi He, Jiangming Wang, Xinyu Wang, Mark Fong, Songchun Zhang, Yuxuan Xue, Hai-Tao Zheng, Yue Ma
Computer Vision
SICAGE: Speaker-Independent Culture-Aware Gesture Generation using TED4C-L Dataset
Ariel Gjaci, Antonio Sgorbissa and Vittorio Murino
Recent co-
speech gesture generation methods often overlook cultural differences, limiting their effectiveness in human-
agent interaction. Moreover, culture-conditioned models are rarely evaluated under speaker-disjoint splits, so apparent "cultural" behavior may be confounded with speaker-specific gesturing style. We introduce SICAGE, a modular framework for culture-aware co-
speech gesture generation that conditions motion synthesis models on speaker-independent cultural representations. SICAGE learns these representations from audio and text by treating each speaker as a separate domain while imposing invariance across speakers. This encourages representations to remain culture-discriminative while reducing dependence on speaker identity. The resulting cultural
embeddings condition a
multimodal generator to produce culturally appropriate gestures. We instantiate this idea with two domain generalization approaches:
adversarial learning and Fishr regularization. We further introduce ALaDiT, a real-time
diffusion-based gesture generator designed to efficiently incorporate the learned cultural
embeddings. To validate our method, we built TED4C-L, a 106-hour
multimodal dataset of 764 TED speakers from four cultural groups. Experiments show that SICAGE improves motion realism, diversity, beat synchronization, semantic relevance, and cultural consistency.
Read the full abstract ▾
Computer Vision
Rigel: Self-Distilled Score Adaptation for Image and Video Captioning Evaluation
Shuitsu Koyama, Kazuki Matsuda, Yuiga Wada et al. (+3 more)
Automatic evaluation of image and
video captioning is essential for benchmarking
multimodal systems, although standard evaluation metrics show limited
alignment with human judgments. Recent approaches using large
language models (LLMs), commonly referred to as LLM-as-a-Judge, have improved
alignment with human judgments but still suffer from a mismatch between large-vocabulary
language modeling and evaluation over a small label set. To address this, we propose Rigel, an automatic evaluation metric for image and
video captioning, based on self-distilled score adaptation. The metric employs an evaluation-specific scoring head distilled from a frozen LLM, which captures judgment signals in a task-aligned space without relying on large-vocabulary
token sets. We then refine the LLM backbone with human judgment data. To train Rigel, we constructed the Vid-Lepus
dataset, which contains 3,338
video clips, 33,380 reference captions, and 5,637 candidate captions. Experiments on multiple
benchmarks show that Rigel outperforms state-of-the-art metrics, achieving over 10-point improvements on ActivityNet-Fact in the reference-free setting.
Read the full abstract ▾
All authors Shuitsu Koyama, Kazuki Matsuda, Yuiga Wada, Shinnosuke Hirano, Daichi Yashima, Komei Sugiura
Computer Vision · Cross-listed from eess.IV
A multi-architecture study of specificity refinement and false-positive mechanism analysis in prostate MRI
Yongbo Shu, Kewen Chen, Yifeng Yuan et al. (+5 more)
Objectives: To characterize residual false positives in prostate MRI detection, and to evaluate a lightweight post-hoc refinement head for case-level specificity. Materials and Methods: This retrospective study used PI-CAI (5-fold cross-validation) and Prostate158 (n=158; external). A context-aware evidence head and an 89,216-parameter refinement head were trained on a frozen detection backbone; the evidence head was also trained on four further backbones (bare nnU-Net, bare U-Net, bare Mamba, MIGF-Mamba). For each false-positive region, T2-weighted, apparent-
diffusion-coefficient, and high-b-value contrast ratios versus peri-lesional rings were compared against ground-truth lesions and contralateral benign regions. Results: False positives were closer to true cancers than to benign tissue in evidence and raw T2-weighted and apparent-
diffusion-coefficient contrast, reproducing 35/35 across five architectures (Cohen's d 1.10; FP/benign evidence ratio 2.38x) and 105/105 across modality-perturbation scenarios. On PI-CAI fold-0, refinement raised case-level specificity from 0.469 to 0.549 (+17.2%) at preserved sensitivity (0.943); 5-fold cross-validation showed fold-conditional behavior (9/15 observations positive; range -22% to +28%). On Prostate158, both models saturated (McNemar pooled p=0.69), while the false-positive contrast-matching finding replicated. Conclusion: Residual false positives are contrast-matched to cancer (sharing raw imaging features rather than histologically confirmed mimicry), reproducing across five architectures – a data-level imaging property, not model-specific artifacts; post-hoc refinement adds practical specificity in-domain but is fold-conditional.
Read the full abstract ▾
All authors Yongbo Shu, Kewen Chen, Yifeng Yuan, Zirui Xin, Luo Lei, Yang Yang, Xi Chen, Aijing Luo
Computer Vision
Learning Motion-Expressive 4D Gaussian Representations from Monocular Videos with Flow Splatting
Jinzhao Li and Miao Liu
Reconstructing dynamic 3D scenes from
monocular videos remains challenging because limited observations provide insufficient motion constraints. Conventional 4D Gaussian representations model spatiotemporal appearance efficiently, but the restricted temporal
evolution of each primitive makes complex nonlinear dynamics difficult to represent compactly. We propose a motion-expressive 4D Gaussian representation with time-varying means and covariances. By analytically differentiating Gaussian trajectories, we obtain Gaussian velocities and introduce Flow Splatting to render optical flow while accounting for camera motion, enabling dense motion supervision. This formulation jointly models scene appearance and motion within a unified set of 4D Gaussian primitives. We further introduce an initialization strategy based on the sampling
theorem and a velocity consistency regularizer to stabilize optimization. Experiments on DAVIS and NVIDIA Dynamic Scenes demonstrate a favorable trade-off between reconstruction quality and efficiency under challenging
monocular settings.
Read the full abstract ▾
Computer Vision
Stratified Multi-View Aggregation for Score Distillation
Marian Lupascu, Mihai-Sorin Stupariu, Wongi Park et al. (+1 more)
Score
distillation turns a pretrained 2D
diffusion model into a 3D generator, but the per-step gradient is estimated from a single random view: this one-sample estimate has high variance (different views of the same partial scene disagree) and is blind to global shape consistency. Existing multi-view approaches address this by retraining the
diffusion prior on multi-view data; this improves consistency but conflates the sampling contribution with the quality of the retrained prior. We instead isolate the sampling axis, leaving the prior frozen. We introduce Multi-View Aggregated Score
Distillation (MV-SDI), a training-free sampler that replaces the single-view per-step gradient with an average over K views at a fixed UNet-call budget. Averaging K views lowers the per-step gradient variance toward 1/K of its single-view value. Drawing the K views as antithetic antipodal pairs adds no further variance reduction (measured antipodal correlation rho approximately 0) but stratifies angular coverage (every step covers both hemispheres) removing the same-hemisphere clustering of independent sampling. At a fixed 10,000-UNet-call budget on the 43-
prompt SDI
benchmark, K=2 halves the optimization steps and raises CLIP R-Precision from 74.8% to 83.8% and CLIP score from 0.297 to 0.312 over the single-view SDI baseline, with consistent gains on HPSv2 and ImageReward and a 0.0% divergence rate. K=4 gives a fourfold step reduction at R-Precision 86.9% and CLIP 0.307. The gains concentrate on hard
prompts where single-view
distillation collapses, at a measured cost in CLIP-IQA. MV-SDI is drop-in for gradient-based score-
distillation pipelines, including Score
Distillation via Inversion and plain SDS, and requires no retraining and no multi-view data. Code is available at: https://github.com/marianlupascu/MV-SDI
Read the full abstract ▾
All authors Marian Lupascu, Mihai-Sorin Stupariu, Wongi Park, Ionut Mironica
Computer Vision
Explainability-Aware Frustum Attack: Exposing Structural Vulnerabilities in LiDAR-Based 3D Object Detectors
Chengzeng You, Binbin Xu and Soteris Demetriou
The structural vulnerabilities of point cloud-based 3D object detectors remain poorly understood. Prior work has studied
adversarial robustness primarily on isolated 3D object models, while recent
LiDAR spoofing attacks target richer and more realistic driving scenes but focus mainly on physical realizability rather than understanding detector behavior or attack efficiency. In this work, we investigate how
LiDAR-based detectors rely on spatial evidence in complex scenes and whether these reliance patterns can be exploited to induce failures more efficiently. To this end, we propose an explainability-guided
adversarial analysis methodology. We introduce the Saliency-
LiDAR (SALL) method, which aggregates Integrated Gradient attributions across scenes to produce universal saliency maps for
LiDAR-based 3D object detectors. Guided by these maps, we design the Explainability-aware Frustum Attack (EFA), which selectively perturbs only the most influential frustums rather than uniformly attacking entire object regions. Experiments on KITTI and nuScenes, across detectors such as PointPillars and SECOND, show that EFA reduces detection recall by more than 15 percentage points while requiring 25-50% fewer perturbed frustums than the state-of-the-art non-saliency-aware baseline. These findings reveal that modern 3D detectors concentrate discriminative evidence in a small subset of spatial regions, exposing a structural robustness vulnerability in current
LiDAR perception systems. Our code is released at https://github.com/SecMindLab/Saliency_
LiDAR.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Exploiting Local Flatness for Efficient Out-of-Distribution Detection
Seonghwan Park, Hyunji Jung, Dongyeop Lee et al. (+1 more)
Detecting out-of-distribution (OOD) data is crucial for reliable machine learning deployment. Among detection strategies, post-hoc methods are particularly attractive due to their efficiency, as they operate directly on pre-trained networks without requiring retraining. Within this paradigm, one promising direction exploits loss-landscape
curvature to estimate model uncertainty; however, such methods incur substantial computational cost and rely on implicit assumptions about how landscape flatness differs between in-distribution (ID) and OOD data. In this work, we provide the first systematic investigation of this
curvature discrepancy and show that OOD inputs exhibit larger Hessian
curvature than ID data, with the gap widening under stronger distributional shifts. Motivated by these observations, we propose Fold, a lightweight flatness-modulated OOD detector that leverages the feature Hessian and partial feature normalization to improve ID-OOD separability while avoiding costly parameter-space
curvature approximations. To optimally adapt this normalization across diverse
datasets, we further introduce AutoFold, a self-supervised tuning scheme that synthesizes pseudo-OOD samples via ID logit masking for automatic calibration without requiring external data. Experiments on OOD
benchmarks show that Fold outperforms prior methods, improving the average AUROC by 1.63% and reducing FPR95 by 2.30%, while maintaining computational efficiency comparable to a standard forward pass. Supported by theoretical analysis and extensive ablations, Fold provides a principled and practical solution for robust real-world deployment.
Read the full abstract ▾
All authors Seonghwan Park, Hyunji Jung, Dongyeop Lee, Namhoon Lee
Computer Vision
Scene-aware Prediction of Diverse Human Movement Goals
Qiaoyue Yang, Amadeus Weber, Magnus Jung et al. (+2 more)
Anticipation of human behaviours facilitates autonomous systems in proactive planning. Human behaviour could be
stochastic due to varying goals. Human goals typically guide their own movement and could therefore help to predict the human trajectory and human motion in the long-term. To infer the human movement intentions, the environmental context plays a significant role, in addition to the social cues expressed by the individual. Previous works on human goals prediction either require semantic knowledge of the scene, or only tackle interactions with objects. In this paper, we propose a novel multi-goal prediction method using the generative model to address the stochasticity of human movement. It leverages the current RGB scene and the human pose to predict diverse potential future goals of human movement based on the Conditional Variational Autoencoder (CVAE). Our results demonstrate that our approach is capable of generating multiple movement goals in the scene via samplings in latent space of the CVAE and exhibits generalization capability across scenarios in GTA-IM
dataset and PROX
dataset. Code is publicly available at \href{https://github.com/Q-Y-Yang/DiverseGoalsPrediction.git}{https://github.com/Q-Y-Yang/DiverseGoalsPrediction}.
Read the full abstract ▾
All authors Qiaoyue Yang, Amadeus Weber, Magnus Jung, Ayoub AI-Hamadi, Sven Wachsmuth
Computer Vision · Cross-listed from cs.RO
Seeing Touch from Motion: A Unified Modality-Aware Visuo-Tactile Policy with Tactile Motion Correlation
Shengqi Xu, Guojin Zhong, Yang Liu et al. (+7 more)
Visuo-Tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation. These sensors achieve high spatial resolution and multi-dimensional force sensing by utilizing an internal camera to monitor the deformation of their elastic gel surface, thereby indirectly inferring tactile cues. Despite their advantages, extracting fine-grained contact states necessary for contact-rich manipulation remains an open challenge. Existing methods typically use either raw images or cumulative motion fields to represent tactile cues. However, both are prone to perception ambiguity. Raw tactile images mainly capture appearance changes, while cumulative motion fields only reflect the aggregate gel deformation. Consequently, distinct fine-grained contact states can exhibit highly similar patterns, making it difficult to explicitly distinguish subtle contact variations. To address this issue, we explore the dynamic priors of tactile motion and discover that the correlation between transient and cumulative motion can explicitly distinguish fine-grained contact states. Based on this insight, we propose a motion-aware tactile representation to facilitate contact-rich manipulation. Beyond tactile representation, effective fusion of tactile and visual modalities is also critical. Most existing fusion methods either directly concatenate features from each modality or train modality-specific networks separately and fuse their outputs. However, these strategies struggle to simultaneously model cross-modal interactions and preserve modality-specific characteristics. In this work, we take advantage of the Mixture-of-
Transformers architecture and propose a unified modality-aware visuo-tactile policy that captures cross-modal complementarity while maintaining modality-specific properties.
Read the full abstract ▾
All authors Shengqi Xu, Guojin Zhong, Yang Liu, Fanjie Wang, Hu Luo, Hanyu Zhou, Weiyao Zhang, Ziyi Ye, Zuxuan Wu, Yu-Gang Jiang
Computer Vision
Latent-CURE for Breast Cancer Diagnosis
Weiyi Zhao, Xiaoyu Tan, Lu Gan et al. (+2 more)
Multimodal Large Models have significantly advanced automated breast ultrasound diagnosis. However, most existing frameworks utilize opaque, end-to-end paradigms prioritizing global statistical correlations over structured clinical
reasoning. Consequently, these models remain susceptible to shortcut learning amid extreme real-world epidemiological imbalances, often bypassing rare but decisive malignant indicators for dominant benign patterns. To address this disconnect, we propose Latent-CURE, a novel diagnostic framework driven by asymmetric weighted chain-of-thought methodology grounded in latent space
reasoning. Unlike traditional approaches, our framework constructs an implicit
reasoning trajectory forcing the model to sequentially infer standardized BI-RADS morphological descriptors before converging on a final diagnosis. Furthermore, to combat the extreme scarcity of critical malignant features, we couple this architecture with a dual-asymmetric optimization strategy. By dynamically adjusting margins and weights, this strategy safeguards high-specificity malignant descriptors from being overshadowed by common benign priors. Comprehensive evaluations demonstrate that our knowledge-injected approach provides transparent clinical evidence while achieving robust, accurate diagnostic performance in imbalanced medical cohorts.
Read the full abstract ▾
All authors Weiyi Zhao, Xiaoyu Tan, Lu Gan, Liang Liu, Xihe Qiu
Computer Vision
DCGrasp: Distance-aware Controllable Grasp Generation
Hiroyasu Akada, Jesús Pérez, Emre Aksan et al. (+6 more)
Generating 3D hand-object interactions is essential for applications in robotics, AR/VR, and synthetic data generation, where flexible controllability and strong generalization to diverse object geometries are required. However, existing methods rarely satisfy these requirements, limiting their practical applicability. We present DCGrasp, a multi-modal
diffusion framework for distance-aware controllable grasp generation. DCGrasp is built on a novel grasp energy term that computes Distance Profile, a signed distance from each hand vertex to the nearest object point, coupled with distance-aware weighting, effectively capturing the semantically similar hand-object interaction in near-contact regions while remaining invariant to object and hand identity. Given various controllable signals, DCGrasp first generates a Distance Profile based on a
Diffusion Transformer, together with a corresponding candidate hand pose. We then refine the candidate pose through optimization, enforcing consistency between the optimized hand pose and the generated Distance Profile in near-contact regions. Our experiments show that DCGrasp produces high-quality, geometrically plausible grasps with flexible user control, generalizing to diverse object and hand shapes and scales. Our work establishes a robust and versatile pipeline for the synthesis of controllable 3D hand-object interactions. The supplementary
video with dynamic visualization is available on our project page.
Read the full abstract ▾
All authors Hiroyasu Akada, Jesús Pérez, Emre Aksan, Vasileios Choutas, Cristian Romero, Alberto Garcia-Garcia, Vladislav Golyanik, Christian Theobalt, Thabo Beeler
Computer Vision
H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning
Eric Peh, Debaditya Roy, Chen Li et al. (+1 more)
Vision-
language models (VLMs) have achieved strong performance on visual
reasoning tasks, yet a correct final answer does not necessarily imply that the model followed a correct or visually grounded
reasoning process. We study how VLMs can be trained to produce explicit intermediate
reasoning steps that are both semantically meaningful and grounded in visual evidence. We represent each
reasoning step as a structured triplet consisting of a sub-question, sub-answer, and supporting bounding box, enabling the
reasoning process to be evaluated along both semantic and spatial dimensions. To optimize this representation, we introduce Hungarian-GRPO (H-GRPO), a reinforcement-learning framework that aligns predicted and reference
reasoning steps using permutation-invariant Hungarian matching, avoiding the assumption that valid
reasoning steps must follow a fixed ordering. Experiments show that optimizing grounded
reasoning improves both intermediate
reasoning quality and answer accuracy, with an H-GRPO variant achieving the strongest overall grounding and decomposition performance. Our analyses show that grounding quality and answer correctness are related but distinct, while counterfactual evidence masking confirms that the localized regions contain information important to the model's predictions. These findings motivate evaluating VLMs beyond answer accuracy and demonstrate the value of explicit, spatially grounded
reasoning.
Read the full abstract ▾
All authors Eric Peh, Debaditya Roy, Chen Li, Basura Fernando
Computer Vision
Traffic-CBM: A Structurally Interpretable Multimodal Framework for Encrypted Traffic Classification
Honglei Jin, Wenshuo Chen, Shaofeng Liang et al. (+6 more)
Encrypted traffic classification has achieved strong performance, but its decision process remains difficult to interpret. Existing methods usually combine flow statistics, packet sequences, and byte-level representations into opaque latent features, making it unclear which type of evidence actually drives the prediction. In this paper, we propose Traffic-CBM, a structurally interpretable
multimodal framework for encrypted traffic classification. Instead of directly fusing heterogeneous traffic signals into a black-box representation, Traffic-CBM organizes them into a unified hierarchical concept space. These concepts are not manually annotated semantic attributes; rather, they are scalar evidence summaries constrained by predefined traffic evidence groups. More specifically, grouped flow statistics are mapped to statistical concepts, dedicated temporal encoders learn temporal concepts from disjoint feature subspaces, and byte-level evidence is further organized into packet-level and cross-packet concepts. This design turns heterogeneous traffic evidence into an explicit concept representation and makes different levels of traffic evidence easier to analyze. We evaluate Traffic-CBM on multiple encrypted traffic
benchmarks. Results show that it achieves competitive and balanced classification performance while providing a clearer structural interpretation interface than conventional end-to-end fusion models. Further analyses suggest that the learned concept space is actively used in the prediction process and provides a clearer structural explanation of
multimodal traffic evidence.
Read the full abstract ▾
All authors Honglei Jin, Wenshuo Chen, Shaofeng Liang, Haozhe Jia, Runwei Guan, Menshuo Zhao, Shuxu Jin, Songning Lai, Yutao Yue
Computer Vision
StrucTab: A Structured Optimization Framework for Table Parsing
Gengluo Li, Shangpin Peng, Chengquan Zhang et al. (+10 more)
Table parsing aims to convert table images into structured, machine-readable representations, a task requiring the joint perception of complex spatial layouts and textual content. While recent
vision-
language models (VLMs) enable end-to-end parsing, they typically rely on direct supervision of the final output, thereby bypassing the explicit intermediate
reasoning that is crucial for understanding complex table structures. Furthermore, attempts to optimize these models using reinforcement learning (RL) are often hindered by unstable or ambiguous reward designs, limiting potential performance gains. To address these limitations, we propose StrucTab, a table parsing model learned through intermediate structural supervision and reward decomposition. At the modeling level, by decomposing the parsing process into human-inspired subtasks, such as row-column counting and merged-
cell analysis, StrucTab progressively unifies them through a sequential
reasoning strategy. At the optimization level, we introduce Uni-TabRL, a unified RL framework that leverages decomposed rewards (validity, structure, and content) to provide stable and informative optimization signals. Finally, at the evaluation level, we present TableVerse-5K, a large-scale, challenging
benchmark encompassing diverse, real-world table scenarios. Extensive experiments demonstrate the state-of-the-art performance of StrucTab across all evaluated public
benchmarks and significant improvements on TableVerse-5K, validating the effectiveness of explicit structural modeling and decomposed reward optimization. Code and
benchmark are publicly available at https://github.com/VirtualLUOUCAS/StrucTab.
Read the full abstract ▾
All authors Gengluo Li, Shangpin Peng, Chengquan Zhang, Binghong Wu, Hao Feng, Weinong Wang, Pengyuan Lyu, Huawen Shen, Xingyu Wan, Zhuotao Tian, Han Hu, Can Ma, Yu Zhou
Computer Vision
LLM-based Multimodal Personality Recognition via Facial Action Unit-Text Semantic Fusion
Tianyi Zhang, Wei Shan, Yuan Zong et al. (+2 more)
Personality recognition in asynchronous
video interviews (AVIs) has become increasingly important due to their widespread adoption in modern recruitment. Existing approaches often rely on large
language models (LLMs) to analyze textual responses of interviewees in AVI. However, unimodel methods often suffer from information loss (e.g., ignore facial cues). In contrast,
multimodal methods that employ full-face images or sparsely sampled frames can discard fine-grained temporal dynamics critical for accurate personality assessment. To overcome these limitations, we propose an LLM-based framework that semantically fuse facial action units (AUs) with textual responses of AVI. AU sequences are first converted into interpretable textual descriptions, which are then fused with participants' textual responses through an LLM. A lightweight regression head transforms the resulting
embeddings into continuous personality scores without disrupting the underlying semantic space. Experiments on the AVI-6
benchmark demonstrate consistent improvements over most baselines, with lower prediction errors and stronger correlations with human-rated scores across multiple traits. Further analysis reveals that AU-derived semantic representations offer complementary non-verbal cues to textual responses. Decoupling semantic understanding from regression prediction within the LLM also leads to greater training stability and clearer interpretability. Overall, these findings demonstrate that AU-text fusion provides a psychologically grounded and computationally efficient framework for personality recognition in AVIs.
Read the full abstract ▾
All authors Tianyi Zhang, Wei Shan, Yuan Zong, Tianhua Qi, Wenming Zheng
Computer Vision · Cross-listed from cs.LG
Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders
Chungpa Lee, Jihoon Kwon, Kyle Min et al. (+1 more)
Vision-
language models map images and text into a joint
embedding space. However, these
embeddings often entangle multiple semantic features, which limits their interpretability and controllability. While sparse autoencoders have emerged as a useful tool for decomposing these
embeddings into monosemantic features, their application to joint
embedding spaces has largely relied on an implicit, untested assumption that semantically corresponding features share the same directions across modalities. In this paper, we challenge this assumption by identifying discrepancies in feature directions for the same concept across image and text modalities, a phenomenon we term cross-modal feature heterogeneity. We demonstrate that this heterogeneity is a key driver of the modality split, where a shared concept activates different latents depending on the modality. This finding further reveals why aligning latent activations alone is insufficient to resolve the underlying feature mismatch. Motivated by this observation, we propose an approach that trains modality-specific sparse autoencoders to preserve each modality's feature geometry, and then aligns corresponding features post hoc. Our method improves reconstruction fidelity and enhances performance in cross-modal
retrieval and concept steering.
Read the full abstract ▾
All authors Chungpa Lee, Jihoon Kwon, Kyle Min, Jy-yong Sohn
Computer Vision
Building artificial intelligence virtual tissue (AIVT) for tissue state representation, feature prediction, and dynamic simulation
Qiqi Lu, Qianjin Feng, Shaoqun Zeng et al. (+1 more)
Modeling tissue states and their transitions is essential for understanding tissue homeostasis in health and pathological remodeling in disease. However, conventional computational modeling approaches are inadequate to capture the complexity of tissues as spatially organized, multiscale biological systems. Artificial intelligence (AI) has shown a remarkable ability for representing intricate systems, creating new opportunities to characterize tissue states and their transitions. Here, we propose the concept of AI virtual tissue (AIVT), an AI framework grounded in spatial
multimodal data for modeling tissues in health and disease. AIVT is designed to learn unified, spatially resolved, and dynamically manipulatable representations of tissue state, enabling tissue state representation and analysis, molecular and morphological feature prediction, and simulation of spatiotemporal tissue dynamics. We outline the fundamental assumptions, core capabilities, architectural components, as well as data and
algorithm foundations of AIVT as a framework for AI-driven tissue modeling.
Read the full abstract ▾
All authors Qiqi Lu, Qianjin Feng, Shaoqun Zeng, Shenghua Cheng
Computer Vision
IREU: Identity-Related Encoder-Only Unlearning for Customized Portrait Generation
Chaoyi Shi, Shanshan Zhang and Jian Yang
Customized Portrait Generation (CPG) technologies have been widely used to generate high-fidelity person images given an input image indicating the identity and a text
prompt indicating the required edits. Yet these methods pose significant
privacy risks by spreading fake visual information. Against such risks, each public generator should be able to suppress its generation ability for a particular person when requested. Therefore, in this work we investigate the identity unlearning problem for CPG. Since there are no previous methods in this field, we propose a simple baseline that updates the image encoder by minimizing identity similarity between generated and input images for target identities to be unlearned, while maximizing it for identities to be retained. However, we find such a global perturbation in the feature space harms the fidelity of generated images for other identities to be retained. To solve this problem, we propose a novel method IREU, which first locates identity-related features in an offline manner and then only performs feature perturbations on them. The experimental results show that our proposed method IREU achieves better identity unlearning performance for target identities to be unlearned, and also keeps high fidelity for other identities to be retained. In addition, our unlearned image encoder is generalizable across different generators with the same encoder without fine-tuning, which is friendly for deployment in practice.
Read the full abstract ▾
Computer Vision
LWDrive: Layer-Wise World-Model-Guided Vision-Language Model Planning for Autonomous Driving
Chen Yang, Yuhao Wei, Ze Xu et al. (+6 more)
Vision-
Language Models (VLMs) provide powerful semantic understanding and commonsense
reasoning for End-to-End Autonomous Driving (E2E-AD) planning. However, trajectories directly generated by VLMs often encode only coarse driving intentions and remain insufficient for geometrically accurate, future-aware, and multi-view-grounded planning. To address these limitations, we develop the Layer-Wise World-Model-Guided Driving framework (LWDrive). LWDrive is a VLM planning framework that refines coarse trajectories through layer-wise world-model guidance. Instead of treating the VLM output as the final trajectory, LWDrive uses it as an intent-aware coarse plan, expands a diverse candidate space around it, and progressively refines the candidates through a Foresight Cascade Planner (FCP). Specifically, we introduce future-frame generation supervision to encourage the VLM to learn forward-looking scene representations, thereby injecting planning-relevant predictive dynamics into its internal hidden states. Built upon these world-model-supervised representations, FCP exploits VLM features across multiple layers and integrates historical temporal states, Action-Query representations, and current-frame multi-view Bird's-Eye-View (BEV) features to refine candidate trajectories in a coarse-to-fine manner. This design enables progressive correction of spatial positions and motion trends while grounding trajectory refinement with multi-view scene cues and preserving the high-level driving intention produced by the large model. Finally, a score head evaluates the refined candidates and selects the best trajectory as the final planning output. Experiments show that LWDrive achieves a score of 92.0 on the NAVSIM
benchmark and 89.6 on NAVSIM-v2. Code and models will be made publicly available.
Read the full abstract ▾
All authors Chen Yang, Yuhao Wei, Ze Xu, Ziheng Zou, Shuang Liang, Delin Ouyang, Lingfeng Qi, Jie Li, Guofa Li
Computer Vision
SUMO: Segment and Track Any Motion with Nonlinear State Space Models
Kexin Tian, Sixu Li, Keshu Wu et al. (+2 more)
Visual Object Tracking (VOT) and Moving Object
Segmentation (MOS) are two fundamental tasks in computer
vision that involve both spatial and temporal object dynamics. Existing methods rely predominantly on visual cues and thus often falter in real-world scenarios where object motions are inherently complex and nonlinear. To address this limitation, we propose SUMO, a zero-shot, training-free, unified framework integrating nonlinear dynamics with
vision-based
segmentation for accurate and consistent VOT and MOS. Specifically, we develop a nonlinear State Space Model (SSM) inspired by robotics principles to capture the complex object dynamics. Building on this model, we propose a Selective Unscented Filter (SUF) for accurate state estimation, which features a joint scoring mechanism and dynamically fuses multi-source predictions to identify the most plausible object state over time. Furthermore, we apply a
memory selection mechanism to evaluate the reliability of
memory frames. Our extensive experimental results show that SUMO achieves state-of-the-art performance on both VOT and MOS tasks.
Read the full abstract ▾
All authors Kexin Tian, Sixu Li, Keshu Wu, Yang Zhou, Zhengzhong Tu
Computer Vision
RainODE: Continuous-Time Precipitation Forecasting with Latent Neural ODEs
Yeeun Seong, Doyi Kim, Minseok Seo et al. (+1 more)
In precipitation
forecasting, not only accuracy but also temporal resolution is critical. However, increasing temporal resolution is constrained by observational limitations and the computational cost of dense discrete modeling. To overcome this limitation, we reformulate precipitation
forecasting as a continuous-time dynamical system and propose RainODE, a framework that models precipitation
evolution in latent space using a Neural ODE. This formulation enables derivative-consistent temporal dynamics and captures the dominant large-scale advective motion of precipitation systems. Nevertheless, a purely deterministic ODE struggles to represent non-advective intensity changes such as localized growth, decay, and sub-grid variability, often leading to over-smoothed predictions. To address this issue, we introduce a
stochastic source modeling module based on a Brownian Bridge formulation, which refines residual intensity variations and restores fine-grained structures while preserving advective consistency. By combining deterministic continuous dynamics with
stochastic refinement, RainODE enables arbitrary-time inference while maintaining sharp predictions. Experiments on SEVIR and the newly introduced Radar-based Precipitation Integrated
Dataset (RAPID) demonstrate consistent improvements across multiple temporal intervals and precipitation regimes. The code is available at https://github.com/SeongYE/RainODE.
Read the full abstract ▾
All authors Yeeun Seong, Doyi Kim, Minseok Seo, Changick Kim
Computer Vision
Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction
Zijian Hong, Qi Lv, Yuxiang Xie et al. (+4 more)
Visual pointing maps a
language instruction to pixel co ordinates, a core skill for
embodied AI. We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the
benchmark. The ap proach targets three failure modes. First,
agent-driven syn thesis builds large semantic and anchor-relative candidate pools; the server inventory contains 55,372 processed out puts, 53,772 de-duplicated sample IDs, and 37,574 train able completed or accepted rows. Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification. Third, two model-side modules address complementary errors: AttnRes adds gated cross-block at tention for steerability, while ABC correction encodes per turbed coordinates with visual features for general coordi nate grounding. Category-aware routing combines comple mentary specialists; local validation used to select experts records 93.9% Affordance, 82.6% Spatial Relation, 78.2%
Reasoning, 70.4% Counting, and 63.0% Steerability.
Read the full abstract ▾
All authors Zijian Hong, Qi Lv, Yuxiang Xie, Jianming Xing, Xiang Deng, Weili Guan, Liqiang Nie
Computer Vision
See Only When Needed: Context-Aware Attention Intervention for Mitigating Hallucinations in LVLMs
Yuqing Lei, Wenbo Lyu, Yingjun Du et al. (+3 more)
Large
Vision-
Language Models (LVLMs) excel at
multimodal tasks but remain prone to object
hallucinations. Prior training-free remedies often uniformly strengthen visual signals, which may also amplify irrelevant regions and introduce spurious evidence, harming fluency. We propose Context-aware
Attention Intervention (CAI), a training-free inference-time mechanism that enforces a see only when needed principle via two-axis selectivity: where to look and when to intervene. At each decoding step, CAI derives
token-specific visual relevance from early-layer representations to localize semantically aligned regions, and applies a conservative,
entropy- and depth-gated
attention tilt only for uncertainty-spiking
tokens in deeper layers where visual grounding degrades, leaving confident
tokens and irrelevant regions largely unchanged. This targeted intervention strengthens visual grounding while preserving linguistic fluency, and it yields consistent improvements even without contrastive decoding, which remains optional as an auxiliary bias-suppression module. Extensive experiments across multiple LVLM backbones and
benchmarks show that CAI achieves state-of-the-art
hallucination mitigation, and our analysis characterizes CAI as a KL-minimal
attention reweighting with bounded interference under inactive gates or small tilts. Code is available at https://github.com/Iris1946/CAI.
Read the full abstract ▾
All authors Yuqing Lei, Wenbo Lyu, Yingjun Du, Xiantong Zhen, Cees G.M. Snoek, Ling Shao
Computer Vision
Bricker to BRACE: A Bracket Exposure RAW Dataset and Restoration Model for Flicker-Banding
Zihan Zhou, Libo Zhu, Jue Gong et al. (+4 more)
Flicker-banding (FB), arises from temporal aliasing between a camera's rolling shutter and a display's brightness modulation, degrading screen-captured image readability with color shifts and jagged patterns. Existing single-frame methods with simplified parametric stripe models cannot reliably distinguish these artifacts from genuine texture. To address this, we conduct a systematic analysis of complex FB morphologies and reveal their significant variation across exposure settings, motivating a multi-frame bracketed RAW restoration paradigm. We construct Bricker, a synthetic-real bracketed RAW
dataset built via ray-tracing-based physical simulation and automated multi-exposure capture tool. We further propose BRACE: Bracketed RAW Flicker-Banding Removal, a multi-frame restoration model that utilizes frequency-aware banding prior and a multi-scale spatial cross-
attention modulator (MSCAM) for cross-exposure spatial fusion. We also introduce the Stripe Frequency Consistency (SFC) metric to evaluate banding removal. Experiments demonstrate state-of-the-art performance on both synthetic and real
benchmarks. Our
dataset and code are available at: https://github.com/ZZH-qwq/BRACE.
Read the full abstract ▾
All authors Zihan Zhou, Libo Zhu, Jue Gong, Zhiyi Zhou, Jiezhang Cao, Yong Guo, Yulun Zhang
Computer Vision
Robust Trajectory Distillation: Hybrid Reweighting Meets Teacher-Inspired Targets
Kaifeng Chen, Lechao Cheng, Jiyang Li et al. (+6 more)
Dataset distillation (DD) condenses large corpora into compact, information-rich subsets for efficient training and reuse. However, under noisy supervision, DD risks condensing corrupted associations together with useful signals, degrading robustness. Conventional noisy-label remedies (sample selection, loss weighting, label correction) tightly couple noise estimation with model optimization, often require clean anchors, and can amplify confirmation bias-assumptions that are misaligned with DD's goal of compact, plug-and-play supervision. We therefore propose a trajectory-based DD framework that jointly suppresses noise and preserves transferable knowledge without relabeling or clean subsets. It comprises two complementary components: Selective Guidance Reweighting (SGR), which fuses global forgetting patterns (second-split forgetting) with local neighborhood consistency into a progressive reweighting scheme that prioritizes clean supervision along the teacher trajectory; and Teacher-Inspired Auxiliary Targets (TIAT), which inject auxiliary residual guidance distilled from intermediate teacher dynamics to reinforce informative signals while remaining internally consistent. Together, SGR and TIAT produce distilled
datasets with cleaner and richer representations under noisy supervision. The framework is robust, label-preserving, computationally lightweight, and broadly applicable, yielding consistent gains over state-of-the-art DD baselines across symmetric, asymmetric, and real-world noise.
Read the full abstract ▾
All authors Kaifeng Chen, Lechao Cheng, Jiyang Li, Shengeng Tang, Fan Zhang, Yantao Pan, Yaxiong Wang, Tuanrui Hui, Zhun Zhong
Computer Vision
HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes
Guoqiu Li, Jin Song and Yiyun Fei
Recently, zero-shot object customization generation methods have rapidly developed and shown tremendous potential for applications. For instance, in the e-commerce domain, consumers can observe the visual effect of furniture placed within their personal living spaces or clothes worn on their own bodies. Many existing approaches perform object customization generation based on
diffusion models and extracted reference object features. However, the generated object significantly diverges from the original reference object in details such as patterns and curves. Particularly for asymmetrical reference objects, the absence of comprehensive multi-viewpoint information prevents the generation of object poses that harmonize with the background scene. To address these shortcomings, we have constructed a novel
dataset comprising multi-angle images of furniture and indoor scenes. Based on
diffusion models, we introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to accurately generate visually harmonious object poses within specified areas of the background scene. During the
diffusion process, we further extract high-fidelity details of the reference object and perform cross-
attention with the noise latents in the latent space, thereby ensuring the preservation of details in the customized object generation. Extensive qualitative and quantitative experiments demonstrate that our method achieves superior performance over other existing zero-shot as well as few-shot object customization approaches.
Read the full abstract ▾