Computer Vision
READ: Reading the Selective Scan for Faithful Video Attribution
Magdalena Trędowicz, Piotr Kubaty, Arkadiusz Lewicki et al. (+2 more)
Selective state-space models are an efficient approach to long-sequence
video modeling. However, their recurrent computation makes it difficult to determine which
video tokens drive a prediction. Existing attribution methods often require additional backward passes or repeated model evaluations, making them substantially more expensive than the forward inference they aim to explain. In this work, we introduce READ, a parameter-free attribution method for selective state-space models that unrolls the recurrence at the position read by the classifier. READ traces the contribution accumulated in the recurrent state back to the
video tokens that contribute to the final prediction. By anchoring attribution at the classifier's readout, it produces a single spatiotemporal relevance map from which frame- and region-level relevance can be obtained directly. Despite requiring only one forward pass, READ achieves competitive faithfulness and localisation compared with substantially more expensive attribution methods. Experimental results show that READ avoids the systematic relevance biases of existing methods while recovering meaningful temporal and class-specific structure across
datasets and backbones. More broadly, our results show that the recurrence underlying selective state-space models is not only an efficient mechanism for inference, but also a direct source of interpretable evidence for their predictions.
Read the full abstract ▾
All authors Magdalena Trędowicz, Piotr Kubaty, Arkadiusz Lewicki, Jacek Tabor, Łukasz Struski
Computer Vision
Shortcut Learning in Geospatial Foundation Model Embeddings for Cross-Disaster Damage Assessment
Ali Shibli, Andrea Nascetti and Yifang Ban
Geospatial foundation models can support disaster damage assessment, but high accuracy on familiar disasters may hide shortcut learning: prediction from event or scene characteristics rather than evidence that transfers to a new disaster. We study
embeddings from TerraMind, Prithvi-EO-2.0, DOFA, and DINOv3 on the multi-disaster
datasets xView2 and BRIGHT. Holding out an entire disaster, instead of randomly splitting image tiles, reduces mean balanced accuracy from 0.807 to 0.638 on xView2 and from 0.818 to 0.568 on BRIGHT. Four sets of tests examine what information supports prediction: event and acquisition context, before/after image inputs, building count, and correct image pairing. Pre-event images remain predictive, and a classifier given only the annotated number of buildings reaches balanced accuracy of 0.629 and 0.653. Combining before/after features performs best on average on xView2, but does not improve over pre-only features on BRIGHT. Shuffling image pairs within the same disaster and damage class causes modest losses. Motivated by these findings, we test whether feature transformations learned only from training disasters improve generalization to unseen disasters. These transformations improve performance, with supervised contrastive learning performing best on xView2 and PCA on BRIGHT. Our results highlight the importance of evaluation on unseen domains for assessing model robustness and show that simple feature-level transformations can improve generalization. Code and experiments can be found at https://github.com/alishibli97/gfm-shortcut-learning.
Read the full abstract ▾
Computer Vision
Neurons++: A Unified Visual-Stream Framework for Image and Video Reconstruction from fMRI with Neural-to-Visual Interpretability
Haonan Wang, Yuchen Wang, Jingyu Lu et al. (+4 more)
Decoding visual experience from fMRI is important for understanding human perception and developing
brain-computer interfaces. Recent generative decoding methods can reconstruct plausible images and
videos, but visual plausibility alone does not ensure that the output is faithfully grounded in
brain activity. In this work, we propose
Neurons++, a visual-stream fMRI reconstruction framework for both static image and dynamic
video reconstruction. Compared with our conference version
Neurons,
Neurons++ extends the paradigm from
video-only reconstruction to visual-stream reconstruction for both images and
videos, and replaces parallel task conditioning with serial cortex-inspired decoding. The model first uses image-fMRI pretraining to obtain a stable spatial-semantic initialization, and then performs subject-specific fine-tuning for image and
video reconstruction separately. The serial design organizes fMRI-derived representations into explicit spatial, semantic,
language-level, and temporal cues. Extensive experiments show that
Neurons++ achieves state-of-the-art
video reconstruction and competitive image reconstruction across standard
benchmarks. Beyond reconstruction fidelity, neural-to-visual analyses indicate that
Neurons++ exhibits qualitative directional controllability under neural perturbation, that its fMRI encoder read-out is sensitive to stimulus-relevant cortical features, and that it supports ROI-guided semantic modulation. Code and model weights are available at: https://github.com/xmed-lab/NeuronsPlusPlus.
Read the full abstract ▾
All authors Haonan Wang, Yuchen Wang, Jingyu Lu, Qixiang Zhang, Lehan Wang, Xuanqi Huang, Xiaomeng Li
Computer Vision
Gaussian Splatting with Reflections Guided by What is Seen
Yiming Liang, Tianyu Xiao and Hiroshi Ishikawa
While novel view synthesis methods like Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have shown impressive performance on common scenes, they often struggle with capturing view-dependent specular effects, particularly near-field reflections. Existing methods utilize environment maps, which only represent lights from infinity, or environment Gaussians, which rely solely on indirect supervision from reflections, are insufficient for accurately modeling such effects. Here, we propose a framework based on 2D Gaussian Splatting (2DGS) that leverages directly observable environment information in input images. Our approach also integrates a hybrid reflection appearance representation and view-dependent specular modeling. Experiments on public
benchmarks and a newly curated near-field reflection
dataset demonstrate that our method achieves superior rendering quality and
geometric accuracy.
Read the full abstract ▾
Computer Vision
KVeil: First-Place Solution to ECCV 2026 LifeGenIP Challenge on Unlearnable Videos
Boheng Li, Yueming Xing, Yiming Li et al. (+3 more)
The rapid development of personalized
video generation has raised serious concerns over identity
privacy, as a few facial
videos of a person now suffice to reproduce their identity in synthetic
videos. This report presents the first-place solutions to both phases of the ECCV 2026 LifeGenIP Challenge on Unlearnable
Videos, which aims to protect facial
videos from unauthorized customization under a limited and imperceptible perturbation budget. For Phase 1, we introduce Reference Capacity Collapse, a pair of complementary losses that degrade the information carried by the reference condition while keeping subsequent
video representations dependent on it, together with a Motion-Guided EOT mechanism that improves robustness across unknown inference conditions. Phase 2 further considers tuning-based customization and unseen generative models. To address both threats within a single perturbation, we design a two-layer framework that protects the facial and background regions separately, constructing structure-aligned perturbations from a synthetic target identity in the former and strengthening the white-box disruption of Phase 1 in the latter. Our method achieves nearly perfect protection under white-box reference-based customization, and remains effective under other customization paradigms, various models, and temporal attacks. Both solutions are built upon our recent work LETHE. We hope our solutions can inspire future work on protecting facial
videos against unauthorized customization. Our code is available at https://github.com/AntigoneRandy/KVeil.
Read the full abstract ▾
All authors Boheng Li, Yueming Xing, Yiming Li, Jie Zhang, Han Qiu, Tianwei Zhang
Computer Vision
LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
Zhiyuan Xiang, Peidong Liu, Mingyang Li et al. (+4 more)
Scaling generalist robotic policies is limited by the high cost of collecting embodied interaction data. Large-scale egocentric human
videos offer a rich alternative, but they lack
robot action labels, and existing action annotation methods are often noisy under real-world visual complexity and diverse manipulation dynamics. We introduce LAPT-Scale, a scalable latent action pre-training paradigm with two components: LAPT-Scale-LAM learns continuous latent actions by explaining frame-to-frame visual transitions in human
videos and uses them as human-derived supervision, while LAPT-Scale-Policy trains a VLA policy through human-
video latent-action pre-training, target-
robot embodiment
alignment, and task-specific post-training. Unlike prior vector-quantized LAMs, LAPT-Scale-LAM uses a continuous latent space to preserve fine-grained motion and interaction dynamics. We further combine perceptual, semantic, and optical-flow
alignment signals to improve transition-level supervision beyond pixel reconstruction. Trained on 23.3K hours of egocentric
video, LAPT-Scale-LAM provides implicit action supervision for policy pre-training, yielding consistent scaling gains and improving real-
robot success from 63.3% to 85.0% as annotated human
video scales from 1K to 5K hours. The learned policy also generalizes to unseen visual conditions.
Read the full abstract ▾
All authors Zhiyuan Xiang, Peidong Liu, Mingyang Li, Jiawei Li, Wenhow Li, Wenbo Ding, Xiang Deng
Computer Vision
State-Isolated Vision Mamba via Update-Tendency Routing for Underwater Image Enhancement
Yan Luo, Laibin Chang, Zefan Feng et al. (+3 more)
Vision Mamba has recently emerged as an efficient alternative to convolutional and
Transformer-based architectures by modeling long-range spatial dependencies through recurrent state
evolution. Existing studies primarily improve
token serialization, scanning strategies, or expert routing, yet largely overlook a state level question question: how should recurrent states be organized for spatially heterogeneous visual restoration? In this paper, we show that forcing
tokens with heterogeneous update behaviors to recurrently modify a shared hidden state leads to persistent cross-content interference, which we term State Contention. We further argue that effective recurrent modeling should organize state
evolution according to
token compatibility rather than
token ordering. Based on this observation, we introduce State Isolation, a new recurrent state organization principle for
Vision Mamba,
tokens with compatible state-update tendencies evolve within independent recurrent state streams. To instantiate this principle, we propose IsoMamba, a $\Delta$-guided architecture that constructs model-intrinsic update representations, discovers update-compatible
token groups through label-free clustering, and routes them to isolated Mamba experts following a cluster–route–isolate paradigm. In addition, we develop a dynamic channel-wise reweighting objective to compensate for wavelength-dependent reconstruction difficulty in underwater images. Extensive experiments demonstrate that IsoMamba consistently achieves state-of-the-art performance on multiple underwater image enhancement
benchmarks while providing cleaner hidden state responses, validating the effectiveness of state isolated recurrent modeling.
Read the full abstract ▾
All authors Yan Luo, Laibin Chang, Zefan Feng, Yi'ang Chen, Kui Jiang, Zhengguo Li
Computer Vision
GLIMPSE: Global-Local Indexing with Multi-Precision Spectral Encoding for Explorable Video World Models
Zhufeng Xu, Henglei Lv, Bailin Deng et al. (+5 more)
A camera-controlled
video world model must keep a scene consistent when the camera returns to it.
Memories that render stored features into the target frame can turn depth and pose errors into ghosting or duplicated structure. Positional transport avoids that resampling by relocating a historical patch's address rather than its content, but assigns full positional precision even to inaccurate projections. We present GLIMPSE, a global-local
memory interface with band-wise multi-precision coordinates and geometry-conditioned spectral attenuation to reduce sensitivity to inaccurate projected addresses. Local patch
memory supplies view-specific appearance, while recurrent global
memory provides scene context. Footprint-guided
retrieval selects historical patches under a fixed local read budget. Under matched
retrieval and read budgets, GLIMPSE improves
geometric correspondence and photometric consistency over deterministic positional transport.
Read the full abstract ▾
All authors Zhufeng Xu, Henglei Lv, Bailin Deng, Yikang Ding, Xiaoqiang Liu, Guoxin Zhang, Hongbo Fu, Lin Gao
Computer Vision
Improving Semantic Coverage in AI Test & Evaluation through Generative Augmentation
Connor Hashemi, Bharadwaj Ravichandran, Connor Anderson et al. (+4 more)
In test and evaluation (T&E) of AI models, evaluators typically have access only to a held-out test
dataset and no knowledge of the training data or its semantic composition. If critical input conditions—such as time-of-day, weather, or scene location—are underrepresented in this test
dataset, T&E results can be misleadingly due to this semantic coverage gap. We propose a three-stage generative AI pipeline to detect and fill such semantic gaps without requiring ground-truth labels of these semantics. First,
vision-
language models (VLMs) perform zero-shot binary labeling of semantic factors, and an
embedding-space clustering step identifies which semantics most affect downstream model behavior. Second,
diffusion-based image-to-image models transform existing covered images into the underrepresented semantic domain, preserving scene content while augmenting high-level appearance. Third, statistical analysis of the augmented image
embeddings verifies that the synthetic imagery improves
dataset coverage before it is incorporated. Crucially, the synthetic data is used only to enrich the test set and no model retraining is performed for T&E. We validate the full pipeline on both image classification and object detection under day/night semantic gaps, showing that the augmented test
datasets better distinguish models with and without exposure to the missing semantic, thereby yielding T&E conclusions that more closely match those of an ideal, fully covered
dataset. Code available at https://github.com/Hashe037/DEDUCE
Read the full abstract ▾
All authors Connor Hashemi, Bharadwaj Ravichandran, Connor Anderson, Shitij Govil, Brandon RichardWebster, Brian Hu, Keith Fieldhouse
Computer Vision
Saumyaranjan Mohanty, Aravind Reddy and Konda Reddy Mopuri
Audio-visual
dataset distillation aims to synthesize a small
dataset from a large paired audio-visual
dataset, while nearly preserving the training utility for neural networks. Existing approaches such as DM and AVDD face two critical limitations: 1) they are
memory-intensive, making them difficult to scale to complex architectures and higher samples-per-class (SPC) settings, and 2) they often fail to preserve strong cross-modal semantic correspondence. To tackle these issues, we propose X-AVDD, a cross-
attention-based
distillation method that explicitly captures inter-modal relationships during synthesis. By decoupling cross-modal
alignment from joint optimization, X-AVDD is substantially more
memory-efficient, reducing end-to-end synthesis cost by $\sim 8\times$ GPU-hours, while producing highly aligned synthetic data. Empirically, X-AVDD significantly outperforms other audio-visual
distillation methods. On VGGSound, an audio-visual
dataset of short YouTube clips, X-AVDD improves top-1 test accuracy from $8.2\%\rightarrow24.0\%$ at SPC=$10$ and $9.8\%\rightarrow26.5\%$ at SPC=$20$. Furthermore, X-AVDD improves cross-architecture generalization (e.g., $25.5\%\rightarrow35.3\%$ for a ViT target on AVE
dataset at SPC $=50$) and successfully scales to complex architectures such as AudioCLIP, achieving $62.3\%$ accuracy using only $5.65\%$ of the full
dataset, against the full
dataset accuracy of $72.4\%$ while completing in $9\times$ lower training time compared to the full
dataset.
Read the full abstract ▾
Computer Vision
PhysForge: Material-Controllable 3D Reconstruction with Separable Components
Yuchen Wang, Yanqi Bao, Zheng Gu et al. (+3 more)
Recent advances in physical 3D generation have enabled visually faithful assets with simulatable mechanical behavior. However, existing methods provide limited control over material behavior beyond appearance-based inference and struggle to estimate physical properties from complex interactions between objects and semantic parts. Therefore, we present PhysForge, a
Video Diffusion Model (VDM)-based framework for addressing these challenges, comprising two complementary modules for material-controllable physical fitting and componentlevel interaction modeling. The first, PhysFit, leverages material-conditioned VDM dynamics as references for physical-property fitting, introducing windowed fitting to focus on material-revealing interactions and Hallucinated Forcing to disentangle nonphysical motion hallucinated by the VDM from material responses. The second, PhysComp decomposes complex interactions into individual objects and semantic components, enabling their physical parameters to be optimized independently and flexibly composed for joint simulation. Experiments on PhysAssets and Google Scanned Objects demonstrate that, compared with existing methods, PhysForge provides stronger control over material-dependent physical responses while supporting component-level physical modeling for complex interactions. It achieves the best CLIP and physical controllability scores on both common and uncommon materials, with qualitative comparisons, component-level ablations, and a user study further supporting these improvements. Our project page is available at https://masterwanna.github.io/PhysForge.html .
Read the full abstract ▾
All authors Yuchen Wang, Yanqi Bao, Zheng Gu, Zhengqing Chen, Jieqi Shi, Jing Huo
Computer Vision · Cross-listed from eess.IV
Can Multimodal Decision Models Classify Breast Ultrasound Images? Benchmarking CLEF and CLEF-Flash Across Prompts and Datasets
Tran Cao Minh, Hai Tran and Ha Minh Tan
Multimodal decision models return probabilities over predefined alternatives, offering a structured interface for image classification. This study
benchmarks inference with CLEF and CLEF-Flash on 2818 breast ultrasound images from
Dataset B, BUSI, and BUS-BRA.
Dataset B and BUS-BRA use binary benign-malignant classification, whereas BUSI includes normal, benign, and malignant categories. Model parameters remain unchanged, and original images are submitted without user-defined preprocessing. Three
prompts provide minimal labels, an explicit classification task, or sonographic criteria. We report accuracy, macro F1, balanced accuracy, macro precision, throughput, and latency. CLEF-Flash with the clinical
prompt achieves the highest accuracy on all three
datasets: 74.85% on
Dataset B, 66.28% on BUSI, and 51.63% on BUS-BRA. On BUSI, however, CLEF with the task-guided
prompt achieves the highest macro F1 (64.51%) and balanced accuracy (68.72%). CLEF-Flash has lower latency in every matched comparison. Clinical instructions improve CLEF-Flash accuracy relative to the minimal
prompt but reduce CLEF accuracy on all
datasets. Confusion
matrices reveal frequent benign-to-malignant errors, particularly on BUS-BRA, where all configurations fall below the 67.63% accuracy of a majority-class predictor. These findings demonstrate
dataset-and
prompt-dependent performance and motivate class-specific evaluation before applying
multimodal decision models to breast ultrasound analysis.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Model-agnostic Knowledge Augmented Machine Learning for Weakly Annotated Bioimaging Data
Jayanie Bogahawatte, Sachith Seneviratne, Simon Maksour et al. (+6 more)
Machine learning models are increasingly used for automated feature extraction from weakly annotated bioimaging data, such as microscopy and cytomorphology images. Due to the sole dependency of automated feature extraction, existing methods yield inconsistent classification performance across weakly annotated
datasets. Here, we propose a model-agnostic knowledge augmentation method that enables ML models to exploit domain knowledge beyond what automated feature extraction can capture. The proposed approach improves classification performance across seven ML architectures. When integrated with masked autoencoder-based self-supervised learning, our method achieves the strongest overall performance, improving precision by 2.9% over the next-best method on three in-house microscopy
datasets and by an average of 6.8% across three public cytomorphology
benchmarks. We perform
nucleus-level classification based on mislocalization of TAR DNA-binding
protein 43 (TDP-43) from the
nucleus to the cytoplasm, a phenotype linked to Amyotrophic Lateral Sclerosis (ALS), using three independent microscopy
datasets of motor
neurons derived from human induced pluripotent stem
cells. Accurate quantification of this phenotype at scale remains challenging and may contribute to inconsistencies across studies. Application of the best-performing model to these microscopy
datasets reveals biological interpretations related to ALS, including dose-dependent effects of the proteasomal stressor, MG132, and the oxidative stressor, sodium arsenite (SA), on TDP-43 mislocalization, variable pathology in motor
neurons from sporadic and familial ALS donors, and a lack of protection of the pharmacological compounds, edaravone and niclosamide, against SA-induced TDP-43 mislocalization. Together, our pipeline enables in-vitro cellular image analysis to advance understanding of ALS-associated pathology and potential treatments.
Read the full abstract ▾
All authors Jayanie Bogahawatte, Sachith Seneviratne, Simon Maksour, Dzung Do-Ha, Elizabeth Qian, Samuel Brennan, Matthew Keon, Lezanne Ooi, Saman Halgamuge
Computer Vision
Benchmarking Answer-Grounding Consistency in Remote Sensing Visual Question Answering
Ping Guo, Zhiqi Huang, Xinran Li et al. (+3 more)
In grounded remote sensing visual question answering, a model produces a textual answer and identifies relevant objects in the image as grounding evidence. Existing evaluations measure answer correctness and localization quality, but do not directly assess answer-grounding consistency. We propose the Answer-Evidence Support Protocol (AESP) to assess this consistency through evidence support: whether the grounding evidence supports the model's own answer. AESP applies task-specific rules to existence, counting, and comparison questions. We further construct the Grounded Remote Sensing Visual Question Answering (G-RSVQA)
benchmark, containing 15,027 samples with questions, reference answers, and all related object-level annotations. Experiments on seven grounded
vision–
language models reveal failures of evidence support across all three question types. Up to 50.82% of correct answers in the evidence evaluation subset are not supported by the provided grounding evidence.
Read the full abstract ▾
All authors Ping Guo, Zhiqi Huang, Xinran Li, Linkun Yu, Jingyin Pei, Kamata Seiichiro
Computer Vision · Cross-listed from cs.CL
Multi-Stream Attention Residuals
Xiaohan Qin, Xiangdong Zhang, Tuo Dai et al. (+7 more)
Attention Residuals (AttnRes) enables content-dependent
retrieval over network depth, which can be viewed as single-stream softmax
attention along the depth axis. Inspired by the success of multi-head
attention along the sequence axis, we introduce Multi-Stream
Attention Residuals (MSAR), an extension of AttnRes that generalizes this mechanism to multi-head depth
attention, yielding substantial performance gains with minimal additional overhead. MSAR draws stream diversity from sequence context through multi-scale causal convolutions, allowing different heads to retrieve historical representations through distinct contextual views while preserving the integrity of the hidden representation. The retrieved streams are combined through lightweight adaptive mixing to generate the single-stream layer input. By retaining single-stream sources and expanding them on demand, MSAR supports multi-stream computation without requiring persistent multi-stream storage or communication. Across all 12 downstream
benchmarks at the 18B and 28B MoE scales, MSAR consistently outperforms prior residual variants, including standard residual connections,
manifold-constrained Hyper-Connections (mHC), and AttnRes. Scaling-law experiments yield an iso-loss compute speedup of $1.58\times$ over the standard residual baseline, compared with $1.35\times$ for AttnRes and $1.30\times$ for mHC. These results make MSAR a promising residual architecture for large-scale pre-training.
Read the full abstract ▾
All authors Xiaohan Qin, Xiangdong Zhang, Tuo Dai, Sunan Zou, Huaijin Wu, Yebin Yang, Zhuo Xia, Lin Yao, Yuliang Liu, Junchi Yan
Computer Vision
CUBE: Class-conditioned Unknown Boundary Evidence for Open-Set Insect Species Recognition
Hao Tan, Qimin Zhang, Zhe Xiao et al. (+3 more)
Open-set insect species recognition faces significant challenges when deployed in real-world biodiversity scenarios, particularly when encountering unknown species that are visually close to known training classes. Unlike generic out-of-distribution samples, fine-grained unknown insects are often not visually anomalous. They instead tend to appear near the decision boundary of specific known species, leading to unreliable closed-set predictions. To address these challenges, we propose Class-conditioned Unknown Boundary Evidence (CUBE), a novel evidence-based framework that models unknownness around each known species rather than as a single global unknown class. Our key idea is to evaluate a test sample relative to the known species it most closely resembles, enabling local rejection for visually confusable unknown insects. CUBE learns class-conditioned boundary evidence from auxiliary species, preserves auxiliary species as shadow references during inference, and incorporates taxonomic candidate calibration to verify biological plausibility. When ecological metadata is available, we further introduce CUBE-E, which adds a lightweight location-consistency term without modifying the visual backbone. Experiments on the Open-Insect C-America
benchmark show that CUBE achieves superior performance, outperforming state-of-the-art baselines with 95.8%, 98.7%, and 99.1% AUROC on local, non-local, and non-moth unknown splits, respectively, while maintaining 94.9% closed-set accuracy. With ecological metadata, CUBE-E further improves the results to 96.0%, 99.7%, and 99.5% AUROC with 95.2% closed-set accuracy.
Read the full abstract ▾
All authors Hao Tan, Qimin Zhang, Zhe Xiao, Zekai Zuo, Wanting Liu, Jinghui Qin
Computer Vision
Thyroid Nodule Segmentation in Ultrasound Images with Conformal Uncertainty Estimation
Francesco Teodori, Pierpaolo Brutti and Umberto Ferraro Petrillo
Thyroid nodules affect a large proportion of the general population. Although most are benign, a non-negligible fraction is malignant. Ultrasound imaging is the primary diagnostic modality, and accurate nodule segmentation is a prerequisite for CAD systems to classify malignancy and assess metastatic risk.\\ In this work, we propose a thyroid nodule segmentation framework that integrates a DeepLabV3+, channel-
attention modules and post-hoc conformal prediction, providing both accurate mask and pixel-level uncertainty estimates. We address thyroid ultrasound nodule segmentation as the first stage of a pipeline that also includes malignancy classification and metastatic risk prediction. We adopt DeepLabV3+, whose Atrous Spatial Pyramid Pooling (ASPP) captures multi-scale context, with an EfficientNet-B5 backbone for its accuracy-efficiency trade-off. Additionally, we test three channel-
attention modules: Squeeze-and-Excitation, CBAM and Grouped Channel
Attention (GCA), paired with two loss formulations (Dice-BCE and Boundary).\\ Since binary masks do not indicate prediction confidence, we wrap the trained model with a split conformal prediction layer that produces pixel-level uncertainty maps with statistical coverage guarantees.\\ We evaluate all configurations on the TN3K public
benchmark using a fixed split and directly compare them with state-of-the-art methods. The plain backbone already improves upon previously reported results, with CBAM + Dice-BCE emerging as the best overall configuration.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
RARA: Robust Policy Fine-Tuning through Retrieval-Anchored Representation Alignment for Robotic Manipulation
Xuhui Kang, Sung-Wook Lee, Yan Zhuang et al. (+2 more)
Pre-trained
robot policies, such as
Vision-
Language-Action (VLA) models, provide a strong foundation for generalist robotic manipulation, but adapting them to new tasks with only a few demonstrations remains challenging. Directly fine-tuning a pre-trained policy often achieves strong in-distribution performance, yet it can distort the representations learned during pre-training and weaken the model's generalization ability under out-of-distribution shifts. We propose
Retrieval-Anchored Representation
Alignment (RARA), a two-stage adaptation method, to address this issue. First, RARA retrieves representation features from pretraining with similar task intent and motion, and aligns the policy's representations of the target demonstrations with the pre-trained representations of the retrieved demonstrations. Second, RARA fine-tunes the policy on the target demonstrations while limiting how much its representations of the retrieved demonstrations change during fine-tuning. By reducing the representation mismatch before action learning and preserving pre-trained knowledge during adaptation, RARA limits unnecessary representation drift during fine-tuning. Experiments with
Diffusion Policy and GR00T on 13 tasks in RoboTwin, RoboCasa, and on a real
robot show that RARA improves the average success of fine-tuning by 9.5 points in-distribution and 13.6 points out-of-distribution, and outperforms all compared baselines. Project page: https://shua-kang.github.io/RARA_
robot/
Read the full abstract ▾
All authors Xuhui Kang, Sung-Wook Lee, Yan Zhuang, Tianle Zhong, Yen-Ling Kuo
Computer Vision
What We Talk About When We Talk About Emotion Editing
Xingzu Zhan, Lin Gu and Ruogu Fang
Emotional image editing aims to change the emotion a scene conveys while preserving its essential content. Because the same scene can support different emotional interpretations, achieving an emotional goal requires visual changes grounded in the source context. Motivated by the Component Process Model, which links emotional responses to event appraisal, we investigate neural representations as a reference for these editing decisions. Using TRIBE v2, an fMRI encoding model, we analyze 400 source-target pairs across eight emotion categories. Hidden states from EmoEdit, EmoEditor, and AffectDelta correspond more strongly to predicted target-image responses in visual regions than in affect-related regions. This imbalance motivates ROI-ReFT, which combines source visual responses as a content reference with a predicted affective response change as an editing direction. Two interacting branches use both conditions to guide a frozen editor throughout generation. Predicted responses of the generated image supervise affective change and constrain visual drift. On a post hoc selected subset of 400 editing requests, ROI-ReFT achieves 91.75% target-emotion accuracy and 83.25% joint success, with the same output both matching the target emotion and reaching source-output DINO-I of at least 0.80. Our code and
dataset will be publicly released.
Read the full abstract ▾
Computer Vision
Learning Holistic Scene Layout against View Shifts in Cross-View Geo-Localization
Chao Yang, Songlian Li and Zhigang Tu
Cross-view geo-localization remains challenging due to drastic appearance variations and
geometric distortions caused by viewpoint changes. Existing methods primarily rely on discriminative feature learning or explicit
geometric alignment, yet often overlook an important observation: humans tend to match scenes by integrating the spatial context of multiple landmarks rather than relying solely on isolated local visual cues. Inspired by this, we propose a Spatial Context Modeling (SCM) module, which traverses feature maps along four orthogonal directions and models interactions among the resulting directional representations at each spatial position. By aggregating complementary directional observations, SCM constructs richer scene layout representations. Since reliable spatial context depends on robust visual features, we introduce a High-Frequency Information Guidance (HFIG) module that extracts stable structural cues from the frequency domain. By emphasizing edges and contours that are less sensitive to appearance variations, HFIG encourages the model to learn more robust representations. Extensive experiments demonstrate that our method achieves competitive performance on standard cross-view geo-localization
benchmarks while exhibiting strong robustness under more challenging scenarios with unknown orientations and limited fields of view (FoVs).
Read the full abstract ▾
Computer Vision
ELMA: Benchmarking Anaphoric Compositions for Long-Term Text-to-Motion Generation
Shun Kato, Dixin Yang, Kotaro Amaya et al. (+5 more)
Text-guided long-term human motion generation is essential for applications such as continuously operating digital avatars. Despite recent progress on short clips, existing models struggle with multi-segment instructions that involve anaphora, where constraints established in earlier segments must be preserved over time. In this paper, we introduce a novel task of long-term motion generation from such anaphoric compositions. To mitigate this, we present ELMA, an episodic long-term motion
dataset with dense segment-level annotations that preserve cross-segment dependencies. Built via a fully automated collection and filtering pipeline, ELMA extracts high-quality, continuous 30-second clips from in-the-wild
videos, capturing the episodic narratives necessary for long-range state modeling. Leveraging this
dataset, we fine-tune state-of-the-art motion generation models to incorporate long-range context. Extensive experiments reveal fundamental shortcomings in current models when handling context-dependent instructions. We demonstrate that ELMA enables significant improvements in both global physical realism and semantic
alignment, setting a robust new baseline for long-term motion synthesis.
Read the full abstract ▾
All authors Shun Kato, Dixin Yang, Kotaro Amaya, Yuiga Wada, Kazumi Fukuda, Takashi Shibuya, Yuki Mitsufuji, Mariko Isogawa
Computer Vision
Auditing VLM Benchmarks from Existing Evaluation Runs
Denis Degtiarev, Anna Gromova, Sergey Katkov et al. (+5 more)
Every evaluation run of a
vision-
language model (VLM) already produces per-question artifacts (predictions, judged correctness, metadata) that leaderboards reduce to a single score and then discard. We reuse those artifacts to audit the
benchmark itself, and add an audit layer to VLMEvalKit: six detectors that run as post-processing over saved outputs, plus one additional text-only rerun with visual inputs removed, emitting a per-question report. The layer reports per-question findings together with aggregate audit statistics; for the headline quantities, it attaches permutation-null excess where applicable, interval estimates for proportions, and sensitivity to the model pool. Auditing MMStar, a
benchmark curated specifically so that the image is indispensable, with three models, we find zero aggregate accuracy gain from the image on 20.2% of its questions. An item-matched permutation null accounts for 17.0%, leaving a +3.2 pp excess (corrected permutation p = 0.001). This excess is concentrated in the mathematics and science categories and is absent from coarse perception, a localization consistent with knowledge substituting for visual evidence; confirming that mechanism would require item-level review. A second detector flags 134 questions on which every model that answered in parseable form picked the same non-gold option, and these candidates peak in the perception categories, whereas the zero-gain excess is largest in mathematics and science & technology. Replacing the LLM judge with exact string matching changes no conclusion, and every scoring discrepancy between the two traces to one model whose free-form answers a parser cannot read. On a binary yes/no
benchmark the detectors measurably fall out of calibration, a scope boundary that we measure and report. The audit layer, the prediction artifacts and the generated reports accompany this submission.
Read the full abstract ▾
All authors Denis Degtiarev, Anna Gromova, Sergey Katkov, Artem Mikhailin, Safina Mukhamedshina, Albert Nasybullin, Karina Shaikhutdinova, Artem Tuzov
Computer Vision
MOSAIC: Multi-Scale Object-Centric Slot Adaptation with Image-Guided Prototype Conditioning for Remote Sensing Image Super-Resolution
Liuqian Wang, Hanyu Xie, Rongzhen Zhao et al. (+6 more)
Pretrained
diffusion Transformers provide powerful generative priors for image restoration, yet adapting them to structure-sensitive tasks remains challenging. Existing methods mainly rely on dense low-resolution features as conditions, which often entangle salient objects with dominant backgrounds and provide limited abstraction for recurring structures across scales and samples. We propose MOSAIC, an object-centric representation learning framework that learns compact structural conditions for generative model adaptation. MOSAIC organizes degraded observations into structural slots that capture complementary object-level and layout-level evidence, integrates them across resolutions, and retrieves recurring priors from a shared prototype
memory to complement incomplete image-specific information. The resulting representations are injected into selected layers of a frozen
diffusion Transformer through lightweight residual modulation, preserving its pretrained generative prior. We instantiate MOSAIC for remote sensing image super-resolution, where small objects, repetitive structures, and large-scale variation provide a challenging testbed. Experiments on UCMerced, AID, and RSSCN7 show consistent PSNR improvements across $\times 2$ and $\times 4$ settings and favorable LPIPS results at $\times 4$. Qualitative results further demonstrate improved preservation of small objects, structural boundaries, and repetitive spatial patterns.
Read the full abstract ▾
All authors Liuqian Wang, Hanyu Xie, Rongzhen Zhao, Qi Ming, Chunjie Ma, Wentao Chao, Yufei Gao, Chengming Liu, Lei Shi
Computer Vision
Efficient Confidence-Aware Densification for 3DGS
Manuel Hofer, Thomas Köhler, Felix Windisch et al. (+3 more)
3DGS-MCMC has replaced the gradient-driven density control of 3D Gaussian Splatting with opacity-based sampling and relocation, and is now the de facto standard for prior-free densification under a fixed primitive budget. Because opacity stays high where the scene is already well reconstructed, this sampling sends most new primitives to the foreground and leaves the background under-reconstructed. At low budgets, prior-based methods such as ConeGS and EDGS outperform it, at the cost of a pre-trained radiance field or dense image matches and a separate initialization stage. We argue for a comeback of error-driven densification within the fixed-budget MCMC framework. Our confidence-aware densification keeps the fixed budget and gradual growth of 3DGS-MCMC, but places primitives by the image error instead of the opacity. New primitives are drawn mostly where screen-space gradients are large. We favor primitives with a low learned confidence, which marks regions the reconstruction still explains poorly, and relocate the primitives that contribute least to the rendered images to these places. Averaged over 24 scenes, our method outperforms 3DGS-MCMC in PSNR, SSIM and LPIPS at every budget from 100k to 2M primitives. At 100k, without a prior or an initialization stage, it recovers 0.84dB of the 1.05dB PSNR by which ConeGS leads 3DGS-MCMC, and from 500k on, it has the highest PSNR and SSIM of all fixed-budget methods we compare with, including ConeGS and EDGS. While inherently prior-free, our densification can also exploit priors: from the ConeGS or EDGS initialization, it matches ConeGS in PSNR and SSIM at 100k and is best in all three metrics at 500k.
Read the full abstract ▾
All authors Manuel Hofer, Thomas Köhler, Felix Windisch, Bernhard Kerbl, Markus Steinberger, Lukas Radl
Computer Vision
Knowing Whether, Not Just Where: Anchor-Probing Distillation for Open-Set Remote Sensing Visual Grounding
Zhihao Chang, Lei Zhai, Shuyuan Yang et al. (+1 more)
Open-set visual grounding in remote sensing requires models to determine not only where a queried object is located, but also whether the image provides sufficient evidence for the query at all. Existing closed-set grounding models, however, tend to return plausible boxes even for absent objects, blurring localization confidence with genuine Visual Support. We introduce AnchorProbe-Distill (APD), a teacher-student framework that recasts open-set grounding as Visual Support learning. APD builds on the insight that supported queries should induce coherent spatial anchors, whereas unsupported or mismatched queries tend to produce unstable or misleading localization patterns. APD-T therefore uses the Original Query as a Reference Anchor to interpret query perturbations: Semantic-Preserving Queries retain the annotated target, Near-Category Ambiguous Queries are resolved according to whether they form a reliable alternative anchor or collapse onto the Reference Anchor, and Far-Category Counterfactual Queries provide rejection supervision. This reference-aware
reasoning is distilled into APD-S, enabling unified localization and rejection from a single image–query pair. Experiments show that APD enables the resulting model to reject absent targets while also improving its localization of present targets.
Read the full abstract ▾
All authors Zhihao Chang, Lei Zhai, Shuyuan Yang, Zhixi Feng
Computer Vision
S-Space: Exploring Spatial Workspace in Multimodal Language Models
Zhiyuan Shi, Xiang Li, Xingzhuo Guo et al. (+8 more)
Multimodal language models (MLLMs) exhibit promising spatial
reasoning capabilities, yet the internal mechanisms supporting them remain poorly understood. We uncover S-Space, a linear representation subspace in which object-
token representations encode continuous coordinates along horizontal, vertical, and distance axes. Although we identify these axes by differentiating spatial-answer scores with respect to object-
token activations, S-Space is intrinsic rather than induced by specific data or
prompts, remaining readable under non-spatial questions. We show that S-Space functions as an internal spatial workspace of MLLMs. Perception encodes object locations into it, forming spatial representations that can be directly read out to support accurate spatial judgments. Internal
reasoning transforms it for changing viewpoints. Verbal responses selectively access it, evidenced by causal interventions that alter spatial judgments while largely preserving non-spatial responses. Case studies further show that S-Space assigns locations to unseen entities and the observer, and connects physical directions with abstract linguistic meanings, suggesting both the capabilities and limitations of large-scale
language training. Finally, we use S-Space as a diagnostic tool for spatial
reasoning failures. Direction-word readouts reveal systematic coupling between cardinal and image-relative reference frames. More importantly, a perception-computation decomposition exposes a key bottleneck: MLLMs can form capable spatial representations, yet fall short in the spatial transformations required to reason reliably across viewpoint changes.
Read the full abstract ▾
All authors Zhiyuan Shi, Xiang Li, Xingzhuo Guo, Shangchen Miao, Weiliang Chen, Qiyu Dai, Jinshan Ren, Jiangran Lyu, Fangfu Liu, Mingsheng Long, Jialong Wu
Computer Vision · Cross-listed from cs.RO
Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics
Peiyan Tu, Cong Wang, Hanxin Zhu et al. (+2 more)
Action-conditioned world models offer a promising foundation for generalist robotic intelligence by predicting the consequences of actions. However, existing robotic
datasets provide limited coverage of the diverse interaction dynamics required for generalizable world modeling. Large-scale egocentric human
videos offer complementary interaction experience, yet the embodiment gap between human hands and robotic manipulators hinders effective cross-embodiment learning. Meanwhile, the predominance of successful demonstrations leaves failure dynamics substantially underrepresented, limiting the ability of world models to anticipate unsuccessful executions. To address these limitations, we introduce HF-WM, an action-conditioned world model that combines Human interaction learning with Failure-aware post-training. To bridge the embodiment gap, we develop a dual-branch framework that couples
video generation with auxiliary pointflow prediction, using stabilized pointflow as a unified motion representation across human and
robot interactions while suppressing egocentric camera motion. To improve failure modeling, we further perform Group Relative Policy Optimization (GRPO) post-training on perturbed actions derived from successful demonstrations, guided by action
alignment and physical plausibility rewards, enabling the model to learn plausible failure dynamics without paired ground-truth
videos. Extensive experiments demonstrate that HF-WM achieves state-of-the-art performance in action-conditioned
video generation and exhibits strong utility for policy evaluation in downstream robotic tasks.
Read the full abstract ▾
All authors Peiyan Tu, Cong Wang, Hanxin Zhu, Xin Jin, Zhibo Chen
Computer Vision
SplatQ: Interaction-Aware Pruning of 3D Gaussian Splatting via Quadratic Selection
Hongbin Ye, Haixu Liu, Shenwei Kang et al. (+13 more)
Existing 3D Gaussian Splatting (3DGS)
pruning methods primarily rely on global Top-K selection based on independent importance scores, lacking regional budget control and ignoring joint deletion effects. Consequently, they cannot ensure balanced reconstruction quality across a scene. We propose SplatQ, a combinatorial
pruning algorithm that coordinates regional budget allocation and joint Gaussian selection under a fixed budget. SplatQ partitions Gaussians by spatial proximity and appearance similarity into size-limited blocks, decomposing global selection into subproblems of a specific size. It then smooths importance-based block retention quotas to prevent excessive budget concentration. Finally, it constructs a quadratic approximation of joint deletion effects using the responses of representative pixel colors to changes in Gaussian opacity. Individual responses, pairwise interactions, and quota penalties are incorporated into a quadratic unconstrained binary optimization (QUBO) objective, solved using simulated annealing (SA) or a coherent Ising machine (CIM) to jointly select Gaussians. Experiments on 5
benchmarks demonstrate that SplatQ outperforms state-of-the-art
pruning methods in aggregate reconstruction quality, spatial quality balance, and rendering speed. Ablations confirm that budget smoothing improves local detail reconstruction, while interaction modeling improves Gaussian selection quality under a fixed budget.
Read the full abstract ▾
All authors Hongbin Ye, Haixu Liu, Shenwei Kang, Wenxiu Wei, Tinglan Wang, Hongdong Zhu, Fan Tang, Qi Gao, Yin Ma, Kai Wen, Hai Wei, Nuo Qian, Wei-Shi Zheng, Shaobo Chen, Fa-Ting Hong, Jingke Meng
Computer Vision
Stride: Scale-Aware Multi-Agent Evolutionary Refinement for Infographic Generation
Yan Zhou and Yang Yang
Infographic generation challenges text-to-image models because a single image must satisfy numerous visual, textual, and semantic constraints. Iterative agentic refinement with a
multimodal verifier is a natural remedy, but existing methods face a scale mismatch: local edits cannot correct global composition errors, while full regeneration may destroy already-correct regions. We introduce Stride, a constraint-grounded multi-
agent framework that treats refinement over any frozen generator as a history-aware
evolution strategy. Stride separates constraint evaluation - whether each requirement is satisfied - from visual grounding - where and how it is violated. It then routes global revisions to a composer subagent and local corrections to a retoucher subagent. Each subagent conducts internal planning and generates multiple candidates, while Stride accepts a candidate only when it improves the verifier score, reducing regression across iterations. Because
stochastic generators and imperfect verifiers make any single step unreliable, it is this combination of scale-aware routing, per-step redundancy, and monotonic acceptance that yields stable improvement across iterations. With fully open-weight generator-verifier systems, Stride outperforms proprietary systems including Nano-Banana-2.0 and Seedream-5.0-Pro on BizGenEval, matches GPT-Image-2.0 and surpasses Seedream-5.0-Pro on IGenBench, and achieves a new state-of-the-art score on WISE, while substantially outperforming existing agentic methods across all evaluated
benchmarks.
Read the full abstract ▾
Computer Vision
GraftOOD: Grafting Out-of-Distribution Exemplars into Advantage-Weighted Rectification for Video Reward Optimization
Zheming Fu, Ruizhe He, Wenbin Ning et al. (+7 more)
Video reward optimization can increase proxy rewards while reducing
video dynamics, a mismatch we identify as motion collapse, including under a
video-level motion-quality reward. We propose GraftOOD, which replaces part of the online rollouts in selected
prompt groups with out-of-distribution
video exemplars. A shared velocity-regression objective combines group-relative reward rectification with sequence-level exemplar supervision, requiring neither exemplar generation trajectories nor an additional supervised-loss coefficient. We characterize its pointwise optimal field as a source-posterior-weighted combination of the two source-specific target fields. We construct GraftSet-
Video, containing 9,588 format-matched exemplars for 2,397
prompts, to supply external supervision. On Wan2.1-1.3B, GraftOOD raises Dynamic Degree from 4.86 to 44.41 relative to reward-only FlowAWR, while HPSv3 decreases from 8.10 to 7.32 and VisionReward remains close. At the main setting, GraftOOD leads nine of ten metrics among three exemplar-augmented methods. Ablations of exemplar frequency and within-group count reveal dynamics–reward trade-offs rather than uniform gains from increasing exemplar usage. These results support external exemplar supervision as a complement to scalar reward feedback, without equating higher measured dynamics with coherent, visually plausible motion in the generated
videos. Code is available at \href{https://github.com/Fuzm7/FlowAWR}{GraftOOD}.
Read the full abstract ▾
All authors Zheming Fu, Ruizhe He, Wenbin Ning, Xiaoxiao Ma, Xuan Yang, Lianrui Mu, Haoyang Huang, Chang Liu, Lei Wang, Siming Fu