Computer Vision
Predicting Multiple Clinical Outcomes Related to Functional Recovery and Social Isolation Among Older Adults After Lower-Limb Fracture or Hip Replacement
Santosh Ray, Pratik K. Mishra, Ali Abedi et al. (+3 more)
Older adults recovering after lower-limb fracture or hip replacement may experience complex recovery trajectories. Most of the time, these clinical aspects are studied in isolation, masking their joint impact on recovery. This study used the MAISON-LLF
dataset, which contains
multimodal sensor and clinical assessment data from 18 older adults recovering in the community after lower-limb fracture or hip replacement. Participants were monitored for up to eight weeks, corresponding to a maximum of 1,008 participant-days of sensor monitoring. Forty-six daily features were extracted from indoor motion, acceleration, step count, heart rate, out-of-home mobility, and sleep data. Five clinical outcomes were assessed every two weeks: the Social Isolation Scale, Oxford Hip Score, Oxford Knee Score, Timed Up and Go test, and 30-second Chair Stand test. We utilize an inherent relationship between multi-modal sensor data and different clinical scores and formulate it as a multi-output regression problem. We tested various machine learning and deep learning single- and multi-output regression
algorithms to predict these scores simultaneously. The results showed that predicting clinical scores jointly was better than separately. The tabular DL multi-output regressor, NODE, gave a remarkable performance of MSE=3.96 and MAE=1.02 in comparison to other multi- and single-output regressors. The SHAP feature analysis further showed the importance of including
multimodal sensors to provide a good estimate of patients' recovery trajectory. This work may support the simultaneous assessment of functional recovery and social engagement among community-dwelling older adults and ultimately help improve their care and quality of life.
Read the full abstract ▾
All authors Santosh Ray, Pratik K. Mishra, Ali Abedi, Charlene H. Chu, Amir Ahmad, Shehroz S. Khan
Computer Vision
Investigating Relational Reasoning in VLMs
Adhithya Laxman Ravi Shankar Geetha, Aulia Kharis Rakhmasari, Haleema Ramzan et al. (+1 more)
Vision-
Language Models (VLMs) achieve strong performance in visual
reasoning tasks, but it remains unclear whether they understand visual relations, or simply employ shortcuts such as
language cues or priors. To investigate this, we use the Qwen3-VL-4B (Bai et al., 2025), a modern VLM, to decode how visual information is encoded across depths. For this, we propose a synthetic
dataset of simple
geometric shapes for controlled analysis, along with queries crafted to precisely test
language cues. Furthermore, the
dataset is modified to test causal reliance on visual evidence. Our results show that current VLMs combine genuine visual
reasoning with shortcut strategies primarily rooted in
language cues.
Read the full abstract ▾
All authors Adhithya Laxman Ravi Shankar Geetha, Aulia Kharis Rakhmasari, Haleema Ramzan, Xander Yap
Computer Vision
Action-Aligned Retrieval with Pairwise Multimodal Reranking for Text-Based Person Anomaly Search
Thanh-Khoi Nguyen, Thanh-Nhan Vo, Trong-Thuan Nguyen et al. (+1 more)
Text-based person anomaly search requires distinguishing individuals based on fine-grained, context-dependent behaviors rather than mere appearance. Existing methods struggle to capture these context-conditioned actions, frequently relying on isolated skeletal geometry, discarding raw query details during reformulation, or utilizing absolute pointwise scoring for
multimodal verification. To address these limitations, we propose ActPair, a unified three-stage coarse-to-fine framework that combines action-aligned
retrieval with pairwise
multimodal reranking to bridge the pose-semantic gap. First, we fine-tune a
vision-
language model (VLM) with an action-aligned multi-task objective that encourages the representations to encode action-discriminative semantics. Second, we perform parallel late-fusion
retrieval using the original query and a large
language model (LLM)-generated context-grounded rewrite, retaining complementary details from both semantic views. Finally, we propose an efficient off-the-shelf reranking module that leverages a pivot-promote
algorithm to perform direct pairwise visual comparisons, mitigating residual spatial and compositional ambiguities without the prohibitive inference costs of exhaustive evaluation. Extensive experiments demonstrate that our framework achieves the best results among the compared methods on the Pedestrian Anomaly Behavior (PAB) public test and transfers effectively to an unseen, non-anomaly-specific
dataset.
Read the full abstract ▾
All authors Thanh-Khoi Nguyen, Thanh-Nhan Vo, Trong-Thuan Nguyen, Minh-Triet Tran
Computer Vision
SVD-Based Typicality Maps for Out-of-Distribution Detection in Vision Transformers
Aldo Sean Sartor, Leandro de Souza Rosa, Andriy Enttsel et al. (+2 more)
We present a method for analyzing the internal representations of
Vision Transformers (ViTs) exploiting the geometry of their learned parameters. Each affine layer's weight
matrix is factored via Singular Value Decomposition (SVD), and activations are projected onto the leading right singular vectors to obtain compact, layer-intrinsic representations. A class-conditional density model is then fitted at each layer, producing per-class typicality scores that are stacked across depth into typicality maps: two-dimensional summaries of how class-specific evidence evolves through the network. From these maps, we derive two post-hoc scores for Out-Of-Distribution (OOD) detection: a Prototype
Alignment Score (PAS), measuring agreement with class reference prototype patterns, and a Multi-Layer Soft Voting (MLSV) score, capturing cross-layer consensus without stored prototypes. On ViT-B/16 fine-tuned on CIFAR-100, the proposed scores achieve competitive detection performance without retraining or OOD exposure.
Read the full abstract ▾
All authors Aldo Sean Sartor, Leandro de Souza Rosa, Andriy Enttsel, Mauro Mangia, Riccardo Rovatti
Computer Vision
GeoWAM: Visual Geometry World Action Models for Autonomous Driving
Yiren Lu, Xin Ye, Jiaming Liu et al. (+10 more)
World action models (WAMs) have recently gained increasing
attention as a framework for jointly modeling scene
evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a
video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that point-based geometry provides a more natural state space for driving. It explicitly captures spatial structure and both rigid and non-rigid scene dynamics while remaining aligned with the 3D space of driving actions. Building on this insight, we introduce GeoWAM, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal
evolution. A geometry-conditioned action head then leverages these learned
geometric dynamics to predict future ego-trajectories. Extensive experiments show that GeoWAM outperforms image-based alternatives, achieving a combined EPDMS of 36.6 on navhard without PDMS supervision and strong zero-shot generalization to nuScenes, with a collision rate of 0.24%. Scaling geometry pretraining with unlabeled data further improves performance, increasing the navhard score by 8.2% to 39.6 and strengthening zero-shot transfer to nuScenes, where the collision rate is reduced by 50% to 0.12%. Together, these results establish geometry as an effective state representation for autonomous driving and geometry pretraining as a general, scalable strategy for downstream planning.
Read the full abstract ▾
All authors Yiren Lu, Xin Ye, Jiaming Liu, Philip Jacobson, Jin Yao, Yi-chung Chen, Liam Merino, Dhruva Dixith Kurra, Min Cai, Tom Lampo, Yu Yin, Danhua Guo, Burhan Yaman
Computer Vision
Geometry-Driven Opti-Acoustic Co-Registration and View-Invariant Reflectivity Mapping for Side-Scan Sonar
Taqi Hamoda and Nuno Gracias
Side-Scan Sonar (SSS) is a primary modality for large-scale
underwater mapping, yet automated perception and cross-modal
alignment are severely bottlenecked by acoustic complexities such as speckle noise, shadows, and extreme viewpoint dependencies. Traditional handcrafted descriptors and modern deep learning matchers fail to bridge the physical domain gap between optical and acoustic imagery without 3D
geometric constraints. To overcome these limitations, we propose a novel geometry-driven framework for pixel-level opti-acoustic co-registration and view-invariant reflectivity mapping. Our method utilizes Structure-from-Motion (SfM) to reconstruct a dense 3D seafloor mesh, acting as a
geometric anchor between the visual and acoustic domains. We introduce a First Bottom Return (FBR) extraction
algorithm to dynamically correct non-linear altitude drift caused by uncalibrated SfM reconstruction. Furthermore, we apply an inverse Lambertian model and a dual-Gaussian weighting function to isolate the intrinsic seabed reflectivity, effectively neutralizing slant-range propagation loss and
geometric view-dependence. By deterministically associating these isolated acoustic properties with optical pixels, our pipeline generates highly accurate, strictly co-registered multi-modal
datasets. This automated, physics-guided approach eliminates the need for manual annotation and paves the way for advanced self-supervised learning in benthic habitat mapping.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models
Sangoh Lee, Sangwoo Mo and Wook-Shin Han
Vision-
Language-Action (VLA) models can turn
multimodal context into
robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather than the shared semantic objective of the forthcoming behavior. We propose Intention
Distillation (INDI), which distills behavior-level intent into the action decoder. During training, a frozen teacher VLM interprets a demonstrated segment from the current observation, instruction, coarse action summary, and corresponding execution
video. From its standard inputs, the deployed VLA recovers the resulting
multimodal intent representation at an intermediate decoder layer and uses it to organize action prediction together with representations of how the behavior unfolds and what it achieves. On SimplerEnv-Bridge, INDI improves GR00T-N1.7 from 64.3% to 84.7%, and on RoboCasa Kitchen it improves the controlled GR00T-N1.7 baseline from 64.1% to 70.3%, with consistent gains on $\pi_{0.5}$ across both
benchmarks. In real-world tasks, INDI improves average success from 62.0% to 68.7%, with gains of up to 12.0 pp on longer-horizon tasks. Further analyses show that the recovered latent is used by the decoder, captures behavior objective and execution progress, and organizes downstream predictions in an objective-dependent manner. These results show that action decoders benefit from explicitly modeling the semantic objective of the behavior they generate.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.CL
What's the Catch? Evaluating Temporal Consistency in Vision-Language Models
Marek Hradil and Danae Sánchez Villegas
Vision-
language models (VLMs) achieve strong performance on
video and image-sequence
benchmarks, yet it remains unclear whether they capture temporal structure. To study this question, we formulate temporal grounding as an anomaly detection problem, providing a simple and controlled evaluation that directly tests sensitivity to temporal consistency. We introduce TimeCatch, where temporal anomalies are created by swapping consecutive frames and frame-level anomalies by replacing a frame with Gaussian noise. Models are evaluated on anomaly detection and localization tasks across four synthetic and real-world
datasets, alongside a human study. Our evaluation reveals a substantial gap between frame-level and temporal anomaly detection. While VLMs consistently detect frame-level anomalies and often localize them accurately, under our main evaluation setting they generally perform near chance on temporal anomaly detection and show limited localization performance. Humans, in contrast, achieve near-ceiling performance on both tasks. Additional analyses across model scales, prompting strategies, sequence lengths, and visual similarity show that performance can improve under some conditions, while substantial gaps in temporal anomaly detection and localization remain. Together, these findings reveal a gap between frame-level and temporal anomaly detection. TimeCatch provides a controlled
benchmark for evaluating temporal consistency in
vision-
language models.
Read the full abstract ▾
Computer Vision
Towards Comprehensive Basketball Understanding
Yirong Hu, Jiayuan Rao, Yu Zhang et al. (+2 more)
Understanding a basketball game requires recognizing events, localizing actions, identifying players, and relating these to structured game knowledge. Existing
benchmarks primarily evaluate these abilities one at a time, leaving the interactions among these abilities under-explored. We introduce BasketballBench, a
multimodal benchmark comprising 7,980 questions across ten tasks in text, image, and
video. It is built from the 2025-2026 NBA season and includes official playby-play, rosters and profiles for 530 active players, and 2,501 possession-level broadcast clips. We further propose BasketballSkills, an
agent that composes eight basketball-specific perception and
retrieval tools under four reusable skills that specify tool order, evidence bindings, and stopping conditions. Experiments show that current MLLMs struggle particularly on questions requiring the integration of multiple capabilities, whereas BasketballSkills outperforms them, highlighting the effectiveness of explicitly composing domain-specific capabilities for comprehensive basketball understanding.
Read the full abstract ▾
All authors Yirong Hu, Jiayuan Rao, Yu Zhang, Shangzhe Di, Weidi Xie
Computer Vision
Image-Conditioned Diffusion Models for Quality Assurance of Organ-at-Risk Segmentations in Radiotherapy
Clea Dronne, Catharine H Clark, Xavier Loizeau et al. (+3 more)
Accurate organ-at-risk
segmentation is essential for radiotherapy planning, but reviewing
segmentations is time-consuming and subjective. We investigate normative modelling for
segmentation error detection in head-and-neck CT, comparing a VAE framework with an image-conditioned
segmentation diffusion model. Models were evaluated on RADCURE brainstem and spinal cord
segmentations using simulated boundary and width perturbations. Error detection was assessed using the Dice similarity coefficient and the Distance to Agreement (DTA) between the input and reconstructed
segmentations. While both models detected some simulated errors, regional DTA showed that the
diffusion model localised subtle boundary errors more consistently. These results support image-conditioned
diffusion reconstruction as a promising framework for localised, anatomy-aware
segmentation QA.
Read the full abstract ▾
All authors Clea Dronne, Catharine H Clark, Xavier Loizeau, Elizabeth Miles, Peter Hoskin, Jamie R McClelland
Computer Vision
Photorealistic Novel View Synthesis of Human Faces using Next-Scale Transformers
Federico Stella, Fei Jiang, Zhongshi Jiang et al. (+4 more)
Photorealistic novel view synthesis of people remains challenging at high spatial resolutions and across multiple target cameras, where preserving identity, fine appearance details, and
geometric coherence is critical. We build on the next-scale autoregressive paradigm and adapt it for human-centric view synthesis by enabling higher image resolutions, multi-view outputs and stronger cross-view consistency in a single forward pass. We train on a synthetic
dataset of human faces spanning diverse identities and apparel. Contrary to
diffusion models, this paradigm does not need 2D pre-training and, thanks to its next-scale architecture, it benefits from lower-resolution, general-purpose pre-trainings, with the full-sized purpose-specific images being used only in the last training stages. This enables our architecture to converge with a smaller amount of purpose-specific training data, allowing us to use a smaller but more realistic training
dataset. The resulting model produces sharp and realistic views, with the option to synthesize multiple novel viewpoints simultaneously for improved agreement across views. Empirically, we observe gains in perceptual fidelity and cross-view coherence on human subjects, demonstrating that next-scale autoregression is an effective backbone for scalable, multi-output human view synthesis. We also couple our pipeline with an existing
transformer-based model for pixel-aligned 3D gaussian lifting from multi-view facial inputs, resulting in accurate and photorealistic 3D models of human faces.
Read the full abstract ▾
All authors Federico Stella, Fei Jiang, Zhongshi Jiang, Zohar Barzelay, Emanuel Garbin, Amin Jourabloo, Liuhao Ge
Computer Vision
MomADv2: Reliable Temporal Memory for End-to-End Autonomous Driving
Ziying Song, Shengkai Zhang, Lin Liu et al. (+8 more)
Long-horizon planning is critical for safe autonomous driving in complex scenarios. Existing methods improve planning continuity with temporal
memory, but such
memory may become invalid and mislead decisions when the driving command changes. Thus, selectively leveraging useful history while suppressing command-inconsistent
memory remains a key challenge. To address this issue, we propose MomADv2, a reliable state-space
memory framework for long-horizon end-to-end autonomous driving. At its core, MomADv2 introduces a Selective State-Space Planning
Memory Query Module, which filters historical planning queries based on temporal continuity and command consistency, selects planning modes relevant to the current command, and models the
evolution of planning intentions through a selective state-space mechanism. To further alleviate local trajectory deviations and error accumulation in long-horizon planning, we design a Flow-Matching Trajectory Residual Refiner. It learns a continuous residual correction field from the refined planning output to the expert trajectory, enabling fine-grained trajectory refinement while preserving the stability of anchor-based planning. Extensive experiments on closed-loop NAVSIM and Bench2Drive, as well as open-loop nuScenes, demonstrate that MomADv2 improves long-horizon planning consistency and reduces the average collision rate by 15.6% over MomAD under 6-second planning.
Read the full abstract ▾
All authors Ziying Song, Shengkai Zhang, Lin Liu, Peiliang Wu, Lei Yang, Dongyang Xu, Bin Sun, Li Wang, Shaoqing Xu, Caiyan Jia, Yadan Luo
Computer Vision
Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
Nan Duan, Haoyang Huang, Weiyang Jin et al. (+13 more)
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts. We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants. The long-
video variant introduces composable cross-shot
memory that aggregates visual evidence across multiple prior shots and speaker cues derived from
speech-filtered full-shot audio, enabling persistent character appearance and voice identity across flexible combinations of text, image, and
memory conditioning. The world-model variant converts heterogeneous
navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway, enabling controller-agnostic interaction across flexible viewpoints. To support efficient long-horizon generation, we transform a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. Experiments demonstrate strong performance in both settings. JoyAI-Echo-1.5 achieves improvements over existing long-
video baselines in cross-shot consistency, visual quality, text
alignment, and
speech fidelity. Its world-model variant ranks first on WBench, with an average score of 81.7, and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench. Together, these results indicate that
memory,
geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds. Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/.
Read the full abstract ▾
All authors Nan Duan, Haoyang Huang, Weiyang Jin, Haoran Li, Yaowei Li, Yuming Li, Yijun Liu, Xin Lu, Xiaoxiao Ma, Yanwen Ma, Yaofeng Su, Yilang Sun, Haoyu Wang, Zeyue Xue, Songchun Zhang, Junhao Zhuang
Computer Vision
DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts
Vlad Hondru, Florinel Alin Croitoru, Iuliana Georgescu et al. (+2 more)
Audio-visual deepfake detection is an actively studied topic, where one of the main challenges is to develop detectors able to generalize across deepfake generation methods. We
conjecture that overfitting can be mitigated by extracting multiple high-level cues from the available audio and visual modalities via pre-trained models. We therefore assemble a wide variety of pre-trained models to extract features that encode mouth movements, face parsing, facial expressions, head pose, gaze tracking, heart rate, audio
emotion and
speech activity. We further integrate both unimodal and
multimodal cues via a Mixture-of-Experts (MoE) backbone to detect deepfakes. We perform in-domain and cross-domain experiments on five
benchmarks for deepfake detection (MAVOS-DD, AVLips, PolyGlotFake, BioDeepAV, FakeAVCeleb) to compare our framework (DF-MoE) with state-of-the-art methods. Our results indicate that DF-MoE obtains superior deepfake detection results, surpassing all competing methods. We release our code at https://github.com/vladhondru25/DF-MoE.
Read the full abstract ▾
All authors Vlad Hondru, Florinel Alin Croitoru, Iuliana Georgescu, A. Sophia Koepke, Radu Tudor Ionescu
Computer Vision · Cross-listed from cs.RO
Alperen Avan and Jordi Sanchez-Riera
Autonomous indoor
navigation requires both semantic understanding and precise
geometric control. We propose OptiSight, a hybrid framework that combines
Vision-
Language Model
reasoning with deterministic visual servoing through a finite-state Chain-of-Thought architecture. Grounded-SAM localizes open-vocabulary targets, while camera projection geometry converts visual observations into
navigation commands without requiring dense mapping. The VLM is queried only at key decision points, reducing computational overhead while
geometric control handles continuous
navigation. Experiments in AI Habitat demonstrate reliable zero-shot
navigation across diverse indoor scenarios, including obstacle avoidance and semantic ambiguity, while operating within an 8~GB VRAM budget. The source code is available at https://github.com/avanalperen/OptiSight-Python-
Multimodal-CoT-for-Visual-
Reasoning.
Read the full abstract ▾
Computer Vision
Controllable blind deblurring with diffusion models
Imane Si Salah, Emile Cribelier, Thomas Veit et al. (+2 more)
Image acquisition with a camera involves several degradations due to the optical system, sensor, or low-level processing steps. We address blind deblurring in professional photography: we aim to invert unknown isotropic blur without knowledge of the degradation kernel. For such inverse problems,where some high-frequency information is lost, it is challenging to use generative models to produce details that are both photo-realistic and faithful to the input. We propose SuperSharpen, a
diffusion-based blind deblurring method offering explicit control over restoration strength through a blur measure. We compare two conditioning strategies: a ControlNet-style adapter on a frozen backbone, and full finetuning of the
diffusion prior. Our experiments show that finetuning achieves better fidelity with fewer hallucinated details. We validate our approach on synthetic and real-world blur, demonstrating improved perceptual quality and controllable restoration strength.
Read the full abstract ▾
All authors Imane Si Salah, Emile Cribelier, Thomas Veit, Wolf Hauser, Arthur Leclaire
Computer Vision
Can Coding Agents Build Robust Baselines? A Skill-Based Approach for Automating the Medical Imaging Model-Development Pipeline
Eugenia Moris and José Ignacio Orlando
Developing competitive deep learning baselines for medical imaging remains a highly iterative process requiring literature review, implementation, experimentation, and expert refinement. Existing automation approaches typically optimize isolated components, such as architecture search or hyperparameter tuning, rather than the complete baseline development process. We present an agentic AI Scientist workflow that combines literature-guided
reasoning, automated code generation, and hypothesis-driven experimentation to generate competitive baseline models for medical imaging challenges. The framework is evaluated on four public
benchmarks spanning
segmentation, classification, and detection. Across all tasks, the Experimentation Pipeline consistently improves validation performance, achieving competitive leaderboard results, including 6th place on both PUMA tracks (15 teams) and 31st place on MILK10k (125 teams). On MIDOG25, the resulting model also demonstrates strong domain generalization across scanners, tumor types, and species. Using the same workflow across all challenges without task-specific redesign, we demonstrate that skill-based, literature-guided agentic workflows can substantially reduce the engineering effort required to develop competitive medical imaging baselines.
Read the full abstract ▾
Computer Vision
IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning
Jiapeng Li, Ping Wei, Wenjuan Han et al. (+2 more)
Video understanding requires intelligent
agents to transcend mere recognition of visual facts and comprehend the underlying intents behind human actions (often termed the "dark matter" of social intelligence). To bridge the gap between visual observation and intent
reasoning, we introduce a novel task, IntentQA, and contribute a large-scale VideoQA
dataset specifically tailored for this purpose. However, recognizing that standard metrics may overestimate capabilities due to
dataset biases, we go beyond simple accuracy to rigorously evaluate model robustness. We augment the
benchmark by generating five distinct contrast sets via Large
Language Models (LLMs) and introducing a "Contrast Performance Decline" metric. We propose the X-CaVIR (eXplainable Context-aware
Video Intent
Reasoning) framework, which leverages three types of "Cognitive Context" to enhance
video analysis: i) Situational Context via a cross-modal
Video Query
Language (VQL) module, ii) Contrastive Context via a Contrastive Learning module, and iii) Commonsense Context via a Commonsense
Reasoning module. Crucially, to overcome the opacity of traditional black-box models, we refine the integration of LLMs within X-CaVIR by employing a transparent pipeline that synergizes
video captions with VQA model outputs. This approach not only improves performance by effectively utilizing rich commonsense knowledge but also renders the
reasoning process explicitly interpretable. Extensive experiments demonstrate the effectiveness of our components, the superiority of X-CaVIR over state-of-the-art baselines, and its stability against perturbations on the contrast sets.
Read the full abstract ▾
All authors Jiapeng Li, Ping Wei, Wenjuan Han, Song-Chun Zhu, Lifeng Fan
Computer Vision
Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents
Wenqi Liu, Shijie Ma, Yunxiao Wang et al. (+21 more)
Open-world
video understanding often requires a model to locate sparse visual evidence and acquire external knowledge that is absent from the
video and its parametric
memory. While Thinking-with-
Videos enables active temporal perception and Deep Research supports multi-step information seeking, the two capabilities are typically developed in isolation. We introduce VideoRover, a unified
Video Deep Research framework that iteratively coordinates
video cropping,
multimodal search, and webpage browsing. Given a
video-question pair, VideoRover uses each tool result to select the next action, so localized
video clips guide external
retrieval and retrieved evidence triggers further
video inspection and verification. To develop this capability, we construct an automated data curation pipeline, producing 26K verified SFT trajectories and 3K challenging RL instances. We also introduce VideoRover-Bench, a
benchmark stratified by
video duration and research difficulty. Experiments on VideoDR and VideoRover-Bench show that our VideoRover-8B-RL achieves performance comparable to proprietary models in the direct-answer setting without tool use while outperforming larger open-source models equipped with the same tool suite. Ablation studies and training dynamics further validate the complementary roles of active
video grounding, external
retrieval, and long-horizon reinforcement learning.
Read the full abstract ▾
All authors Wenqi Liu, Shijie Ma, Yunxiao Wang, Meng Liu, Qile Su, Han Liu, Bohan Hou, Zeyu Wang, Xuanyu Zheng, Changyi Liu, Tianke Zhang, Haonan Fan, Kaiyu Jiang, Yingxin Li, Jiankang Chen, Xu Wang, Hongyi Fu, Jianxiong Wang, Bin Wen, Tingting Gao, Han Li, Jianhua Yin, Yinwei Wei, Xuemeng Song
Computer Vision
Grounding Free-Form Instructions for Fashion Complementary Image Generation
Matteo Attimonelli, Claudio Pomo, Alessandro De Bellis et al. (+3 more)
Fashion complementary image generation (CIG) aims to create garments that stylistically match a seed item based on user intent, making it a natural
multimodal grounding problem where models must interpret
language in visual context. Existing CIG
benchmarks rely on rigid template
prompts (e.g., "a photo of a skirt"), failing to reflect natural user queries and obscuring model behavior across levels of linguistic specificity. We introduce fashion complementary image generation with free-form instructions, a
multimodal language-grounding setting where a model generates a compatible garment from a seed image and a natural-
language instruction. To this end, we enrich three CIG
benchmarks with low-, medium-, and high-specificity instructions generated by a
vision-
language model and validated by human annotators. We instantiate the task with StyleFlow, a Rectified Flow Matching model that jointly conditions on the seed image and instruction within a single
multimodal transformer. Across image quality metrics, catalog-
alignment analysis, ablations, and human evaluation, StyleFlow consistently produces instruction-aligned and stylistically coherent garments while reducing architectural complexity and inference cost relative to auxiliary-module approaches.
Read the full abstract ▾
All authors Matteo Attimonelli, Claudio Pomo, Alessandro De Bellis, Danilo Danese, Dietmar Jannach, Tommaso Di Noia
Computer Vision
What Remains Normal? Clean Images Miss Useful Near-Defect Normal Patches for Anomaly Detection
Joongwon Chae, Runming Wang and Peiwu Qin
Normal-only industrial anomaly detectors use patches from clean training images as normal references or reconstruction targets. This assumes that clean patches are sufficient for the normal regions encountered at test time. We test that assumption directly. On MVTec AD, admitting ground-truth-normal patches from real defect images to a DINOv2
memory candidate pool raises pixel average precision (P-AP) from 73.34 to 76.95 while keeping the encoder, test-time score, and number of stored references fixed. Patches within two patch
cells of the annotated defect recover 94.70% of this gain. We then ask whether useful patches of this kind can be exposed using clean training images alone. BoundarySupport inserts a procedural synthetic defect to alter surrounding context, excludes every
token intersecting the nominal insertion or a detected RGB change, and learns only from pixel-preserved neighboring patches. Across three paired seeds, the same principle improves P-AP in all six
memory and reconstruction settings across MVTec, VisA, and Real-IAD. Matched controls identify the altered-context feature itself as the useful normal evidence: with synthetic input or selected positions fixed, altered-context features outperform their clean-view counterparts as both reconstruction targets and
memory references. On MVTec
memory, the final score change is also spatially selective, with larger reductions on normal patches next to defects than on mid-distance or far-normal patches in all 15 categories. Code is publicly available at https://github.com/jw-chae/boundary_support.
Read the full abstract ▾
Computer Vision
What Memory Composition Does Not Tell Us About Anomaly Detection
Joongwon Chae, Runming Wang and Peiwu Qin
Memory-based anomaly detectors store nominal training patches and score test patches against this
memory. A patch selected for coverage therefore becomes a nor- mal reference without a separate check that
geometric rarity makes it safe to trust. We probe this coupling with sparse training contamination. Under fixed representa- tions and
memory budgets, we compare random, medoid, local, and global coverage selectors. We then use CLEANCON, an out-of-bag cross-image support gate that changes candidate-image eligibility while fixing the representation, absolute mem- ory size, builder, and inference rule. Global coverage strongly over-represents sparse contamination. CLEANCON reduces final-
memory contamination to approx- imately zero and increases category-macro P-AP in all 12 matched comparisons. Yet along a retention sweep, the lowest-contamination
memory does not attain the highest P-AP; performance continues to improve while contamination rises. Mem- ory contamination therefore does not order the resulting
memories by P-AP
Read the full abstract ▾
Computer Vision
Spotter: Efficient Urban Visual Localization via Geo-Referenced Facade Landmarks in GPS-Degraded Environments
Antoni Valls and Jordi Sanchez-Riera
Accurate visual localization on robotic and wearable platforms remains challenging in dense urban environments. Existing methodologies typically rely on GPS for absolute positioning, yet GPS signals frequently degrade in urban canyons due to multipath propagation. Consequently, standard solutions like visual odometry suffer from unmitigated drift over time, while map-matching techniques struggle to acquire the reliable GPS priors they need, on top of being too computationally heavy for real-time edge execution. To address these limitations, we propose Spotter, a robuts and real-time visual localization framework that uses building facades as a reliable source of global geo-reference, while retaining the capability to integrate GPS signals when available. In an offline stage, Spotter processes Google Street View panoramas by semantically segmenting facades and pairing multi-view stereo depth with cartographic data to build a compact metric database. At runtime, query images are matched via a cascaded
retrieval and
geometric verification pipeline to recover fine-grained global camera localization. We
benchmark Spotter on a newly collected
dataset of pedestrian sequences acquired with wearable smart glasses across several districts of Barcelona. Experimental results show that Spotter outperforms odometry-based baselines and achieves localization accuracy comparable to state-of-the-art map-based methods while operating at significantly higher frame rates.
Read the full abstract ▾
Computer Vision
Spatiotemporally Decoupled Autoregressive Diffusion Model for Human Motion Generation
Chengqun Yang, Liang Xu, Yanping Li et al. (+4 more)
Text-driven human motion synthesis has made substantial development with two core modules of motion representation and generative architecture. For representation, Vector Quantization (VQ)-based methods compress motion data into discrete
tokens while latent-based models operate directly in continuous space. However, both of these representations exhibit significant limitations. VQ-based methods suffer from inherent information loss, which compromises the quality, diversity, and generalization of generated motions, while continuous representation on holistic whole-body motion hinders part-level flexibility. For architecture,
diffusion and autoregressive
diffusion models have demonstrated their superiority, yet the fine-grained controllability over individual body parts is also limited. Thus, we propose a unified spatiotemporally decoupled framework named DeMoDiff, which jointly redesigns representation and architecture. To enhance representation extraction capabilities and offer greater part-level controllability, we present a spatial-temporal VAE that encodes each body joint rather than compressing the whole-body motion into a single latent space. Then, we incorporate spatial-temporal masking and
attention mechanisms into an autoregressive
diffusion generator, achieving both generative capability and controllable editability. Extensive experiments on the HumanML3D and KIT-ML
datasets demonstrate that our model achieves state-of-the-art reconstruction performance and compelling motion generation results. Moreover, our framework demonstrates strong temporal and spatial editing capabilities, further validating its effectiveness. Our project page: https://rex0191.github.io/DeMoDiff/
Read the full abstract ▾
All authors Chengqun Yang, Liang Xu, Yanping Li, Fulong Liu, Jingnan Gao, Weili Zeng, Yichao Yan
Computer Vision
Dual-Grained Agent Memory and Shapley Context Attribution for Multimodal Agentic Learner
Jieke Wang, Tiancheng Shen, Yibo Yang et al. (+1 more)
Frontier
multimodal large
language models (MLLMs) deliver impressive perception yet still falter on scientific and mathematical
reasoning. Parameter-level adaptation is unavailable for closed-weight or on-device backbones, and stateless prompting forfeits any compounding benefit from problems already solved. We propose DG-Mem, a dual-grained agentic
memory framework that augments a frozen MLLM with a non-parametric, externally stored
memory built once from training-time rollouts and consulted read-only at test time. Motivated by the Complementary Learning Systems (CLS) account of human
memory, DG-Mem factors its store into an instance-grounded exemplar
memory and a category-level schema
memory of IF-THEN rules, with a transient reflection store mediating their construction so that schemas are synthesized only from abstract reflections, never from exemplar text. Two design choices distinguish DG-Mem: an online concept categorizer that grows the category space incrementally during training rather than committing to a predefined taxonomy, and a Shapley context attribution procedure that decomposes correctness across the entire retrieved rule set and yields a per-rule utility that re-weights
retrieval at test time. The pipeline introduces no gradient updates and is deployable on closed-weight or on-device backbones. Across MathVista, MMMU, and MMMU-Pro on four open-weight and proprietary backbones (Qwen3.5-27B, Qwen3.5-122B-A10B, GPT-5-Nano, Gemini-3-Flash), DG-Mem improves consistently over no-
memory and competitive
memory baselines.
Read the full abstract ▾
All authors Jieke Wang, Tiancheng Shen, Yibo Yang, Ming-Hsuan Yang
Computer Vision
Progressively Learning Heterogeneous Skills in a Unified Latent Space
Yue-Yi Zhang, Ming Gong, Linpu He et al. (+2 more)
We propose HetSkills, a novel framework designed to progressively learn heterogeneous skills within a unified latent space for physics-based character control. The core idea is to treat this latent space as a shared executable interface, enabling seamless integration of skills learned from diverse data sources, supervision forms, and tasks. HetSkills begins by learning a tracking skill that establishes a strong foundation in motion control and creates a shared motion decoder, which can be reused across tasks without the need for retraining or separate controllers. To prevent the text-to-motion skill from exploiting shortcut pathways instead of learning
language semantics, we introduce motion intuition
distillation to ground text-to-motion generation in
language semantics and a task-guidance module that dynamically adjusts actions based on high-level
language instructions. This enables HetSkills to preserve natural motion while continuously expanding its skill repertoire, making it highly adaptable for long-horizon tasks. Experimental results demonstrate the effectiveness in motion tracking, text-to-motion generation, motion completion, and downstream task adaptation, achieving impressive success rates even under challenging conditions.
Read the full abstract ▾
All authors Yue-Yi Zhang, Ming Gong, Linpu He, Wei-Shi Zheng, Zhilin Zhao
Computer Vision
E2S-Pruner: Progressive Two-Stage Evidence Fusion for Visual Token Pruning in Vision-Language Models
Taoyu Qian, Qi Wang, Daqian Shi et al. (+3 more)
Vision-
language models typically encode an image into hundreds of visual
tokens, incurring substantial inference latency and GPU
memory overhead. Existing
pruning methods largely rely on
attention scores and directly aggregate outputs across
attention heads and network layers, making it difficult to characterize evidential uncertainty and conflict. We propose E2S-Pruner, a progressive two-stage evidence-fusion framework for visual
token pruning that requires no auxiliary model, trainable parameters, or fine-tuning. In the first stage, E2S-Pruner treats each
attention head as an independent evidence source, estimates its reliability from evidence clarity and inter-head consistency, and represents each visual
token using three states: important, unimportant, and uncertain. In the second stage, Dempster–Shafer evidence theory is used to quantify inter-layer conflict and fuse complementary evidence from multiple network layers. We further introduce a spatial novelty constraint that promotes coverage of distinct image regions and prevents the retained
tokens from concentrating in a few locally salient areas. On LLaVA-1.5-7B, E2S-Pruner retains 98.0%, 96.8%, and 90.6% of the aggregate performance when the average numbers of retained visual
tokens are 192, 128, and 64, respectively, while improving throughput by 1.96x and 2.09x under the 128-
token and 64-
token settings. Experiments on Qwen2-VL-7B further demonstrate cross-model generalization. Code is available at https://github.com/taoyu-qian/E2S-Pruner.git.
Read the full abstract ▾
All authors Taoyu Qian, Qi Wang, Daqian Shi, Yuanhao Jiang, Shang Gao, Hualong Yu
Computer Vision
Semantic Reconstruction and 3-D Detection via Learned Multi-Pair Fusion in RF Imaging
Amir Rezaei, Wen-Xin Pan and Giuseppe Caire
We consider a multistatic
radio-frequency imaging problem with anisotropy, in which the reflection from a point depends on the positions of the transmit (Tx) and receive (Rx) arrays. The goal is to label the voxels of a field of view by a finite set of semantic classes and to group them into object instances. For the image formation of each Tx–Rx pair we apply a standard inverse-problem solver, and we feed the resulting per-pair reconstructions into a trained three-dimensional (3-D) U-Net that performs the fusion implicitly and the per-voxel classification explicitly. On a controlled, under-determined multistatic setup, we consider the following image formation methods: back-projection (BP) and the least absolute shrinkage and selection operator (LASSO) from a single deterministic snapshot, and incoherent BP and group-LASSO from multiple fading snapshots. For each imaging method we train a separate U-Net that fuses the six Tx–Rx pairs (its input channels) and assigns each voxel a probability vector over the classes. Taking the most probable class gives a labeled volume—the semantic reconstruction. Object instances and their oriented bounding boxes then follow by
geometric post-processing (clustering and principal-component analysis). Across a wide range of signal-to-noise ratio, the semantic reconstruction (scored against ground truth by
segmentation intersection-over-union) and the resulting 3-D detection degrade far more gracefully than the classical intensity reconstruction: the detection in particular stays reliable well into noise levels at which that reconstruction has dissolved. Because real scenes contain objects of classes the network was not trained on, we add an explicit unknown class trained by outlier exposure, which labels held-out novel objects as unknown instead of mislabeling them as a known class by reconstructed shape.
Read the full abstract ▾
Computer Vision
Mover360: Controllable Object Manipulation in 360{\deg} Panoramic Images
Haoyi Zhong, Fang-Lue Zhang, Andrew Chalmers et al. (+1 more)
We present Mover360, a controllable object manipulation framework for 360{\deg} images. Unlike perspective images, 360{\deg} images in equirectangular projection (ERP) exhibit horizontal wrap-around, latitude-dependent distortion, and global scene continuity, which makes object-level edits difficult for existing perspective editors to produce and for users to specify. To address this, Mover360 centers on object Translation (relocating a specified object within an existing panorama) while supporting reference-guided Insert and Remove as auxiliary tasks. Its interface unifies point-, bbox-, and mask-guided control by encoding each task into a fixed
prompt and a compact, ERP-aligned instruction map. In the default point mode, a single click relocates an object, allowing the model to infer a plausible size, support, and illumination using panoramic context and an auxiliary depth condition. Structurally, Mover360 is a lightweight adaptation of a pretrained
diffusion transformer. To generate paired supervision, we construct a UE5 data-generation pipeline with surface-aware object placement and randomized illumination, yielding large-scale paired data and a dual-domain
benchmark of synthetic and real panoramas with ground truth for all three tasks. Across both test domains and two evaluation protocols, Mover360 outperforms strong baselines for perspective editing, insertion, and inpainting in reconstruction fidelity, semantic consistency, and distributional quality. Code and our
benchmark dataset are available at https://zhonghaoyi.github.io/Mover360/.
Read the full abstract ▾
All authors Haoyi Zhong, Fang-Lue Zhang, Andrew Chalmers, Taehyun Rhee
Computer Vision
MLLM-Assisted Audio VOS: A 3rd Place Report for the MeViS-Audio Track, 8th LSVOS Challenge
Liangtao Shi, Jinxia Xie, Xiantao Hu et al. (+1 more)
In this technical report, we present a training-free framework for audio-guided
video object
segmentation, which integrates
Multimodal Large
Language Models (MLLMs) with SAM-based
segmentation models. We decompose the task into several stages and identify suitable foundation models for each stage. Without introducing additional model training or task-specific fine-tuning, our approach leverages the strong
multimodal reasoning capabilities of MLLMs to model text-visual correspondence and employs SAM-based models for accurate object mask generation. The proposed framework demonstrates the effectiveness of leveraging foundation models for audio-guided
video segmentation and achieves competitive performance in the MeViS-Audio Track of the 8th LSVOS Challenge.
Read the full abstract ▾
All authors Liangtao Shi, Jinxia Xie, Xiantao Hu, Ting Liu