Computer Vision
Closed-form Bayesian homography estimation from noisy point correspondences
Hanne Beuter and Sebastian Dorn
While homographies are fundamental to many computer
vision tasks, the majority of conventional estimation techniques provide only point estimates without directly quantifying uncertainty introduced by noisy observations. Uncertainty, though, propagates to subsequent processing steps such as camera calibration and 3D reconstruction and is particularly relevant in safety-critical and socially relevant fields including medical imaging, autonomous driving, and defense. We present a fast
Bayesian formulation for homography estimation from point correspondences that explicitly incorporates measurement uncertainty and prior knowledge while providing a posterior distribution over the homography parameters. A closed-form solution of the posterior mean of the homography is derived in homogeneous coordinates and supplemented by an iterative
Bayesian approach to handle non-linearities. Synthetic experiments demonstrate the applicability to projective transformations and show improved estimation accuracy over DLT under varying noise conditions. Image stitching experiments further demonstrate applicability to real image correspondences while additionally providing uncertainty information.
Read the full abstract ▾
Computer Vision
Deep Learning-based Intelligent Diagnosis of Congenital Uterine Anomalies in 3D Ultrasound
Yueyue Xu, Yuhao Huang, Jiaxiao Deng et al. (+11 more)
Objective: To develop an intelligent framework, termed CUA-Net, for the automated classification of congenital uterine anomalies (CUA) without requiring coronal plane reconstruction, and to evaluate its clinical applicability. Methods: CUA-Net was built on 3D ResNet-18, equipped with a dynamic data resampling strategy to mitigate the data imbalance issue and a hard sample mining technique to fully learn from the difficult cases by loss adjustment. We further proposed the self-supervised reconstruction to comprehensively explore the volumes and the online data augmentation to refine the wrong predictions and enhance the model's generalization. We compared the CUA-Net with different deep-learning methods and junior/senior sonographers in the testing set. The evaluation metrics included accuracy, precision, recall, F1-score, micro-AUC, and macro-AUC. Results: The proposed CUA-Net exhibited satisfactory performance in both internal and external test sets. In the internal cohort, the model achieved accuracy of 93.88%, precision of 87.01%, recall of 95.92%, F1-score of 88.09%, and micro-AUC of 0.9982 and macro-AUC of 0.9997. In the external set, it maintained good performance with accuracy of 91.52%, precision of 83.27%, recall of 88.63%, F1-score of 81.49%, micro-AUC of 0.9945 and macro-AUC of 0.9990. Our CUA-Net outperformed the junior sonographers across all performance indicators and achieved performance comparable to that of the senior sonographers across most metrics. Conclusion: The CUA-Net demonstrates favorable accuracy and generalizability in classifying common CUA categories, while showing preliminary potential for recognizing less prevalent anomalies. These capabilities may help optimize clinical workflows and support more standardized diagnosis.
Read the full abstract ▾
All authors Yueyue Xu, Yuhao Huang, Jiaxiao Deng, Yuanji Zhang, Haoming Zhang, Jiajia Qu, Shiying Zheng, Xiaomei Tang, Haining Chen, Chengcai Chen, Yiyi Wu, Xin Yang, Dong Ni, Hongyu Zheng
Computer Vision
Long-to-Short Video Evidence Reasoning for Grounded Question Answering
Kaiyan Chen, Junbin Xiao and Xun Yang
We present LOVER, a \underline{L}ong to sh\underline{O}rt \underline{V}ideo \underline{E}vidence \underline{R}einforced model for grounded question answering (GQA). LOVER highlights three innovations over existing reinforcement-learning (RL) based
video reasoning models: (1) Long-to-short
Video Evidence Curriculum Learning, which organizes RL training according to evidence duration and progressively adapts the model from long-range grounding to short-term
reasoning; (2) GQA Rewards, which underscore the benefit of IoP reward over IoU for evidence spotting rather than strict temporal span overlap; (3) Adaptive Timestamp Rendering, which adaptively renders timestamps onto
video frames using background-aware position and color selection to enhance temporal observability. The three designs are model-agnostic and reciprocal. They effectively improve QA, grounding, and grounded QA performance over different backbones. Notably, LOVER built on Time-R1 achieves new state-of-the-art (SOTA) results among open-source models on popular GQA
benchmarks: NExT-GQA and ReXTime. Comprehensive ablation studies further validate the effectiveness of our three innovative components.
Read the full abstract ▾
Computer Vision
Reconstructing Is Not Acting: Action-Centric Latent Dynamics Modeling
Dingjie Fu, Dianxing Shi, Yangyang Xu et al. (+1 more)
Latent action models (LAMs) learn action representations from unlabeled
videos by inferring latent actions from visual transitions and reconstructing future states. However, we identify a fundamental $\textbf{reconstruction-action mismatch}$: lower reconstruction error does not necessarily yield better latent dynamics or downstream performance. We attribute this mismatch to two underconstrained aspects of reconstruction-based latent dynamics modeling: (i) the inverse dynamics model (IDM) is not explicitly encouraged to distinguish action-related transitions from nuisance appearance, and (ii) the forward dynamics model (FDM) can underutilize the inferred latent action by exploiting predictive shortcuts from the current state. To address both limitations, we propose $\textbf{ACT-LAM}$, a lightweight action-centric framework that strengthens both action extraction and action utilization. Specifically, its Action Query IDM (AQ-IDM) employs learnable action queries and gated aggregation to selectively extract rich action-related transition cues without strong information bottlenecks. And its Action
Token FDM (AT-FDM) projects latent actions into action
tokens that progressively interact with evolving state representations, enabling continuous state-aware action conditioning. ACT-LAM further streamlines feature processing to concentrate model capacity on latent dynamics modeling. Extensive experiments on several robotic
datasets and the VP$^2$
benchmark demonstrate stronger latent action consistency, forward dynamics, and downstream visual planning performance with fewer trainable parameters and lower computational overhead. In particular, ACT-LAM surpasses the previous state of the art by $\textbf{7.6%}$ on the aggregated VP$^2$ success rate. Codes at $\href{https://github.com/DingjieFu/ACT-LAM}{url}$.
Read the full abstract ▾
All authors Dingjie Fu, Dianxing Shi, Yangyang Xu, Jun Yu
Computer Vision
Does Attention-Guided Masking Really Help Object Discovery in Object-Centric Learning?
Youliang Tao, Yanhua Han, Bin Zhao et al. (+3 more)
Object-Centric Learning (OCL) aims to decompose images into objects without human annotations. A major family of mainstream methods uses Slot
Attention to aggregate image features into object-level representations and then from them reconstructs masked image content, i.e., Random Masking (RM), to provide self-supervision. The recent method DIAS simply masks image patches at uniform randomness yet achieves competitive object discovery accuracy. Since
attention during aggregation already possesses object discovery ability, we explore using it to develop a better image patch masking strategy, i.e.,
Attention Guided Masking (AGM), thereby providing better self-supervision. Results on six recognized
datasets show that AGM does not always outperform RM. Under unconditional slot initialization, AGM substantially improves background
segmentation on
datasets with realistic textures (COCO and VOC); Regardless of conditional or unconditional slot initialization and across
datasets, foreground object discovery remains comparable or decreases. We suggest peer researchers in the OCL community that attempts to exploit internal
attention semantics to improve OCL with masked decoding are risky. Our source code, model checkpoints and evaluation logs is available on https://github.com/und-
entropy/Does-
Attention-Guided-Masking-Really-Help-Object-Discovery-in-Object-Centric-Learning-.
Read the full abstract ▾
All authors Youliang Tao, Yanhua Han, Bin Zhao, Juho Kannala, Joni Pajarinen, Rongzhen Zhao
Computer Vision
EECTracker: Swarm Motion Prior-Guided Feature Compensation for Airborne Optical UAV Swarm Tracking
Zhaochen Chu, Tao Song, Ren Jin et al. (+2 more)
Airborne optical tracking of uncrewed aerial vehicle (
UAV) swarms is challenging due to extremely small target scales, rapid viewpoint changes, and cluttered backgrounds, which can weaken target feature responses and lead to intermittent or temporarily missing detector responses. Existing multi-object tracking methods generally depend on reliable target-specific detector responses to maintain target states and identities across frames. When such responses become unreliable, target states cannot be reliably updated and cross-frame association cues become ambiguous, resulting in fragmented trajectories and identity switches. To address this problem, we propose EECTracker, a swarm-motion-prior-guided joint detection-and-tracking framework for airborne optical
UAV swarm tracking. EECTracker constructs a probabilistic swarm motion prior from reliable historical tracklets to capture the shared short-term image-plane motion tendency of the swarm and its uncertainty, providing spatial guidance for cross-frame feature compensation. Building on this prior, we introduce Energy–
Entropy Consistency Activation (EEC Activation) to evaluate motion-prior-conditioned feature consistency using feature residual energy and local residual
entropy. The resulting Local EEC score guides pixel-level feature compensation by enhancing motion-prior-consistent feature responses in potential target regions while suppressing inconsistent background responses. Experiments on AIRMOT and UAVSwarm show that EECTracker achieves superior overall tracking performance compared with state-of-the-art methods. Compared with the strongest competing method SCT-MOT, EECTracker improves MOTA/IDF1 by 3.89/1.79 percentage points on AIRMOT and by 2.81/1.74 percentage points on UAVSwarm, while maintaining online inference speed.
Read the full abstract ▾
All authors Zhaochen Chu, Tao Song, Ren Jin, Mingdong Jia, Defu Lin
Computer Vision
GRAVA: Grounded Reasoning-to-Action Representation and Learning for Autonomous Driving
Xiao Liu, Haoyu Li, Jianghao Leng et al. (+2 more)
Driving
vision-
language-action (VLA) models increasingly reason before acting, but their intermediate
reasoning is often weakly grounded in physical scene evidence and loosely connected to executable behavior. We present GRAVA, a framework built around Grounded
Reasoning-to-Action (GRA), which unifies grounding,
reasoning, and action generation in a single autoregressive stream. GRA links action-relevant
language references to 2D visual regions and ego-centric physical states, organizes object interactions and decisions in a trajectory-anchored typed graph, and serializes this structure into grounded
reasoning. A single VLM generates this
reasoning followed by a compact Executable Planner action that is deterministically decoded into a continuous trajectory. We further introduce an agentic GRA data construction pipeline that combines forward scene grounding with backward trajectory anchoring, and use it to build GR-NavSim with 2.2M grounded question-answer pairs and 70K GRA
reasoning traces. A progressive training strategy develops grounded cognition through pre-training, establishes the
reasoning-to-action interface through imitation, and improves driving behavior through reinforcement learning and exploration. Using about 60% of the available human driving demonstrations for action supervision, GRAVA-8B achieves state-of-the-art performance among purely autoregressive driving models on the full NAVSIM
benchmark. On an internal long-tail
benchmark, full GRA improves key-object compliance and Closed-loop Driving Score by 19.3% and 20.5% over action-only prediction, respectively. These results show the benefit of preserving action-relevant physical evidence from grounded
reasoning through executable action generation.
Read the full abstract ▾
All authors Xiao Liu, Haoyu Li, Jianghao Leng, Lin Wang, Chao Sun
Computer Vision
PACE: Progressive Angular-to-Norm Contrastive Embedding
Yanping Li, Wei Zhou, Yawen Liu et al. (+7 more)
Multimodal embedding models encode heterogeneous inputs into a shared
embedding space, enabling efficient similarity computation across modalities and tasks. Most existing methods optimize cosine-based contrastive objectives, which promote stable training but restrict semantic compatibility to angular geometry, precluding
embedding norms from serving as an additional semantic signal. However, directly optimizing the more expressive dot-product similarity, which leverages both angular and norm information, underperforms cosine-based training and exhibits unstable training dynamics. We attribute this discrepancy to premature optimization-space expansion, manifested as angular–norm
entanglement and directional anisotropy in the representation space and further compounded by full-parameter fine-tuning. In this paper, we propose PACE, a two-stage framework that progressively expands both the representation and trainable parameter spaces. Stage I combines cosine-based objective with low-rank adaptation to establish a reliable angular geometry within constrained optimization spaces. Stage II switches to dot-product similarity and full-parameter fine-tuning, enabling
embedding directions and norms to jointly encode semantic information. We further introduce Focal
Embedding Loss, a confidence-adaptive objective that downweights queries with high positive
retrieval confidence while emphasizing ambiguous queries with competitive negatives. Experiments across multiple backbone scales and diverse
multimodal embedding tasks consistently validate the effectiveness of PACE.
Read the full abstract ▾
All authors Yanping Li, Wei Zhou, Yawen Liu, Yibo Wang, Ke Zhu, Guangda Huzhang, Qing-Guo Chen, Zhao Xu, Jun Zhang, Wei Wei
Computer Vision
Weakly Supervised Spatial Grounding for Discriminative Attention-Based Ultrasound-Histopathology Alignment in Prostate Cancer Grading
Obed Korshie Dzikunu, Emma Willis, Mohammad Mahdi Abootorabi et al. (+7 more)
Unpaired cross-modal
distillation transfers grade structure from histopathology into a micro-ultrasound (micro-US) encoder by aligning a pooled needle-region
embedding to a frozen histopathology teacher under grade-group correspondence alone. A single objective is thereby required to serve two distinct functions: rendering patch features discriminative of tissue state, and selecting which patches enter the pooled representation. We decouple them. Weak spatial supervision derived from percentage involvement, recorded routinely at biopsy, constrains the predicted proportion of malignant tissue within each core, acting on the encoder features independently of the
alignment objective. The
alignment loss then operates on features that differ across a core, and
attention concentrates on a subset of patches rather than remaining near-uniform. On 7,166 biopsy cores from 811 patients across seven centers under patient-level 5-fold cross-validation, the method reaches 67.1 macro AUC and 68.5 csPCa AUC, against 61.2 and 52.8 for the existing unpaired
alignment method and 63.1 and 62.6 for the strongest unimodal baselines. Ablation against existing
attention regularizers designed to prevent
attention-uniformity collapse shows that such regularizers do not substitute for label-derived supervision: they constrain the
attention distribution, whereas the signal required acts on the features that
attention reads.
Read the full abstract ▾
All authors Obed Korshie Dzikunu, Emma Willis, Mohammad Mahdi Abootorabi, Mohamed Harmanani, Zhuoxin Guo, Ferdinand Luger, Adam Kinnaird, Brian Wodlinger, Parvin Mousavi, Purang Abolmaesumi
Computer Vision
Automated Perceptually-Motivated Assessment of Photographic Consistency in Paired Clinical Photographs: Pipeline Development and Internal Evaluation
Derrick Lin, Samantha Rabinovich, Joclin Rabinovich et al. (+5 more)
Purpose: Paired pre- and post-operative photographs are the standard unit of evidence for plastic
surgical outcomes, yet no objective metric verifies whether two images of the same patient were captured under conditions consistent for comparison. Approach: We developed a perceptually motivated pipeline that analyzes pre/post pairs across thirteen calibrated sub-metrics, partitioned by unsupervised correlation-structure analysis into five data-driven clusters (photometric, texture / sharpness, pose, illumination direction, and pitch), averaged within each cluster and combined across clusters by a weighted sum into a single consistency score. Each sub-metric is calibrated so that its median difference across published within-patient pairs scores 0.5, which is a reference point and carries no pass/fail meaning. The pipeline was calibrated on 134 matched within-patient published pre/post pairs and evaluated against identical-image pairs, synthetic-perturbation pairs, and 134 mismatched cross-publication pairs. Results: The master consistency score S separated matched from mismatched pairs (sensitivity index d' = 2.15, 95% confidence interval (CI) [1.83, 2.55]; area under the receiver operating characteristic curve AUC = 0.928, 95% CI [0.896, 0.959]), closely matching Gaussian-equal-variance predictions. The three head-pose angles did not fall in one cluster: yaw and roll grouped together while pitch separated. Identical pairs scored at ceiling (S = 0.99) and the master score fell monotonically with perturbation magnitude on all five perturbation axes. Conclusions: The score quantifies photographic comparability, not aesthetic or
surgical quality, and provides a freely available web tool for
auditing the photographic comparability of pre/post pairs, pending validation against expert judgment.
Read the full abstract ▾
All authors Derrick Lin, Samantha Rabinovich, Joclin Rabinovich, Kassra Garoosi, Sumun Khetpal, Evan Delanoy, Neel Bhardwaj, Jason Roostaeian
Computer Vision
SparseTalk - Sparsifying 3D Gaussian Language Fields for Efficient 3D Visual Question Answering
Davit Soselia, Joseph JaJa and Amitabh Varshney
3D Gaussian
language fields provide an explicit, spatially grounded representation for 3D visual question answering (VQA), but their dense semantic features can require tens of thousands of
embeddings per scene, resulting in substantial storage,
memory, and inference costs. We investigate how much of this representation is actually necessary for downstream
reasoning. Starting from a full
embedding representation, we systematically sparsify its semantic
embeddings, including the previously underexplored regime below a single image-equivalent block down to 8 visual
tokens. We compare random,
geometric, semantic, and joint spatial-semantic selection strategies and introduce an object-based sparsification method that distributes the
token budget across detected object instances while retaining background context. Experiments on ScanQA and MV-ScanQA reveal substantial redundancy in dense Gaussian
language fields. Strong VQA performance is retained with only a few hundred semantic
embeddings, corresponding to less than 1% of the original representation. Object-based selection performs well relative to others, with only modest observed changes down to 256
tokens. At this budget, SparseTalk retains 0.80% of SplatTalk's 32,076-
token inference input and 0.332% of the mean 77,207-Gaussian dense field, increasing inference throughput while reducing decoded-feature
memory 125-fold.
Read the full abstract ▾
Computer Vision
AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference
Yuyao Sun, Tao Deng, Shuang Li et al. (+1 more)
Multimodal large
language models (MLLMs) require substantial computation to process numerous visual
tokens across all
transformer layers. Most methods for efficient MLLM inference exploit horizontal redundancy by compressing visual
tokens. Beyond
token reduction, recent studies exploit vertical redundancy through early exit or fixed-layer skipping. However, we find that the extent and distribution of this redundancy vary across inputs and differ between self-
attention and MLP modules. Motivated by these observations, we propose AdaVSkip, which equips each layer with two lightweight routers that independently determine whether visual
tokens pass through by or skip the self-
attention and MLP modules. These decisions collectively define an input-specific visual-computation path, but their discrete and non-differentiable nature makes learning effective paths challenging. To address this challenge, we develop a progressive two-stage training framework that updates only the routers while keeping the backbone frozen. Stage I establishes an initial routing policy through supervised training with input-specific targets derived from module-wise necessity scores. To further align the routing policy with task performance, Stage II uses reinforcement learning to optimize routing decisions with direct feedback from generated answers. It combines an answer correctness reward with a skip-consistency reward that discourages excessive retention of visual-
token computation. Across three MLLM backbones, AdaVSkip maintains strong task performance with substantially less computation. On LLaVA-NeXT-7B, AdaVSkip reduces FLOPs by 53.2% while preserving the original model's average performance. Combining it with visual
token compression increases this reduction to 91.2%, while retaining 97.2% of the original performance on average.
Read the full abstract ▾
All authors Yuyao Sun, Tao Deng, Shuang Li, Deqing Wang
Computer Vision · Cross-listed from cs.SE
woma: a real-time foundation model and its fine-tuned models for endoscopy
Thang Tran and Lan Dang
woma is a real-time foundation model for gastrointestinal endoscopy: a network trained without labels on about a million endoscopy frames, from which task models are fine-tuned. We contribute a systematic design for production. Requirements and pass marks were fixed before any run, eight candidates screened under pre-registered rules, self-supervised training taken to a stopping rule, then fine-tuning and deployment optimisation, all on one self-contained library, numbat. We also contribute woma itself with two fine-tuned models, every outcome reported met or missed. Our colonoscopy model finds and outlines polyps, names which colon segment is in view, suggests polyp type and grades bowel preparation. Our gastroscopy model names a station out of 22 protocol sites, flags and outlines lesions, and names one of seven findings. Every number was read on data never seen in training, and shipped weights were chosen on that record. In colonoscopy, 96% of polyps in a six-hospital PolypGen set are found at precision >=0.85, and 19 of 19 polyps across fifteen full REAL-Colon
videos at 1.6 false alarms per procedure. In gastroscopy, landmark region is named correctly on 92% of frames from unseen patients, and 37 of 39 held-out neoplasia frames are flagged at specificity 0.91. On one workstation GPU every task runs over 1080p
video at about 100 frames per second, faster than PyTorch, ONNX Runtime and TensorRT in all four precision regimes tested. TensorRT comes closest: one pass of our foundation model takes it 3 to 27% longer than ours, and we deliver 6 to 31% more frames per second from frame to results. A second build links no vendor library at all – our own kernels over Vulkan – so a site deploys two files and needs no toolkit, no cuDNN and no framework; in f32 it beats the CUDA build on the same card.
Read the full abstract ▾
Computer Vision
DNF-SR: Dual-Input and Negative-Aware Feature Fine-Tuning for Real-World Image Super-Resolution
Shuhao Han, Wenjie Liao, Hayden Vance et al. (+4 more)
Benefiting from the powerful generative priors of
diffusion models,
diffusion-based real-world image super-resolution (Real-ISR) methods have demonstrated impressive performance.To achieve efficient Real-ISR, several recent works have designed one-step
diffusion-based models.Howerver, unmediatedly feeding LR into a
diffusion model creates a distributional gap with the model's original input.A straightforward approach to reduce the distribution gap is to introduce noise to the LR latents. However, directly adding noise inevitably corrupts the content of the LR images.In this study, we propose DNF-SR, a Dual-input and Negative-aware Feature fine-tuning method for Real-ISR.Specifically, we use a dual-input strategy that concatenates the original LR image with the noisy LR input and feeds them into a
diffusion-based image editing model, ensuring both high-fidelity one-step super-resolution and improved perceptual and content consistency.Additionally, the noise present in the noisy LR input introduces randomness and diversity into the outputs. We exploit this property and propose a post-training optimization method, Negative-aware Feature Fine-Tuning (NF2T), which guides the model toward producing higher-quality results.NF^2T classifies multiple outputs into positive and negative subsets and then defines implicit policy improvement directions in both the image and feature spaces, thereby further enhancing the stability of the optimization.Extensive experiments show that DNF-SR outperforms other methods.Code will be released.
Read the full abstract ▾
All authors Shuhao Han, Wenjie Liao, Hayden Vance, Hang Dong, Rui Zhang, Chun-Le Guo, Chongyi Li
Computer Vision
ChatGPT Images 2.5 in the Wild: A Launch-Period Dataset and Detector Evaluation
Dennis Ng, Xingyu Shen, Ankit Raj et al. (+6 more)
An image tool can change its underlying generator while retaining its public name, making version attribution from online posts ambiguous. We study this problem after the ChatGPT Images 2.5 launch. Our frozen collection contains 3,478 images from 2,440 posts across 8 sources. Recorded posting times fall within the first 51.1 hours after the announcement. It records three attribution tiers and retains standalone images after image-form filtering and targeted review. Caption claims and host records provide admission evidence, not independently verified generator identity. The observed content profile depends on the source mixture: NightCafe supplies 39.0% of images but 77.0% of CLIP-assigned fantasy scenes. We then evaluate six frozen detectors at thresholds calibrated to a 5% flag rate on reference photographs. Collection flag rates range from 3.7 to 56.4%, falling 42-81 percentage points below GenImage recall. Held-out artwork false-positive rates range from 1.5 to 96.5%, so a higher collection flag rate does not by itself establish better detection. An exploratory X-only comparison with our April collection finds a higher September flag rate for Effort, and a suggestive difference for DoU, under fixed-threshold post-clustered bootstrap intervals. Attribution, content and processing differences prevent a causal interpretation of these contrasts. The collection supports analysis of reported model use during a product transition, with source and attribution evidence retained for interpretation. The collection is released at https://scam.ai/research.
Read the full abstract ▾
All authors Dennis Ng, Xingyu Shen, Ankit Raj, Kidus Zewde, Tommy Duong, Yuchen Zhou, Yuxin Zhang, Neo Tiangratanakul, Simiao Ren
Computer Vision
LG-VLN: A Zero-Shot Vision-and-Language Navigation Framework with LangGraph State Orchestration
Jianhe Zhao, Yanhua Qiu, Zhiyu Zhang et al. (+2 more)
Continuous-environment
vision-and-
language navigation (VLN-CE) requires interpreting natural-
language instructions in unseen 3D environments and executing continuous low-level actions. Existing methods often depend on
LiDAR, panoramic cameras, or extra sensors; separate
geometric-mapping and semantic-
navigation visual representations can cause long-trajectory spatial-semantic inconsistencies. We propose LG-VLN, a
monocular zero-shot framework with shared visual features and LangGraph-based state orchestration. An online feed-forward 3D reconstruction network predicts depth, camera poses, and dense point clouds for
agent-pose estimation and global map fusion. Geometry and
navigation share dense CleanDIFT features: semantic consistency rejects incorrect inter-frame correspondences, while target-instance constraints define visual references whose similarity combines with local BLIP-2 image-text relevance to form a semantic value map. LangGraph represents instruction parsing,
geometric perception, semantic value updates, path planning, action execution, and failure recovery as a directed state graph with conditional transitions, persistent state, and modular recovery mechanisms. On a fixed 550-episode subset of the R2R-CE val-unseen split, LG-VLN achieves 21.3% success and 12.1% success weighted by path length. Ablations show shared semantic features improve
navigation, further boosted by combining visual similarity and image-text relevance. Results establish shared visual representations and explicit state orchestration as effective for zero-shot VLN-CE using
monocular RGB alone. Code will be publicly released for reproducibility.
Read the full abstract ▾
All authors Jianhe Zhao, Yanhua Qiu, Zhiyu Zhang, Zibo Zhao, Jinhua Xie
Computer Vision
Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras
Hanzhang Tu, Zhanfeng Liao, Wei Min et al. (+2 more)
Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either rely on computationally expensive optimization or require calibrated cameras and low-resolution inputs, making real-time high-resolution deployment impractical. In this work, we present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization from sparse, unposed RGB streams. Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance in a single forward pass. To achieve this, we start by designing a lightweight sparsity-aware multi-view
transformer backbone that tokenizes foreground human regions while preserving global context through a shared scene
token. We then employ a fully
transformer-based Gaussian decoder to mitigate convolution-induced over-smoothing while keeping decoding sparse and efficient. In addition, we introduce a hybrid feature pyramid that injects multi-scale appearance cues into geometry prediction. We further introduce a lightweight differentiable Levenberg-Marquardt camera refinement layer to enhance multi-view consistency and
geometric alignment. Moreover, to stabilize learning under sparse, unposed inputs, we transfer multi-view geometry priors from a large visual-geometry foundation model via teacher-student
distillation. Finally, the predicted Gaussian maps are streamed with
video codecs to remote devices for interactive free-viewpoint rendering. Extensive experiments show that Tele360 achieves state-of-the-art visual quality on studio
benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU. Additional captured sequences illustrate its performance across varied subjects, clothing, and motions under our multi-camera setup.
Read the full abstract ▾
All authors Hanzhang Tu, Zhanfeng Liao, Wei Min, Jiajun Zhang, Yebin Liu
Computer Vision
TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals
Zihan Xue, Po-Yi Lu, Serhii Honcharenko et al. (+5 more)
In-context learning (ICL) enables models to infer tasks from demonstrations, but existing
benchmarks generally lack matched text and image versions needed to compare ICL performance across modalities. We introduce TwinICL, a procedurally generated
benchmark providing such pairs for controlled comparison. Across six open-weight models and 38 tasks,
multimodal ICL consistently underperforms text-only ICL, with gaps varying by task family. To test whether this gap can be recovered, we target visual access, task framing, and
reasoning through three interventions. Their combination recovers strong
multimodal ICL performance on a diagnostic subset, despite limited or inconsistent individual effects. To distinguish difficulties in executing tasks from those in inferring them, we evaluate models with explicit task instructions, revealing a modality gap even when the task is known. We then examine how adding demonstration inputs and outputs reshapes this gap, highlighting demonstrations' dual role as additional context to process and evidence about the task. The
dataset is available at https://github.com/lab-flair/TwinICL.
Read the full abstract ▾
All authors Zihan Xue, Po-Yi Lu, Serhii Honcharenko, Zih-Ching Chen, Hsuan-Tien Lin, Nanyun Peng, I-Hung Hsu, Kuan-Hao Huang
Computer Vision
G-ray: Ray-Level Relative Geometric Position Encoding in Multi-View Vision Transformers under Camera Heterogeneity
Shuo Zhang, Xin Su, Wei Wang et al. (+8 more)
We study relative position encoding for multi-view
vision Transformers under camera heterogeneity, including varying fields of view (FoVs) or projection models. Existing rotary relative position encodings commonly use image-plane positional coordinates, producing projection-dependent relative phases and inconsistent
geometric cues for cross-projection
attention. We introduce G-ray, a ray-level relative position encoding whose rotary phases are parameterized by camera-local ray angles. The same camera-local ray pair induces the same relative phase across projections, providing projection-invariant positional consistency. G-ray can be used directly or integrated with existing encodings, retaining complementary
geometric cues without additional learned parameters. We validate G-ray in three host encodings, RoPE, GTA, and RayRoPE, across 3D reconstruction and novel-view synthesis (NVS). Across three heterogeneous 3D reconstruction
benchmarks at 50 views, G-ray leads all six averaged metrics and reduces mean pointmap relative error by 45.8% over MapAnything, with calibration supplied to both. Trained exclusively on homogeneous pinhole images, the 3D reconstruction model handles mixed pinhole and non-pinhole inputs without retraining and remains competitive on homogeneous pinhole 3D reconstruction protocols. For NVS, GTA and RayRoPE improve with G-ray under joint viewpoint and FoV variation. The project's webpage is available at https://g-ray-project.github.io/.
Read the full abstract ▾
All authors Shuo Zhang, Xin Su, Wei Wang, Jun Liu, Xinrui Zeng, Yongsen Chen, Chenjie Wang, Guibo Zhu, Jinqiao Wang, Bin Luo, Liangpei Zhang
Computer Vision
PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
DeepCybo Team, Yu Bin, Haipeng Cao et al. (+51 more)
We present PhysBrain 1.5, a unified model for understanding physical environments, generating actions, and predicting future states. Motivated by the physical loop of observation, interaction, and environmental change, we bring these capabilities into a common learning framework. Starting from a general
vision–
language model, we encode
language responses, end-effector motion, and dense visual targets as discrete sequences and jointly optimize them with autoregressive next-
token prediction. Pre-training draws its
embodied supervision entirely from human interaction
videos, using task-centered episodes to pair semantic and spatial context with recovered motion and subsequent observations. We then adapt the model through supervised fine-tuning on a mixture of human demonstrations,
robot trajectories, and simulated experience. Across 28
embodied understanding
benchmarks, our 8B model achieves an average score of 72.5, setting a new open-source state of the art and performing on par with leading proprietary models such as GPT-6-Astra and Gemini 3.6 Flash. It achieves the best open-source results on 14
benchmarks while retaining general
multimodal capabilities. Beyond these understanding evaluations, qualitative examples show the model's ability to produce end-effector trajectories and predict future scenes through spatially aligned RGB, depth, and
robot-mask outputs.
Read the full abstract ▾
All authors DeepCybo Team, Yu Bin, Haipeng Cao, Zheng Chang, Kai Chen, Youning Chen, Kailin Deng, Yichao Du, Xiaotong Fu, Haoyang Ge, Yunlong Guo, Chenliu Hao, Jiyan He, Xuguo He, Yakun Hou, Kai Hu, Cong Huang, Tuopusen Huang, Yu Huang, Hong Li, Peize Li, Shijie Lian, Xiaopeng Lin, Yun Lin, Haibao Liu, Haochen Liu, Qiuzhi Liu, Shengcai Liu, Zhiqiang Liu, Tao Luo, Peng Ren, Shuo Ren, Chaoyi Ruan, Zhaolong Shen, Yukun Shi, Qiyuan Su, Yuxuan Tian, Yining Wang, Changti Wu, Hao Wu, Xueyin Xu, Ruoqi Yang, Zhaoyang Yang, Hang Yuan, Zhaoyang Zeng, Hanwen Zhang, Ruimeng Zhang, Yao Zhang, Yibo Zhang, Yuxiang Zhang, Zhirui Zhang, Ziyi Zhang, Zubin Zheng, Zishen Zhuang
Computer Vision
MoVT: Video-Augmented Motion Tokenizer for Text-to-Motion Generation
Beibei Jing, Tianle Guo, Youjia Zhang et al. (+5 more)
Text-driven 3D human motion generation models face significant challenges in responding to diverse and unconstrained textual
prompts, primarily due to the limited availability of 3D motion training data. To address this, we introduce MoVT, a novel framework that effectively leverages the extensive range of human action
videos to enhance text-to-motion generation. At the core of our approach is the cross-modal augmented motion tokenizer, which projects discrete 3D motion
tokens into the 2D domain. This projection allows us to enrich the motion codebook with complex, real-world motion patterns derived from
videos. The enriched discrete
tokens are then mapped back to the 3D domain, resulting in aligned 3D and 2D codebooks with an enhanced capacity to represent intricate motions. These enhanced codebooks are integrated into a generative masked
transformer, which predicts masked motion
token indices in a modality-agnostic manner. This enables the use of text-index pairs, generated from the 2D codebook and annotated motion
videos, to further enhance the generator. Extensive empirical evaluations show that MoVT performs favorably against prior state-of-the-art methods across multiple key metrics.
Read the full abstract ▾
All authors Beibei Jing, Tianle Guo, Youjia Zhang, Zikai Song, Yawei Luo, Junqing Yu, Tao Guan, Wei Yang
Computer Vision
ThreshGuide: Class-Aware Labeled-Guided Thresholding for Semi-Supervised 3D Abdominal Multi-Organ Segmentation
Hongyu Liu, Yinlong Wang, Lusha Li et al. (+1 more)
Pseudo-labeling is a strong paradigm for semi-supervised medical image
segmentation, yet its effectiveness is highly sensitive to confidence thresholding. In abdominal multi-organ
segmentation, a fixed global threshold is particularly suboptimal because organ classes differ substantially in size, appearance, and learning difficulty. In this work, we propose ThreshGuide, a class-aware threshold adaptation framework that uses labeled data to guide pseudo-label selection on unlabeled data. Built upon a standard teacher-student architecture, the teacher model evaluates labeled samples during training to estimate class-aware threshold targets by maximizing an error-aware Fe̱ta} criterion that balances precision and coverage. These targets are then smoothed with an exponential moving average (EMA) and used to filter unlabeled voxels in a class-dependent manner. Experiments on
FLARE2022 and AMOS2022 show that ThreshGuide performs competitively overall, yielding clear improvements specifically on hard-to-learn organs.
Read the full abstract ▾
All authors Hongyu Liu, Yinlong Wang, Lusha Li, Hui Meng
Computer Vision
What Makes a 3D Scene Editable? A Factorized Benchmark of Fidelity, Locality, Consistency, and Preservation
Sariah Patro, Arjun Mehra and Nikhil Bhatia
Neural 3D scene editing is often evaluated by semantic
alignment alone, although a convincing result may alter unrelated content or become inconsistent across views. We introduce EditBench3D, a representation-agnostic
benchmark that treats editing as controlled information replacement. It evaluates four complementary properties: instruction fidelity, spatial locality, cross-view consistency, and preservation of non-target content. The protocol combines visibility-aware 3D target supports, paired descriptions, held-out cameras, and five edit families covering appearance, material, geometry, and object-level changes. We evaluate eight representative NeRF, 3D Gaussian Splatting, hybrid, and proxy-based editors on 240 scene-edit pairs. The study shows that semantic fidelity is only weakly associated with the other editing properties, and that no single method is optimal across all dimensions. Explicit Gaussian editors offer a strong overall balance, whereas direct proxy manipulation provides the most conservative edits at the cost of open-ended fidelity. These findings support reporting editability as a multi-objective profile rather than a single semantic score.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.GR
MedVA: An End-to-End Neuro-Symbolic Agentic System for Medical Volume Visualization
Haill An, Suhyeon Kim, Minjun Kang et al. (+4 more)
Medical volume visualization requires selecting regions of interest (ROIs) and carefully controlling their relative visual emphasis according to a given clinical intent. Implementing these decisions in conventional workflows demands substantial clinical and visualization expertise and often involves trial-and-error optimization. Recent agentic systems have introduced natural-
language interaction and autonomous visualization operations but largely rely on MLLM-based inference throughout the workflow. Although MLLMs encode broad medical knowledge and provide strong
reasoning capabilities, such inference may be suboptimal for medical volume visualization, potentially leading to clinically incomplete interpretations of user requests and unreliable ROI identification and visualization optimization. In this work, we present MedVA, an end-to-end neuro-symbolic agentic system for medical volume visualization that addresses these limitations through three complementary
agents. The neuro-symbolic intent formulation
agent refines MLLM-based interpretations of natural-
language requests through symbolic
reasoning over established clinical knowledge, which provides more complete, clinically grounded ROI specifications than MLLM-only
reasoning. The multi-model ROI identification
agent directly identifies semantically specified ROIs in the original volume by leveraging complementary large-scale pretrained medical
segmentation models. The objective-driven visualization optimization
agent explicitly evaluates ROI visibility and occlusion in the original volume using a volume-based visibility objective. Extensive
agent-level and system-level evaluations across diverse medical
datasets and interaction scenarios support the effectiveness of the individual
agents. A formative user study further indicates high usability and practical value among users with different levels of expertise.
Read the full abstract ▾
All authors Haill An, Suhyeon Kim, Minjun Kang, Eunwoo Lee, Bin Sheng, Lei Bi, Younhyun Jung
Computer Vision · Cross-listed from physics.optics
Proximal-Only Transmission Matrix Recovery of an Arbitrarily Deformed Graded-Index Multimode Fiber
Cole Reynolds
The multimode fiber is among the thinnest imaging conduits available, carrying hundreds to thousands of spatial modes through a cross-section comparable to a human hair, but its endoscopic capabilities are currently limited by the sensitivity of the transmission
matrix to the fiber's deformed state. Proximal-only recovery of the fiber's transmission
matrix is an appealing approach for enabling general use multimode fiber endoscopy, and within the last decade, machine learning techniques have been applied to both single-ended and double-ended transmission
matrix recovery tasks. We present a new approach to this interdisciplinary problem and show that neural networks can generalize to recover transmission
matrices of an arbitrarily deformed graded-index multimode fiber from proximal measurements alone.
Read the full abstract ▾
Computer Vision
RAIN: Region-Aware Inversion Network for Semantic Watermark Extraction
Zilai Li
Semantic watermarks for
diffusion models embed ownership information into the generative process while preserving perceptual quality, but Gaussian-Shading extraction conventionally requires multi-step
diffusion inversion to recover the initial noise. Recent one-step methods show that this cost can be reduced substantially. We study this problem through extended flow matching and conditional regression. The key observation is that, near the high-SNR image endpoint, recovering a useful noise statistic given by the first-step output of the extended flow matching in the high-SNR regime is much simpler than reconstructing the full inverse trajectory, and Gaussian Shading only requires the recovered latent to remain in the correct watermark decision region. Based on this observation, we propose a lightweight,
prompt-free extractor that decomposes endpoint recovery into an image-like anchor and a noise-oriented residual, which increases the capability of the model to utilize GPU parallel computation. The resulting method avoids iterative inversion and repeated evaluation of a
diffusion-scale U-Net, providing an efficient one-step extraction pipeline with a concise theoretical interpretation. The computational cost of extracting noise is lower than that of both OSI and FARI. The github repo is there: https://github.com/TheLovesOfLadyPurple/RAIN-lightweight-NN-for-one-step-semantic-watermark-extraction
Read the full abstract ▾