Computer Vision
Impact of Patient Orientation in Single- and Multi-View Camera Environments for AI-based Rehabilitation Monitoring
Miriama Jánošová, Andreas Lang, Petra Budikova et al. (+1 more)
Automated quality assessment of rehabilitation exercises relies heavily on accurate human pose estimation from
video data. Although numerous RGB-based pose estimation methods have been proposed, the impact of camera placement on detecting clinically relevant movement errors remains insufficiently explored. To address this gap, we introduce REHAB26-ViewAngles, a
dataset comprising correct and incorrect rehabilitation exercise executions captured from a wide range of camera angles. Furthermore, we propose a novel separability metric to quantify an
algorithm's ability to distinguish between valid and faulty exercise repetitions. Using these tools, we analyze how various RGB-based pose-estimation strategies are suitable for exercise quality assessment under varying camera placements. In particular, we analyze single-camera 2D and 3D pose estimation and four multi-camera strategies: a combination of two orthogonal 2D views, 3D triangulation, weighted 3D fusion, and an AI-based pose-estimation
transformer model specifically trained from two synchronized cameras. Our findings reveal that an optimally placed 2D camera can improve the separability by 16.9% over the commonly used 0{\deg} frontal view and frequently outperforms single-camera 3D estimation, while combining two views can further improve accuracy by up to 13.1%. These results offer practical guidance for deploying rehabilitation monitoring in both home and clinical settings.
Read the full abstract ▾
All authors Miriama Jánošová, Andreas Lang, Petra Budikova, Jan Sedmidubsky
Computer Vision
Superquadric Primitive Decomposition of 3D point clouds via Geometric-Aware Inlier Refinement
Alessandro Rinaldi, Edoardo Tedesco, Andrea Ferraris et al. (+5 more)
The decomposition of 3D point clouds into interpretable
geometric primitives remains a longstanding challenge in Computer
Vision and Computer Graphics. Among the available representations, superquadrics offer a compact and expressive model capable of capturing a wide range of shapes. However, their estimation is inherently challenging, as it requires solving a non-linear optimization problem and is particularly sensitive to noise, outliers, and overlapping structures. While robust estimation methods such as RANSAC and its variants achieve strong performance, they rely primarily on spatial proximity and residual-based criteria, often leading to incorrect inlier assignments across adjacent or complex arrangements of primitives. In this work, we introduce a
geometric-aware framework for primitive decomposition that explicitly incorporates local surface properties into the fitting process. Specifically, we propose an inlier refinement step formulated as an energy minimization problem and solved via graph-cut optimization. Our formulation integrates
geometric priors, such as normal consistency, enabling more reliable inlier selection beyond purely residual-based criteria. The approach naturally applies to both single-model estimation and multi-model decomposition. By leveraging
geometric information beyond point-wise residuals, our method reduces erroneous inlier propagation and stabilizes parameter estimation. Experiments on synthetic and real
datasets show consistent improvements in
geometric accuracy, robustness to noise and outliers, and convergence efficiency compared to state-of-the-art RANSAC-based methods.
Read the full abstract ▾
All authors Alessandro Rinaldi, Edoardo Tedesco, Andrea Ferraris, Filippo Leveni, Daniele Baieri, Filippo Maggioli, Simone Melzi, Luca Magri
Computer Vision
Hard Vision, Easy Vision: What GPT-6 Astra Reveals Across Computer Vision
Hanoona Rasheed, Mohammed Irfan Kurpath, Bin Ren et al. (+3 more)
Frontier general-purpose systems are rapidly expanding beyond visual understanding into capabilities traditionally handled by dedicated computer-
vision models. As these capabilities expand, a central question for the computer-
vision community is how far this reach extends, and what remains hard. We evaluate GPT-6 Astra alongside five frontier general-purpose AI systems across 34 capabilities and 55
benchmarks spanning nine areas of computer
vision. We compare their performance with dedicated models and humans where suitable references are available. Astra demonstrates broad visual capability, with substantial gains over other frontier systems in visual and spatial
reasoning and several forms of structured prediction. Across the state-of-the-art systems, a consistent pattern emerges. Capabilities involving semantic interpretation,
reasoning, and object-centric prediction increasingly approach or reach available reference levels. In contrast, larger gaps remain when tasks require metric
geometric accuracy, faithful reconstruction, temporally consistent dense prediction, or specialized fine-grained visual knowledge. Additional
reasoning and specialist tools close selected gaps, but their benefits vary across capabilities. These results map a changing landscape of computer
vision in which increasingly sophisticated visual tasks are accessible through a general-purpose interface, while precise and fidelity-sensitive perception remains an important frontier.
Read the full abstract ▾
All authors Hanoona Rasheed, Mohammed Irfan Kurpath, Bin Ren, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan
Computer Vision
Lagrangian–Hamiltonian Flows for Video Prediction and Image Generation: A Symplectic Perspective
Jiawei Hu
We introduce LHFM, a
geometric framework for learning image dynamics. Drawing on structures central to classical mechanics, symplectic geometry, and
geometric quantization, LHFM represents each image as an exact Lagrangian graph and models its
evolution through image-dependent
Hamiltonian flows, which yield a transport–source parameterization of image velocities. Our primary application is deterministic
video prediction: LHFM-V is a recurrent model that advances frames by integrating predicted transport and source fields, and achieves the lowest reported FLOP count among the compared recurrent models with similar prediction accuracy. The image variant, LHFM-I, shows that the same construction is compatible with flow matching: in a matched experiment, it attains a lower FID than the flow-matching baseline.
Read the full abstract ▾
Computer Vision
Mind the RefGAP: Correcting Reference Attention in Diffusion-Based Visual Editing
Yanan Wang, Shengcai Liao, Guangyi Liu et al. (+1 more)
Reference-guided
diffusion editors struggle to faithfully reproduce user-provided references. We identify a potential bottleneck in
diffusion editors: many methods provide limited reference-
attention allocation. For example, in LoomVideo, edit-region queries assign less than 1% of their
attention mass to the reference. We introduce RefGAP, a training-free correction that determines logit-offset magnitudes online at each layer from the reference-
attention mass measured during the forward pass. Positive offsets to reference logits strengthen reference usage by edit-region queries, while negative offsets for keep-region queries limit reference-induced changes outside the edit. Two global coefficients control the correction; they are selected once on validation data from four development
diffusion editors and held fixed. Across seven
diffusion-based image/
video editors, RefGAP improves identity fidelity in head swapping and face swapping. RefGAP achieves a fidelity-preservation trade-off comparable to separately tuned constant edit-side biases, without per-approach strength sweeps. Additional experiments on virtual try-on and background replacement evaluate transfer beyond identity editing.
Read the full abstract ▾
All authors Yanan Wang, Shengcai Liao, Guangyi Liu, Xiaodan Liang
Computer Vision
DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time
Ziqi Ma, Hongqiao Chen and Georgia Gkioxari
Video generation must account for two sources of motion, one induced by the observer's camera path and the other caused by scene dynamics. An ideal camera-controlled
video model should account for both motions: let users move the camera while evolving the scene dynamics. While current models handle camera-induced motion well in static settings, they struggle for dynamic scenes: objects are static, move incorrectly, or degrade in generation quality. We introduce DynaTokens, a lightweight set of learnable scene-specific
tokens that teach dynamics to an existing camera-controlled world model. Our method is motivated by a simple asymmetry between the two sources of motion: whereas camera motion affects the generated view globally, object dynamics are spatially localized. Through cross-
attention, DynaTokens trains the learnable
tokens from a few example trajectories for a scene while keeping the base model frozen, and enables dynamics under new query camera paths. DynaTokens achieves a better simultaneous dynamics-camera tradeoff on VBench2 and WorldScore evaluations than LoRA, block finetuning, and specialized trainable-layer baselines. Analyses of
token attention, ablations, and motion temporality suggest that matching the trainable interface to the structure of the learning target is important for effective adaptation. Project website: https://glab-caltech.github.io/dynatokens/
Read the full abstract ▾
Computer Vision
FlowTool: Controlling Tool Parameter in Image Retouching via Flow Matching
Thanh-Long V. Le, Steven Walton, Seunghyun Yoon et al. (+4 more)
Tool-based image editing (image retouching) is commonly formulated with autoregressive
multimodal large
language models (MLLMs) that sequentially generate
reasoning, tool selections, and parameter values. In this work, we present a novel approach to tool-based image editing by framing the task as a flow matching problem. We introduce FlowTool, a framework that directly models the distribution of high-quality tool parameters conditioned on the input image and user instruction using conditional rectified flow. FlowTool combines a
vision-
language model backbone for
multimodal understanding with a
Diffusion Transformer parameter generator that transforms Gaussian noise into an editing plan. We train FlowTool with a two-stage supervised flow-matching curriculum, followed by reward-based post-training. Across MMArt-Bench, FlowTool-Eval, ArtEdit-Bench, and MIT-Adobe5K, FlowTool achieves significantly stronger reference-based performance than specialized MLLM editing
agents and proprietary MLLMs, while remaining competitive with proprietary models under reference-free evaluation. Moreover, FlowTool significantly improves inference efficiency, reducing latency by at least $50\times$ while requiring nearly $2\times$ less
memory than the compared baselines. These results demonstrate that tool-based image editing can be effectively modeled as conditional generation over structured continuous editing parameters, without autoregressive
reasoning.
Read the full abstract ▾
All authors Thanh-Long V. Le, Steven Walton, Seunghyun Yoon, Branislav Kveton, Trung Bui, Eunho Yang, Viet Lai
Computer Vision
Many Eyes, One World: Feed-Forward 3D Reconstruction from Mixed Cameras
Qiaoge Li, Yifan Zhan, Haijun Yang et al. (+3 more)
Real-world capture is heterogeneous: perspective, fisheye, and $360^\circ$ panoramic images can coexist within a single reconstruction task, yet most feed-forward 3D reconstruction models assume perspective imagery and a uniform input representation. Recent models handling several camera types are either informed of the camera type for each view or reconstruct one image pair at a time. No single-pass method reconstructs mixed-camera tuples containing full panoramas from images alone. We present MEOW, a feed-forward system that jointly reconstructs metric pointmaps and camera poses from one N-view tuple mixing perspective, fisheye and full-panorama images, in a single forward pass from images alone: no calibration, distortion parameters, camera-type labels or poses are supplied for any view. Our guiding design philosophy is to treat heterogeneous-camera reconstruction as a data-adaptation problem rather than an architectural redesign. MEOW retains a perspective-pretrained backbone and learns heterogeneous cameras entirely from a procedural data engine, which renders each scene across a continuous
manifold of camera models with exact rays and depth, and certifies covisibility for every camera-sampled training tuple. Trained on synthetic tuples only, MEOW transfers zero-shot to real captures: on heterogeneous 2D3DS tuples it achieves 79.9 mAA@30 against 53.8 for Wid3R given the camera type of every view; on our
laser-scanned mixed-camera
benchmark it registers every four-view mixed tuple with 79.4 AUC@30. The data engine,
benchmark, and complete evaluation pipeline will be released.
Read the full abstract ▾
All authors Qiaoge Li, Yifan Zhan, Haijun Yang, Haiyang Liu, Yiyi Cai, Chenchi Luo
Computer Vision · Cross-listed from cs.AI
Verifiable Visual Rewards Transfer from Synthetic Scenes to Natural Prompts
Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov et al. (+1 more)
Precise instruction following in image generation, such as satisfying object counts and spatial relations, remains an open challenge at least in part because it is learned using unreliable reward models such as object detectors and
vision-
language models. We introduce Verifiable Visual Rewards (VVR), the first framework for programmatically verifiable image rewards, and show that training on it generalizes to natural
prompts. Each VVR task is a scene of
geometric objects and relations among them, from which we derive both the
prompt and a deterministic verifier, so tasks can be generated in any number and at any chosen complexity. We release VVRBench, with 10,000 tasks over 32 constraint types, and VVRBench-Challenge, with 720 more complex tasks; the strongest model we evaluate—GPT-Image-2.5—solves 21.4% of VVRBench-Challenge. Using VVR scores as rewards for reinforcement learning (RLVVR) raises the accuracy of Stable
Diffusion 3.5 Medium on VVRBench from 2.8% to 28.3% and demonstrates consistent easy-to-hard generalization. These gains extend to out-of-domain
benchmarks, and mixing VVR into existing objectives further improves overall performance and human preference, motivating the adoption of VVR into standard image generation post-training recipes.
Read the full abstract ▾
All authors Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov, Luke Zettlemoyer
Computer Vision
RT-Super: Learning Tumor Segmentation from Longitudinal Images and Reports
Pedro R. A. S. Bassi, Wenxuan Li, Hanxue Gu et al. (+11 more)
Multi-tumor segmentation is important for early cancer detection and allows radiologists to visualize, verify, and understand AI predictions. However, tumor segmentation masks are expensive, time-consuming, and unavailable for many tumor types in public data. Instead, hospitals have vast, readily available data that can guide segmentation: radiology reports, longitudinal images, and multi-phase images. We use this readily available data to substitute for tumor masks in training AI for tumor segmentation. To this end, we propose a new architecture, RT-Super. It has a teacher network, which analyzes the patient's longitudinal images and reports to create high-quality tumor masks. These masks train a student network, which sees a single image and no report. At inference, when longitudinal images and reports are unavailable, we use the student. RT-Super uses a new CNN-
Transformer architecture and novel Consistency Losses that exploit tumor location consistency across longitudinal images. We train RT-Super to segment esophagus, uterus and spleen tumors, which have few or no public masks. Even without training masks, RT-Super can segment these tumors and surpass public AI models. Overall, we demonstrate that learning from longitudinal images, multi-phase images, and reports can overcome mask scarcity and advance multi-cancer detection and segmentation. Code: https://github.com/MrGiovanni/RT-Super
Read the full abstract ▾
All authors Pedro R. A. S. Bassi, Wenxuan Li, Hanxue Gu, Jieneng Chen, Xinze Zhou, Zheren Zhu, Sezgin Er, Ibrahim E. Hamamci, Bjoern H. Menze, Gulhan E. Akan, Kang Wang, Yang Yang, Alan L. Yuille, Zongwei Zhou
Computer Vision
EvolvingAvatar: Interactive 3D Head Generation That Adapts as Conversations Unfold
Junjie Chen, Fei Wang, Kun Li et al. (+5 more)
Interactive 3D head generation requires coordinated speaking and listening motion that responds to an evolving conversation. Existing generators use incoming observations as context but keep their parameters fixed, leaving conversational patterns unused as a learning signal. We introduce EvolvingAvatar, a causal generator that uses test-time training to adapt to user face
video and dyadic audio during interaction. Its dyadic context prediction objective provides a self-supervised learning signal from audiovisual context without target motion labels at test time. Persistent fast weights accumulate these updates within each conversation to guide motion generation, while transient jaw adaptation responds to current audiovisual context. Predicted
speech activity controls how persistent adaptation guides motion. We also introduce InterHead-Bench, a unified 455.95-hour
benchmark built from single-view and dual-view conversation
videos. Experiments show improved conversational motion statistics over strong baselines. On the hardest out-of-distribution split, generation improves as conversations unfold, reducing mismatch with recorded user-avatar expression statistics by up to 11.1% from the first interval.
Read the full abstract ▾
All authors Junjie Chen, Fei Wang, Kun Li, Yiqi Nie, Xun Yang, Yanbin Hao, Linfeng Zhang, Meng Wang
Computer Vision
Remote Sensing Sparse-View 3D Gaussian Splatting via Depth Image-Based Rendering
Jiaming Kang, Zhengxia Zou and Zhenwei Shi
Remote sensing novel view synthesis under sparse observations remains challenging due to insufficient
geometric constraints and limited cross-view supervision. Existing Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) methods are prone to overfitting and face challenges of depth ambiguities, missing cross-view information, and insufficient constraints in under-observed regions. To address these challenges, we propose DIBR-GS, a neural Gaussian Splatting framework that exploits Depth Image-Based Rendering (DIBR) to generate pseudo views for cross-view consistency supervision. Specifically, reliable
geometric initialization is constructed by aligning monocular depth priors with sparse SfM reconstruction, and cross-view appearance priors are incorporated into neural Gaussian representations to enhance appearance modeling under sparse observations. Furthermore, we introduce a progressive DIBR-based pseudo-view supervision strategy to provide additional
geometric and appearance constraints, enabling more complete reconstruction of weakly observed regions. In addition, a height-constrained anchor growth strategy is designed to suppress unreasonable Gaussian expansion. Experiments demonstrate that the proposed method achieves superior performance over existing approaches when training with only 3 input views. Compared with the previous best-performing method, it improves PSNR by 6.83 dB, with relative gains of 14% in SSIM and 60% in LPIPS, while maintaining competitive computational efficiency. Our code is available at https://github.com/kanehub/DIBR-GS
Read the full abstract ▾
Computer Vision
On-Policy Self-Distillation for Multi-Turn Image Editing
Liangbing Zhao, Le Zhuo and Mohamed Elhoseiny
Instruction-based image editing has achieved strong performance in single-turn settings, yet practical editing is often iterative, with each instruction applied to the output of the previous turn. We find that existing editing models degrade rapidly under recursive editing and attribute this failure to a train-test mismatch in the conditioning distribution: models are trained on clean source images but must repeatedly condition on their own imperfect outputs at inference time. To address this, we propose MT-OPSD, an on-policy self-
distillation framework that trains the model on self-generated conditioning states with editing supervision from a clean-conditioned teacher, without requiring multi-turn annotations. We further introduce LME-Bench, a
benchmark of 100 ten-turn editing sessions for evaluating long-horizon robustness. Experiments across three editing backbones show that MT-OPSD substantially improves long-horizon editing success and reduces multi-turn collapse while largely preserving single-turn editing quality.
Read the full abstract ▾
Computer Vision
Simultaneous Translation between Sign Languages
Zetian Wu, Bowen Xie, Stefan Lee et al. (+1 more)
Deaf and hard-of-hearing (DHH) signers cannot converse in real time across different sign
languages today: existing sign-to-sign translation systems run offline, requiring the full source clip before any target sign is emitted. Live use cases - e.g. broadcast interpretation and two-way
video calls - instead demand simultaneous output, while the source signer is still signing. We present, to our knowledge, the first simultaneous sign-to-sign (S2S) translation system, with two wait-k regimes: test-time wait-k inference applied directly to a full-sentence model, and a trained wait-k model via
stochastic multi-path supervision. We further introduce ca-Stream-AL, a computation-aware latency metric for streaming output. Averaged across six S2S directions on both a smaller human-verified test set and a larger synthetic S2S corpus, our streaming system achieves a 38% ca-Stream-AL reduction while staying within a 9% DTW-PA-MPJPE increase and a 2.1 BLEU-4 drop compared to the full-sentence baseline. A word-order case study probes how the streaming model handles word order mismatch between different sign
languages - a consequence of simultaneous translation.
Read the full abstract ▾
All authors Zetian Wu, Bowen Xie, Stefan Lee, Liang Huang
Computer Vision
Yongsung Kim, Jaehoon Lee, Minjun Park et al. (+3 more)
3D
vision transformers such as VGGT predict camera poses and scene geometry from multi-view images in a single forward pass, but their global
attention over all concatenated view
tokens dominates computation as the number of views grows. To reduce this cost, SparseVGGT and HeSS sparsify
attention at the block level, and both retain blocks with high
attention probability. However, we observe that
attention probability poorly predicts how much the model's behavior actually changes when a block is removed, and we show that this mismatch is why performance collapses as sparsity increases. In this paper, we propose ReSS (ReSidual-ReStoring Sparse
Attention), which recasts block selection from a problem of maximizing the retained
attention mass to one of minimizing the drift that sparsification leaves in the residual stream. We introduce a drift score that quantifies how much each block shifts the residual, and, since the drift of a drop set depends on the directions of the contribution vectors rather than on their magnitudes alone, an iterative residual restoration procedure that refines the drop set as a whole. Across three backbones and five
datasets, ReSS preserves dense performance better than prior methods at matched sparsity. Two further results support drift as the quantity that governs the cost of sparsification: maximizing drift degrades performance faster than random selection, and plotted against realized drift instead of sparsity, all methods fall approximately onto a single curve. Code is available at https://github.com/libary753/ReSS.
Read the full abstract ▾
All authors Yongsung Kim, Jaehoon Lee, Minjun Park, Wooseok Song, Hun Hwangbo, Sungroh Yoon
Computer Vision
What Paired Evaluations Reveal under Visual Perturbations
Yongda Wei, Chen Zhang, Yifei Wang et al. (+4 more)
Robustness evaluation must examine diverse visual perturbations, while
benchmarks cover only some real-world conditions and physical testing is costly. Paired evaluations link clean and perturbed predictions for the same image, capturing changes in correctness, confidence, and acceptance beyond aggregate accuracy. We investigate how this image correspondence supports two needs in robustness evaluation: interpreting paired evaluation results and prioritizing samples for physical testing. To interpret paired evaluation results, we fix both sets of prediction records and vary their correspondence within each class. We prove that classwise correct-correct counts give the same sharp bounds on lost acceptance and mean true-class probability decrease among retained-correct inputs as any feasible five-state refinement. Distinguishing persistent from changed wrong answers can further constrain accepted-error transitions, while shared correspondence can establish policy orderings left unresolved by separate cost intervals. To prioritize samples for physical testing, we retain each image's synthetic responses and rank clean-correct images by their mean true-class probability under corruption. Across 44 classifiers, testing the highest-risk 20% finds 67% and 45% of failures under mild screen and print recaptures, versus 58% and 36% for clean confidence and 60% and 37% for an equal-size natural-transformation average. With both probability averaging and an A3Rank scoring adaptation, the tested corruption set yields higher mean failure recall than the natural-transform set; differences between scores depend on the source and budget. Together, these findings show that the value of correspondence depends on the evaluation objective: classwise counts suffice for specified reliability bounds, while image-specific synthetic responses improve the allocation of physical tests within the evaluated pool.
Read the full abstract ▾
All authors Yongda Wei, Chen Zhang, Yifei Wang, Xinyu Wang, Bosen Shao, Hanxi Li, Liping Di
Computer Vision
Less Is More: Genetic Frame Selection for Efficient Novel View Synthesis
Diego E. Farchione, Ramzi Idoughi, Alberto Jaspe-Villanueva et al. (+1 more)
Feed-forward novel view synthesis reconstructs a scene from many input images in a single forward pass, yet more views do not necessarily improve performance: redundant or poorly chosen frames increase computational cost and may degrade reconstruction quality. We address the problem of selecting, from an already captured sequence, a fixed-size subset of input views that is most informative for reconstructing specified target viewpoints. We propose a render-free view selector that scores candidate frames based on three complementary criteria: target-view coverage, measured against observed frames that stand in for the targets, redundancy with previously selected views, and image sharpness. A lightweight scoring network then selects the most informative frames without rendering, reconstruction, or per-scene optimization at inference time. To train the selector, we distill an expensive offline search procedure in which a genetic
algorithm identifies high-quality subsets by directly optimizing reconstruction performance on training scenes. The selector learns to reproduce these choices from
geometric and image-level features alone. Across six
datasets and multiple input budgets, our method consistently outperforms both
geometric and reconstruction-aware view-selection baselines while incurring significantly lower selection costs than reconstruction-based alternatives. Moreover, carefully selected subsets can outperform feed-forward reconstruction from the full input sequence. The learned selector generalizes across diverse reconstruction paradigms (feed-forward, 3D Gaussian Splatting, and NeRF), to object-targeted reconstruction and to a cross-capture setting in which the target views come from a separate acquisition pass. More broadly, our results indicate that explicitly
reasoning about target relevance and inter-view redundancy is a fundamental factor in efficient scene reconstruction.
Read the full abstract ▾
All authors Diego E. Farchione, Ramzi Idoughi, Alberto Jaspe-Villanueva, Peter Wonka
Computer Vision · Cross-listed from cs.RO
EdgeVLN: Runtime-Aware Deployment Ready Quantized Vision Language Navigation Model
Rithvik Jonna, Man Namgung, Aakash Gurram et al. (+1 more)
Vision-
language navigation (VLN) models perform well but target compute-rich platforms, limiting deployment on
memory- and power-constrained robotic edge devices. Compression alone does not establish whether a VLN model fits the
memory, latency, and energy budgets of an edge platform while preserving navigation behavior. We introduce EdgeVLN, a runtime-aware, deployment-ready quantized VLN model that closes this gap. EdgeVLN combines a quantized StreamVLN model with Latent Trajectory Termination Extractor (LATTE), a lightweight causal
transformer that improves real-time stopping by predicting a Stop Action verifier rank. Both execute through our llama.cpp VLN driver, which reconstructs streaming context and prunes
memory tokens on-board. We characterize a pretrained StreamVLN backbone across weight quantization from 8 to 2 bits and multiple inference runtimes to identify a feasible operating point. LATTE reuses backbone hidden states within the budget freed by quantization, requiring neither a second
vision encoder nor an additional backbone forward pass. We evaluate six backbone precisions and seven candidate stop heads on BF16 and IQ4 NL across all 1,839 R2R VLN-CE val-unseen episodes. We measure success rate (SR) in simulation and latency, energy, and resident
memory on an NVIDIA Jetson Orin NX 16 GB. LATTE achieves our highest SR, 58.02 percent on the deployed 4-bit model, exceeding the BF16 baseline with only 0.013 s additional latency per navigation step. Four-bit formats achieve nearly identical SR, but step energy varies 36.8 times by execution path. Only IQ4 NL under our VLN driver fits the board, using 11.35 GB resident
memory while running 20.8 times faster and using 13.3 times less energy than storage-streamed BF16. INT2 collapses. Runtime selection,
memory-
token pruning, and quantization are essential for efficient edge deployment.
Read the full abstract ▾
All authors Rithvik Jonna, Man Namgung, Aakash Gurram, Tinoosh Mohsenin
Computer Vision
Revisiting Risky Tackle Detection with Vision Transformers
Syed Ahsan Masud Zaidi, Lior Shamir and Scott Dietrich
This paper is a Track 2 reproducibility companion to an ICPR 2026 study on risky tackle detection in American football prac- tice
videos. The original work fine-tuned a
Video Vision Transformer (ViViT) on 733 clips labeled with the SATT-3 rubric. It used focal loss, Taguchi L18 augmentation, and 5-fold cross-validation. It reported risky- class recall of 0.67 and risky-class F1 of 0.59. This companion documents the released artifact and traces those numbers to specific scripts, fold out- puts, and aggregation files. The reproduced headline is run_15. It com- bines Gaussian noise with static brightness decrease and uses no rotation and no flip. Its fold-mean risky recall is 0.667 and its fold-mean risky F1 is 0.588. These values match the published headline after rounding. The ablation shows that brightness is the dominant factor. Its risky-recall main-effect range is 0.055, which is larger than the ranges for rotation, flip, and noise. Without augmentation, ViViT reaches risky recall of 0.545 and does not exceed the C3D baseline of 0.583. The raw clips show iden- tifiable student athletes, so they cannot be redistributed. The artifact provides a public sample for pipeline checks and a controlled route for full-data review.
Read the full abstract ▾
Computer Vision
WorldPlay2: Extending Real-Time Interactive World Models in Control and Horizon
Haiyu Zhang, Wenqiang Sun, Tengfei Wang et al. (+5 more)
Interactive world models require responding in real time to versatile controls and maintaining long-horizon consistency. However, modeling heterogeneous controls remains difficult, while explosive contexts and unstable
distillation impede achieving both long-horizon consistency and real-time responsiveness. In this paper, we present WorldPlay2, an interactive world model that couples a factorized hybrid control interface with a co-design of compressed
memory and stable
distillation. 1) Our factorized hybrid control interface integrates frame-aligned action control with structured semantic control that explicitly disentangles scene appearance, character identity, and dynamic semantic events, thereby facilitating effective control learning. 2) To achieve efficient long-horizon modeling, we compress historical contexts into compact
memory tokens shared by the autoregressive student and the bidirectional teacher. This design enables clip-wise,
memory-conditioned score evaluation instead of jointly processing an entire long rollout, substantially reducing
distillation overhead. 3) We further propose Stable Forcing, which initializes the autoregressive student via a few-step strategy and leverages full-rollout replay to preserve the quality of long-horizon rollouts, ensuring robust and stable
distillation. Extensive experiments demonstrate the strong generalizability of our model and its superior performance compared to existing methods.
Read the full abstract ▾
All authors Haiyu Zhang, Wenqiang Sun, Tengfei Wang, Junta Wu, Jun Zhang, Yunhong Wang, Yu Qiao, Chunchao Guo
Computer Vision
Learning to Reason with Persistent Object States for Video Instance Segmentation
Yongxue Xu, Boxue Yang, Ziqian Liu et al. (+3 more)
Video segmentation models maintain object identities by carrying instance information across frames. Under prolonged occlusion, reappearance, or interactions between similar instances, however, an unreliable update can overwrite a valid history and cause persistent identity drift. We introduce POSReasoner, a trainable, plug-and-play framework that explicitly decides when an observation should change an object's state. Each persistent state records identity, confidence, and absence history. A sparse state-observation graph supports Propose-Verify
reasoning: provisional associations are revisited using object history, predicted presence, and competition among identities. The verified decisions determine whether to retain, update, reactivate, or suppress each state, while a learned gate controls the evidence written back to
memory. Only verified transitions update the persistent state used in subsequent frames. POSReasoner uses standard
video annotations and keeps the base model frozen, enabling integration with diverse VOS and VIS architectures. Experiments across long-term VOS and VIS
benchmarks show consistent improvements over strong baselines, with the largest gains under occlusion and object reappearance.
Read the full abstract ▾
All authors Yongxue Xu, Boxue Yang, Ziqian Liu, Shaoqiu Zhang, Rui Qian, Haopeng Chen
Computer Vision
Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin et al. (+8 more)
Multimodal large
language models (MLLMs) can explain deepfake verdicts in natural
language, but such explanations are not necessarily visually grounded in the visual evidence underlying the prediction. A model may describe plausible artifacts inferred from
language priors rather than from image evidence. Existing grounding methods improve visual reliance through decoding or
attention interventions, but they generally strengthen grounding over the entire image, making them ill-suited for forensic artifacts that are subtle, spatially localized, and image-dependent. We propose Look Before You Judge, a training-free framework that formulates explainable deepfake detection as a sequential evidence acquisition process. Instead of directly predicting image authenticity from holistic visual
reasoning, our framework first identifies image-specific candidate evidence regions by contrasting the MLLM's decoder-to-visual
attention between an original image and its Gaussian-blurred counterpart. The identified regions are then inspected individually, and the resulting local evidence is integrated with the global image context before reaching a final verdict. The framework operates without manipulation masks, external forensic models, or parameter updates, making it directly applicable to off-the-shelf MLLMs. Across five open-source MLLMs on TriDF and MMTD-Set, our framework improves detection accuracy by up to 12.8%, reduces CHAIR by up to 33.4% and
hallucination rate by up to 21.3%, and outperforms representative training-free decoding and
attention methods.
Read the full abstract ▾
All authors Chia-Ling Chen, Yu-Ting Ta, Jian-Yu Jiang-Lin, Tai-Ming Huang, Ling Lo, Po-Ching Chen, Yan-Tsung Wang, Pei-Heng Li, Ling Zou, Hong-Han Shuai, Wen-Huang Cheng
Computer Vision
AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation
Yuta Oshima, Ku Onoda, Yusuke Iwasawa et al. (+3 more)
Recent image generation models can take multiple reference images as input and combine them into a new image. However, multi-reference image generation remains challenging: models may omit or duplicate subjects from the references, or produce images in which multiple subjects appear unnaturally pasted. Recent work has proposed image generation
agents that combine image generation models,
reasoning models, and a harness, which is an executable program that specifies how reference images are interpreted, how generation is performed, how outputs are diagnosed, and how the final image is selected. In multi-reference generation, however, references play different roles and outputs must satisfy many criteria at once, such as fidelity to each reference and the naturalness of the whole image, so many parts of the harness could be improved, from how references are processed to how outputs are diagnosed. This makes it hard to predict which changes will improve performance and by how much, and good harnesses difficult to design by hand; indeed, human-written harnesses vary widely in performance. We therefore propose AutoRef, which optimizes the harness automatically while keeping both models frozen: a coding
agent iteratively rewrites the harness code. AutoRef separates the tasks whose feedback informs proposals from the tasks used to select candidates, and continues the search from a beam of the top-ranked harnesses on the selection tasks. Using this procedure, we discover AutoRef-Harness, which improves the open-weight FLUX.2 [klein] 4B from 5.72 to 7.37 on held-out four-reference tasks of the MultiBanana
benchmark, matching or exceeding proprietary models including Nano Banana Pro and GPT-Image-1.5. Without re-optimization, the same harness also improves results when the generator, number of references,
benchmark, evaluator, or
reasoning model differs from those used in the search.
Read the full abstract ▾
All authors Yuta Oshima, Ku Onoda, Yusuke Iwasawa, Masahiro Suzuki, Yutaka Matsuo, Hiroki Furuta
Computer Vision
ReVA: A Scene-Centric Dataset Beyond Repetition for Remote Sensing Video Question Answering
Zhen Yao, Likai Wang, Yuming Yang et al. (+9 more)
Multimodal Large
Language Models (MLLMs) have demonstrated remarkable advances in remote sensing. However, existing remote sensing
multimodal reasoning benchmarks exhibit two critical limitations: they rely on (i) template-driven questions, which causes repetitive questions; and (ii) static images that fail to capture the inherent temporal nature of drone/
UAV videos. This leaves systematic evaluation of remote sensing
video reasoning largely unexplored. To address this gap, we introduce ReVA, a new
dataset for remote sensing
video question answering, designed to assess spatiotemporal, scene-centric, and
reasoning-oriented capabilities of MLLMs. ReVA comprises 2,438 drone
videos spanning 18 cities worldwide (580K frames) and 22K high-quality question-answer pairs across 11 challenging QA tasks. We develop a semi-automatic annotation pipeline that leverages Text LLMs and MLLMs for question-answer generation with human verification. We evaluate 23 proprietary and open-source
Video LLMs on ReVA, exposing fundamental limitations of current models. These findings position ReVA as a critical
benchmark toward better remote sensing
video understanding and temporal
reasoning capabilities for real-world deployments. Our code and
dataset are available at: https://github.com/zyaocoder/ReVA
Read the full abstract ▾
All authors Zhen Yao, Likai Wang, Yuming Yang, Zhihao Zheng, Bo Lang, Qiuyu Tang, Jialu Sheng, Jingqi Xu, Yuehai Yang, Jumal Barker, Xiaowen Ying, Mooi Choo Chuah
Computer Vision
SolveEdit: Benchmarking Visual Problem Solving in Generative Models
Wenjie Shu, Yexin Liu, Harold Haodong Chen et al. (+9 more)
Machine intelligence is often evaluated through abstract
reasoning problems, yet many real-world problems are visual, such as arranging objects, repairing layouts, or tracing routes. Solving these problems requires understanding a scene, inferring what must change to achieve a goal, and realizing that change without disturbing unrelated content. However, existing
benchmarks mainly evaluate perception, generation, or explicitly specified transformations, leaving goal-driven visual problem solving underexplored. To bridge this gap, we introduce SolveEpIT, a
benchmark for visual problem solving through scene transformation. Given an image and a goal, a model must infer a valid transformation from the request, the scene, or a visually expressed rule, then execute it while preserving unrelated content. SoLvEEDrr contains 2,728 cases. Atomic transition contracts specify required and protected conditions, enabling SoLvEScoRE to measure completion and unintended changes without a single reference output. The strongest evaluated model achieves only57.0% SolvEScore. We further introduce SolveEdiT-PLAN, a two-stage visual planner that instantiates the transition before generation. Under matched single-generation evaluation, it improves SoLvEScoRE by 9.1 points on average across three tested generators, including a gain from 57.0% to 71.6% for GPT-Image-2, without modifying the editor.
Read the full abstract ▾
All authors Wenjie Shu, Yexin Liu, Harold Haodong Chen, Xuerui Qiu, Zehan Wang, Yidi Zhang, Yizhan Chen, Zunwei Wang, Minghao Liu, Qi Chen, Harry Yang, Xiaogang Xu
Computer Vision
Sprout: Building Dynamic Memory While Reasoning for Agentic Video Understanding
Wei Chen, Xuanyu Zheng, Yancheng Long et al. (+6 more)
Long
video understanding relies on
video memory to overcome the context limits of
multimodal large
language models. Existing methods follow a build-then-
reasoning pipeline:
memory is built offline for the entire
video, then reasoned over as a static source. In practice a long
video is shared by several questions, and this pipeline is costly at both ends: with few questions, building
memory for the whole
video costs far more than answering them; with many questions, the
memory is never updated, so what is learned while answering questions is lost to the next question. To alleviate these, we introduce Sprout, an agentic framework that builds
memory while
reasoning: a temporal tree that sprouts detailed nodes as questions are answered. The
agent watches the
video segment by segment at a low frame rate, stopping when the current question can be answered, remembers each segment as a coarse node of the tree, and revisits key intervals at a higher frame rate to refine the tree with the recovered details. Once a segment is recorded as text, its
video input is removed from the context history, while the original
video remains reachable through the
video tools. The
memory tree and prior question–answer records persist across questions, so the
memory is online and dynamic: built from the first question onward and updated by every question thereafter. We find that replacing accumulated
video inputs with textual
memory substantially reduces context usage while maintaining accuracy, with slight improvements in some settings. Across
benchmarks on three models, Sprout achieves competitive or improved accuracy relative to representative offline
memory methods, with no upfront construction stage and lower context cost per question.
Read the full abstract ▾
All authors Wei Chen, Xuanyu Zheng, Yancheng Long, Haoyang Xu, Kaiyu Jiang, Bin Wen, Tingting Gao, Han Li, Long Chen
Computer Vision
From Scores to Samples: Elastic Forcing for Autoregressive Video Generation
Chi Zhang, Yueyi Liu, Shi Haoyang et al. (+5 more)
Few-step autoregressive
video generation commonly relies on Distribution Matching
Distillation (DMD), requiring a bidirectional
diffusion teacher and an online fake-score model. We instead learn the rollout distribution directly from reference
videos, eliminating both score models during post-training. Our framework minimizes maximum mean discrepancy (MMD) in frozen self-supervised
video representation spaces, using a hybrid Nyström–Monte Carlo
estimator to balance approximation bias and sampling variance.
Memory-efficient replay and gradient subsampling make this objective practical. Using the same architecture and initialization as Self-Forcing, our 1.3B model improves the VBench Total score from 83.80 to 84.64 while retaining 17 FPS. Removing auxiliary score models also enables 14B post-training on eight H200 GPUs. Beyond
distillation, learning from reference
videos enables the acquisition of new visual styles, semantic concepts, and spatial priors without a target-specific
diffusion teacher.
Read the full abstract ▾
All authors Chi Zhang, Yueyi Liu, Shi Haoyang, Ruichuan An, Haoyu Li, Yuhang Wu, Sen Cui, Miao Liu
Computer Vision
AHMAD: Adaptive Hybrid Multi-task Vision Learning with Assisted Distillation for Keypoint Detection
Mohammad Mahdi, Nedyalko Prisadnikov, Yuqian Fu et al. (+3 more)
Generalist multitasking
vision models aim to unify multiple
vision tasks within a single framework, enabling more efficient and versatile learning. However, handling diverse
vision tasks – spanning dense and sparse predictions – remains challenging due to their inherently varying output structures. In this paper, we propose AHMAD, a simple yet effective framework for generalist multitask learning that integrates different key
vision tasks: semantic segmentation, instance segmentation, depth estimation, keypoint detection, and object detection. Our approach incorporates these five tasks into a unified structure: a shared encoder-decoder with several lightweight task-specific projectors. Under the multitask learning paradigm, we observed a complementary performance gain, achieving a state-of-the-art PQ of 53.1 and an mIoU of 66.5 for COCO-val panoptic and semantic segmentation, respectively. Additionally, for top-down keypoint detection, which typically incurs high computational overhead due to multiple forward passes, we introduce a knowledge
distillation-based method that enables a single forward pass over the entire image, greatly improving efficiency. Ultimately, our model delivers a lightweight yet effective generalist multitask learning framework, demonstrating strong performance across five
vision tasks.
Read the full abstract ▾
All authors Mohammad Mahdi, Nedyalko Prisadnikov, Yuqian Fu, Carmelo Scribano, Danda Pani Paudel, Luc Van Gool
Computer Vision · Cross-listed from cs.CL
Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning
Yingjin Song, Denis Paperno and Albert Gatt
High instance-level accuracy can mask inconsistencies in spatial
reasoning when objects exchange positions or their roles are reversed in the query. The internal representations supporting relative-position
reasoning remain poorly understood. We investigate two complementary components of this process: tracking object locations in the input and representing their query roles. Across three VLMs with visual or textual inputs and their
language-model backbones, activation patching reveals a staged progression from early-layer source representations through intermediate-layer query-object representations to late-layer answer states. Targeted interventions further establish causal links along this progression: manipulating source-side representations shifts location information at query-object mentions and ultimately alters relation predictions. Beyond object-location information, we also identify a stable query-side direction associated with the roles of the two objects in the comparison. Steering along directions estimated on synthetic scenes generalizes to natural-image
benchmarks, improving accuracy and both forms of paired consistency in most settings without retraining. Our findings reveal complementary components of relational
reasoning across visual and textual settings and show how targeted interventions can improve the consistency of models' behavior.
Read the full abstract ▾
Computer Vision
Handwritten Text Recognition Lives in the High-Pixel Variance Subspace
Carlos Garrido-Munoz and Jorge Calvo-Zaragoza
In self-supervised pretraining for Handwritten Text Recognition (HTR), pixel reconstruction methods outperform contrastive methods, unlike in natural-image classification. We argue that this difference follows from where discriminative signal lies in pixel space: for HTR, it is concentrated in high-variance directions and largely absent from low-variance ones. This predicts that objectives preserving high-variance pixel content will transfer best. We test six SSL methods from three families (pixel-grounded MIM, JEPA, and contrastive) under matched encoder, data, and evaluation protocols on six handwriting
benchmarks across five
languages. With full labels, pixel-groundrounded SSL achieves the lowest CER on every
benchmark and both frozen probes, exposes per-position character information that other families recover only through the readout, and is the only family to benefit from pretraining on real handwriting. Pixel-grounded representations are also more label efficient. Across
datasets, encoder
alignment with the high-variance pixel subspace predicts CER within every method. With a pretrained LLM decoder, a frozen pixel-grounded encoder is competitive with fully fine-tuned supervised baselines; full fine-tuning achieves the lowest mean CER and ranks first or second on every
benchmark. These results show that the value of pixel reconstruction depends on where discriminative signal lies in the input.
Read the full abstract ▾