Computer Vision
Neural Block-wise Auto-exposure for Single-Shot High Dynamic Range Imaging
Zhengguo Li, Chaobing Zheng and Wei Wang
Block-wise varying exposures are effective for cap- turing appearance of a real-world high dynamic range (HDR) scene via a single-shot. However, brightness order reversal (BOR) artifacts, blocking artifacts and loss of details in highlight/shadow regions are possible issues for block-wise varying exposures. In this paper, a novel neural block-wise auto-exposure
algorithm is proposed to address the issues. The proposed
algorithm maps a temporary image and its exposure time to the exposure times of all non-overlapping 16 X 16 blocks. The temporary image can be either an sRGB or a raw image. Although the sRGB image is widely adopted in existing auto-exposure
algorithms, exposure times of blocks in the highlight and shadow regions might not be accurately learned due to the limited radiance measurement capability of the sRGB image. This paper thus also explores the feasibility of temporary image as a raw image. Due to the better radiance measurement capability of the raw image, the exposure times of blocks are learned more accurately, and more details in the highlight/shadow regions of the real-world HDR scene are captured by the resultant HDR imaging system. Experiment results validate the proposed neural block-wise auto-exposure
algorithm.
Read the full abstract ▾
Computer Vision
VisualCoding: Learning Executable Skills for Reliable Visual Reasoning
Liu Jiaqi, Jin Ruinan and Li Xiaoxiao
Learning reusable skills can improve visual
reasoning agents, yet applying a skill still requires the
agent to implement or invoke a procedure and produce the final answer. Errors and
hallucinations at these steps can prevent acquired experience from translating into reliable single-run performance. We propose VisualCoding, which combines execution-guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution-Guided Program Acquisition composes visual primitives and additional code into family-level solvers, then iteratively tests and revises them on labeled training instances. Scope-Aware Direct Execution runs a retained solver and returns its valid in-scope answer. We evaluate VisualCoding on four models across four
benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini-3.1-Pro and by 14.78 points when reusing the frozen Pro-acquired artifacts with Flash-Lite. Ablations support execution-guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq-liu/VisualCoding.
Read the full abstract ▾
Computer Vision
RealityBench: Simulated Environments to Evaluate a Vision Language Model's Ability to Document Human Physical Work
Andrew Perrault, Anmol Takiar, Dominik Winecki et al. (+1 more)
Augmenting worker capabilities in the physical world through guidance has been a decades-long goal for AI
agents. One key supporting capability is to understand what the worker is doing, seeing through the human's camera or smart glasses, and assist them in documentation. Despite advances in
vision language models, their capabilities in these settings remain uncertain and are challenging to
benchmark because most physical work data is proprietary, and involves too much variability to study in a principled manner. Unlike structured settings such as kitchens and warehouses that have predictable lighting, geometry, and objects, field-facing tasks such as rooftop solar installation and vehicle repair are unconstrained and high variance. Here, we model real-world physical work in Blender across eleven environments, incorporating variants in both image acceptability and documentation. The acceptability task asks whether a particular view would be suitable proof-of-work and documentation asks whether an
agent can understand the task-relevant state of the environment from that view. Across 26
vision-
language models, we find that top models largely solve documentation when the answer is text in frame, but are weak at reading measurements and at judging small parts and spatial relations, such as whether a tape measure leans. Acceptability is harder to pin down because human raters disagree with each other. Measured against a scorer that agrees with the raters' majority about as often as a single rater does, the best models cannot be distinguished from a single non-author rater, at a cost within the range of a human pass. In the three environments where we can trace acceptability errors, most wrong rejections come from misreading the scene rather than the requirement. On the three documentation questions for which we have real images, model rankings on rendered frames transfer to real images.
Read the full abstract ▾
All authors Andrew Perrault, Anmol Takiar, Dominik Winecki, Arnab Nandi
Computer Vision
Learning Transferable World-Action Models from Task-Paired Human Videos
Jeongeun Park, Taekyung Kim, Dongyoon Han et al. (+1 more)
World-action models inherit broad visual and physical priors from
video pre-training. However, standard within-
video next-frame prediction allows them to rely on appearance shared between the context and future, rather than learning interactions that transfer across scenes and embodiments. We introduce WATCH, a cross-
video pre-training that learns from task-paired human demonstrations collected across diverse environments. Conditioned on the
video and actions of one demonstration, the model jointly predicts another demonstration's actions and future frames from its current observation. Predicting one demonstration from another encourages the model to learn task-relevant interactions that transfer across scenes and demonstrators. Our pre-training shows that these capabilities can be learned from readily scalable human-human pairs alone, without requiring matched human-
robot demonstrations. Analyses before
robot post-training show improved motion
alignment across embodiments and better action and
video prediction on unseen
robots, with representations closer to those learned from
robot pairs. After
robot post-training, WATCH improves MimicDroid in-context success from 40.9% to 68.3% with Cosmos Policy and from 59.8% to 77.7% with Cosmos3, with the largest gains on the unseen levels. Gains persist under context-free post-training across
robot embodiments. WATCH outperforms within-
video pre-training and co-training baselines, supporting cross-
video prediction as a scalable approach to learning transferable world-action representations.
Read the full abstract ▾
All authors Jeongeun Park, Taekyung Kim, Dongyoon Han, Sangdoo Yun
Computer Vision
TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling
George Matthew, Guoqiang Zhang and Kenta Niwa
Accurate sampling from continuous-time generative models can require many model evaluations. Low-step sampling reduces cost, but coarse updates introduce endpoint errors that can accumulate and degrade generation quality. We introduce TraceSampler, a post-training method that corrects these errors without modifying the pretrained model. Rather than optimising low-step updates directly, TraceSampler models endpoint error as a sample-specific prediction problem. A higher-step reference integration over the same interval provides the supervised target, while a compact representation makes high-dimensional error predictable from the sampling trace. This representation combines shared residual structure, trace-derived directions, and a low-rank remainder, so shallow per-step MLP heads predict sample-dependent coefficients. These coefficients drive a two-stage learned residual correction: the first stage predicts the full endpoint residual, while an intermediate backbone evaluation provides new trace information to predict the remaining residual; this evaluation is reused by the following step. TraceSampler is evaluated with Rectified Flow on CIFAR-10, SiT-XL/2 on ImageNet-256, and SD3.5 Medium for text-to-image generation. At 10 steps, it reduces trajectory MSE relative to matched-step Euler by 90.5% on CIFAR-10 and 85.6% on ImageNet-256, while achieving FID scores of 2.692 and 2.189, close to 120-step Heun references of 2.601 and 2.056 with 24$\times$ fewer backbone evaluations. On SD3.5 Medium, TraceSampler improves image quality and
prompt following; at 10 steps, ImageReward rises from 0.752 to 0.906 while trajectory MSE falls by 70.7%. These gains preserve Euler's backbone-evaluation budget, add less than 0.4% inference FLOPs, and incur negligible wall-clock overhead, showing that coarse-step sampling error can be made predictable from the sampling trace and efficiently corrected.
Read the full abstract ▾
Computer Vision
Real-Time Traffic Density Estimation: A Hybrid Computer Vision and Predictive Machine Learning Pipeline
Prince Kwabena Appiah Boadu
Traffic density estimation from camera feeds requires connecting visual perception with short-horizon forecasting. We present a two-stage pipeline that combines vehicle detection and tracking with a spatial logic layer, followed by a temporal
estimator. The spatial layer filters bounding boxes by their overlap with a precomputed road mask and extracts per-frame traffic features. LSTM, GRU, and Temporal Convolutional Network (TCN)
estimators use these features to predict five future steps from a ten-frame input window. We
benchmark ten detector variants from six families on a
dataset of 5,716 annotated frames using a single NVIDIA GPU. YOLOv5l achieves the highest mAP@50:95 of 0.524 at 56 frames per second, while YOLOv8n reaches 258 frames per second at 0.487 mAP@50:95; these rates measure detector inference without tracking or spatial filtering. Temporal evaluation on 29,243 held-out windows yields aggregate R-squared values of 0.858 for GRU, 0.856 for LSTM, and 0.840 for TCN. These metrics measure forecasts of detector-derived telemetry and are dominated by direction variance, rather than establishing absolute traffic-density accuracy. No road mask was available for the evaluation stream, limiting validation of the spatial filtering component. A live run on a third camera stream examines forecasting behavior within the complete pipeline. The results characterize detector speed-accuracy trade-offs and the limitations of short-window traffic telemetry forecasting.
Read the full abstract ▾
Computer Vision
Manifold Navigation Learning Using Recurrent Networks: Efficient Training Techniques
Sarah Harkins Dayton, Sathya Rengaswami, Vasileios Maroulas et al. (+2 more)
Position tracking is a well-studied problem in Euclidean navigation, yet comparatively little
attention has been given to navigation on general
manifolds, where motion is governed by the underlying geometry of the space. In this work, we formalize the problem of
manifold navigation as a velocity-driven tracking task and propose a coordinate-chart-based learning framework in which learning models operate entirely in a Euclidean representation. Predicted trajectories are mapped to the
manifold via an immersion function prior to error computation, allowing
geometric consistency to be evaluated without explicitly enforcing
manifold constraints during learning. Within this framework, we focus on Gated Recurrent Units (GRUs) for sequential position prediction on curved
manifolds after comparisons with alternative recurrent architectures. We evaluate our GRU models, trained with pertinent custom loss functions, against both a standard MSE loss function and a naive vector-addition baseline. We evaluate our approach on both analytically (and numerically) defined
manifolds, as well as a surface reconstructed from a
LiDAR point cloud. Our results show that recurrent models can learn to approximate
manifold-constrained dynamics from velocity sequences and consistently outperform Euclidean baselines. Our results demonstrate that certain custom loss functions significantly improve long-horizon prediction accuracy compared to conventional point-wise loss functions.
Read the full abstract ▾
All authors Sarah Harkins Dayton, Sathya Rengaswami, Vasileios Maroulas, David L. Boothe, Ioannis D. Schizas
Computer Vision
EV-JEPA: Elastic V-JEPA for Any Device and Cross-Device Inference
Anxhelo Diko, Jieyu Lin, Stefano Ambrogio et al. (+5 more)
Foundation models like V-JEPA are increasingly deployed across a heterogeneous edge-cloud continuum spanning smart glasses, phones,
robots, and servers. Yet, they are typically trained as rigid, fixed-capacity networks bound to one device, supporting neither reconfigurable compute budgets nor collaborative execution. Existing approaches address only one axis at a time: slimmable and NAS-based methods provide width elasticity but assume single-device inference, whereas split-computing methods enable cross-device collaboration only for fixed partitions and static capacities. We introduce Elastic V-JEPA (EV-JEPA), a single model trained once and deployable everywhere along the continuum. Its width is reconfigurable, and it is natively partitionable across heterogeneous devices. At its core is Transition-Aware Triplet Training (T3). At each step we pick a narrow and a wide width, and train three paths: each width alone, plus a hybrid path that runs the early layers narrow and the rest wide. The hybrid switch point is where one device hands off to another, so the same weights learn both single-device and cross-device execution. Across various
benchmarks, EV-JEPA falls within 1 accuracy point of independently trained baselines while improving over previous elastic training approaches by up to 11 accuracy points. Narrow-to-wide cross-device hybrid execution recovers to within 1.1 points of the highest-capacity standalone model and supports many-to-one, any-to-any, and variable-split configurations. Finally, we show that the elastic principle extends beyond the backbone to an elastic
token compression mechanism that exposes controllable trade-offs among on-device computation, latency, and accuracy.
Read the full abstract ▾
All authors Anxhelo Diko, Jieyu Lin, Stefano Ambrogio, Ziyun Li, Mahmoud Assran, Théo Moutakanni, Barbara De Salvo, Chiao Liu
Computer Vision
MvFFN: Multi-view Floor plan Feed-forward Network for Unposed Wide-Baseline Panorama Layout Reconstruction
Yuguang Li, Yichuan Deng, Zixuan Liu et al. (+5 more)
Reconstructing accurate camera poses and floor-plan layouts from sparse, unposed, wide-baseline RGB panoramas remains a challenging open problem. Direct multi-room floor-plan prediction is especially difficult in this setting: pose, geometry, semantics, and global layout structure are tightly coupled, yet a complete floor-plan is hard to learn as a single global output. Our key insight is to keep the model's predictions strictly local. Multi-view Ordered Wall Instance Segmentation (MOIS) is designed around this concept. Each panorama column predicts only the wall it observes plus the next two adjacent walls in clockwise order. The global wall partition, room layouts, and shared coordinates emerge from deterministic aggregation of overlapping local predictions. Building on MOIS, we propose Multi-view floor-plan Feed-Forward Network (MvFFN), which jointly predicts coarse camera poses, dense global geometry, and per-wall semantic outputs. A downstream Layout Chaining (LC) post processing pipeline then assembles these predictions into multi-room floor-plans with shared global coordinates, consistent wall identities, and connected room structure. The dense MOIS predictions recover cross-view wall identity and local within-room connectivity more accurately than composed modular baselines, and the assembled layouts substantially outperform prior methods on wall and junction level accuracy. We also introduce the ZInD-CrossView
dataset, which augments ZInD with globally unique wall instances and cross-view segmentation labels for this task.
Read the full abstract ▾
All authors Yuguang Li, Yichuan Deng, Zixuan Liu, Ivaylo Boyadzhiev, Xiuyuan Wu, Xinyu Zou, Linda Shapiro, Alex Colburn
Computer Vision · Cross-listed from cs.AI
ConicFlow: constrained Inference-time optimal control for flow matching models
Shervin Khalafi, Kasra Fallah, Sung Cho et al. (+3 more)
We propose ConicFlow, which frames constrained sampling from a flow matching model at inference time as an optimal control problem that balances deviation from the pretrained trajectory against reward, subject to hard terminal constraints. We discretize this problem and adapt receding-horizon control, optimizing only a short window of the trajectory at each step for computational tractability. We address the resulting sequence of receding-horizon subproblems with a receding-horizon primal-dual
algorithm. Its dual multipliers capture the difficulty of satisfying each constraint and are refined throughout sampling, steering generation toward feasible final samples. We further propose a terminal-estimate variant that avoids accumulating local truncation errors by evaluating the reward and constraints at the predicted terminal point rather than at the end of the current horizon. Under standard regularity assumptions, we establish finite-horizon guarantees on suboptimality. Under convex-affine structure, we derive a sharper bound that scales linearly with omitted horizon length, and we show that the terminal-estimate variant is provably tighter when the flow dynamics change slowly. Experiments in both low- and high-dimensional settings with a recent large-scale flow matching model show that ConicFlow achieves high reward and strong constraint satisfaction while deviating less from the pretrained flow than recent baselines.
Read the full abstract ▾
All authors Shervin Khalafi, Kasra Fallah, Sung Cho, Dongsheng Ding, James Anderson, Alejandro Ribeiro
Computer Vision · Cross-listed from cs.AI
Controllable Diverse Sampling through Pullback Directions
Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang et al. (+1 more)
Conditional
diffusion models can generate high-quality images that align closely with a given
prompt. However, samples generated from different random seeds may remain visually similar, limiting creative exploration. Existing approaches can increase variation, but often require additional optimization or sampling overhead, while stronger perturbations may introduce distortions that degrade image quality or
prompt alignment. We propose a training-free method that leverages the local pullback geometry induced by the mapping from the conditioning-
embedding space to the image space to identify directions of maximal output sensitivity. By steering different sampling trajectories along these directions, we encourage distinct generations while maintaining consistency with the conditioning signal. Experiments on MS-COCO and BrushNet show the efficacy of the method in improving diversity on
prompt-conditioned image generation and inpainting.
Read the full abstract ▾
All authors Sylvain Dehayem Kenfouo, Aaron Zweig, Mingxuan Zhang, Elham Azizi
Computer Vision
GeoFix: Geometry Fixing for 4D Consistent Generation
Yeobin Hong, Hyojun Go, Dohun Lee et al. (+7 more)
Video diffusion models increasingly serve as priors for 4D generation, but the generated
videos often lack
geometric consistency. Recent approaches address this through post-training by designing
geometric rewards, yet this can degrade texture and fine detail, requiring auxiliary appearance rewards and careful reward balancing. To address this challenge, we propose GeoFix, a unified framework that improves
geometric consistency and visual quality with a single
geometric reward, applicable to both training and inference-time reward
alignment. Our reward measures cross-view depth consistency in static scene regions, targeting structural inconsistencies without directly constraining appearance or penalizing valid object motion. Crucially, depth agreement increases with camera field-of-view overlap even in real
videos and 4D
datasets, allowing a smaller camera movement to be mistaken for better geometry. We account for this effect by calibrating the reward against the overlap-dependent trend. GeoFix achieves a 39% improvement in WorldScore 3D consistency and a 4% increase in VBench Quality Score.
Read the full abstract ▾
All authors Yeobin Hong, Hyojun Go, Dohun Lee, Jeongjae Lee, Sunghwan Hong, Federico Tombari, Zhaochong An, Jong Chul Ye, Konrad Schindler, Dominik Narnhofer
Computer Vision
TGSR: Text-Grounded Diffusion for Faithful Image Super-Resolution
Zhiming Hu, Ran Zhang, Angela Ye et al. (+7 more)
Diffusion models have achieved remarkable success in image super-resolution (SR) by leveraging strong generative priors to enhance perceptual quality. However, for textual content, these priors often produce text
hallucinations: non-existent or distorted text, which alter scene semantics and compromise model reliability. While recent methods focus on enhancing text generation, they do so indiscriminately, leaving
hallucinations fundamentally unresolved. We argue the core problem is a lack of selective control. Therefore, we reformulate text SR as a selective generation problem: text is resolved only when reliable generation is expected, while other scene content is fully super-resolved. We introduce TGSR (Text-Grounded Super-Resolution), a test-time framework that progressively identifies and suppresses likely hallucinated text regions during generation. To enable this, we train dedicated detectors on a large-scale
dataset curated from diverse
diffusion-based SR outputs. TGSR consistently improves text fidelity across multiple backbones, increasing recognition precision by up to 17.68 points without degrading perceptual quality, underscoring the value of selective control in generative SR.
Read the full abstract ▾
All authors Zhiming Hu, Ran Zhang, Angela Ye, Raghav Goyal, Faraz Ali, Tristan Ty Aumentado-Armstrong, Samrudhdhi B. Rangrej, Konstantinos G. Derpanis, Iqbal Mohomed, Alex Levinshtein
Computer Vision
RoboFake: Are Deepfake Detectors Ready for Humanoid Face Robots?
Hao Chen and Junnan Xu
Humanoid face
robots can now move their lips in sync with
speech and show detailed facial expressions. Give such a
robot the face of a real person and drive it with generated
speech, and you get a face forgery that happens in front of the camera: the camera records a real object, and no pixel is edited. We call this a RoboFake. In this paper we ask what current deepfake detectors do when they see a
robot face. We test six public detectors trained on FaceForensics++ on
video of a physical lip-syncing face
robot, on a synthesized
video of the same
robot, and on rendered
robot faces. All thresholds are fixed on FaceForensics++, and every result is compared with real human faces that went through the same processing as the
robot videos. At a threshold that flags 5% of real faces, three detectors flag almost every physical
robot frame, while two flag fewer than 7%. A matched human control and masking show that the response comes from the face, not from the processing or the background. However, it does not track synthesis: with only the face visible, three of the six detectors give the physical
robot a higher fake score than its synthesized
video. The response is also fragile, and light JPEG compression is enough to push most
robot frames below the threshold for some detectors. Our results suggest that current detectors flag today's
robots because their faces do not look human, not because they recognize a forgery. We release our evaluation protocol and code, and state three testable hypotheses about realistic, interactive RoboFakes.
Read the full abstract ▾
Computer Vision
Thinking with Images without Coordinates
Meng Cao, Haoze Zhao, Yi Wang et al. (+2 more)
Recent advances in multi-modal large
language models have shifted toward the "thinking with images" paradigm to improve
reasoning faithfulness, yet existing methods rely heavily on explicit coordinate- based grounding. We identify two fundamental limitations in this approach, i.e., a semantic gap where sparse regressions miss high-dimensional latent structures, and a resolution gap due to the granularity mismatch between patch-level
tokens and pixel-level coordinates. To address these, we propose TiCO (Thinking with images without COordinates). It reformulates visual
reasoning as a semantic evidence discovery process within the latent space, and evidence is injected as
token-level regions without pixel coordinates. Specifically, TiCO utilizes a dedicated probe
token to estimate similarity distributions and performs importance sampling over spatially connected components to generate diverse, semantically dominant evidence rollouts. Extensive experiments across multiple high-resolution and general visual
reasoning benchmarks demonstrate that our TiCO consistently outperforms state-of-the-art coordinate-based methods with significantly improved inference efficiency.
Read the full abstract ▾
All authors Meng Cao, Haoze Zhao, Yi Wang, Ian Reid, Xiaodan Liang
Computer Vision
BigCell: Generating Gigapixel Whole-Slide Images
Srikar Yellapragada, Alexandros Graikos, Zilinghan Li et al. (+10 more)
Whole-slide images (WSIs) are gigapixel-scale digital scans of histopathology tissue and serve as the primary basis for cancer diagnosis. Existing histopathology generative models operate at the patch scale, producing fixed-size tiles of at most 1024$\times$1024 pixels which encompass only a small fraction of the WSI. In contrast, text reports for clinical diagnoses and slide-level diagnostic labels – the supervision that exists at scale – operate at the whole-slide level. We present BigCell, the first approach to generate coherent whole-slide histopathology images at native gigapixel resolution. BigCell trains a flow-matching
transformer at 2048$\times$2048 pixels on over 120,000 publicly available WSIs spanning multiple organ types. Conditioning on
embeddings from pathology foundation models and using an inference-time sliding-window pipeline, BigCell generates coherent images of up to 32,768$\times$32,768 pixels, outperforming prior tile generators on both tile-level and slide-level fidelity metrics. To make gigapixel synthesis practical, we introduce an adaptive scheduling strategy that prioritizes important slide regions, speeding up image generation by over 3\ttimes with minimal quality degradation. Furthermore, we propose the first text-to-image synthesis at 8k resolution by training a separate model that produces dense conditioning grids directly from pathology text reports. We demonstrate that augmenting with synthetic samples improves slide-level few-shot classification performance by up to $18 \%$ over baselines trained on real data.
Read the full abstract ▾
All authors Srikar Yellapragada, Alexandros Graikos, Zilinghan Li, Kostas Triaridis, Tarak Nath Nandi, Ji Dong K Bai, Beatrice Knudsen, Tahsin Kurc, Prateek Prasanna, Rajarsi R. Gupta, Ravi K Madduri, Joel Saltz, Dimitris Samaras
Computer Vision
Image Diffusion Models Are Good 3D Human Motion Generators
Siyuan Yang, Chong Wang, Hao Cheng et al. (+4 more)
Text-to-motion generation typically relies on
dataset-specific skeletal representations, with models trained separately for
benchmarks with different joint layouts. We investigate whether image
diffusion architectures can model 3D human motion through a common image-compatible representation that accommodates heterogeneous skeletal layouts while preserving motion-specific kinematic structure. To this end, we organize root-centered joint trajectories into
topology-informed joint–time images and model global root dynamics through a complementary latent stream. A motion autoencoder conditions local-motion decoding on the root latent, while a text-conditioned Scalable Interpolant
Transformer (SiT) jointly generates both latent streams, with supervision on decoded 3D joint positions further constraining generation in motion space. In addition to
dataset-specific models, we train a single motion autoencoder on HumanML3D and KIT-ML, followed by a single text-conditioned SiT shared across both
datasets, preserving their respective skeletal definitions without skeleton retargeting to a common joint layout. Experiments on HumanML3D and KIT-ML show that our
dataset-specific models achieve state-of-the-art performance, while shared training yields further gains on both
benchmarks, demonstrating the benefits of shared modeling across heterogeneous skeletal layouts.
Read the full abstract ▾
All authors Siyuan Yang, Chong Wang, Hao Cheng, Jun Liu, Bihan Wen, Alex Kot, Hedvig Kjellstrom
Computer Vision
SliderEdit++: Improved Slider Trajectories in Instruction-Based Image Editing
Arman Zarei, Sayan Nag, Bryan Wang et al. (+5 more)
Slider-based control aims to give users continuous, fine-grained control over the intensity of individual edits in instruction-based image editing. However, existing approaches fail to deliver reliable continuous controllability: traversing a slider can produce abrupt visual jumps and non-monotonic changes, while jointly controlled sliders can interfere with one another. To systematically study these limitations, we introduce SlideBench, a comprehensive
benchmark for continuous image editing spanning diverse editing categories and equipped with a unified suite of metrics quantifying monotonic progression, perceptual continuity, and inter-slider disentanglement. Building on these observations, we propose SliderEdit++, which explicitly regularizes the intermediate editing trajectory rather than supervising only its endpoints. Our method combines velocity- and semantic-space trajectory regularization with spectral control to encourage smooth, predictable slider traversal, together with a disentanglement objective that reduces interference between independently controlled edits. On SlideBench, SliderEdit++ achieves state-of-the-art performance, improving the overall score by 18.5% relative to the strongest baseline. At the same time, using a few-step editing backbone makes our method approximately 3x to 20x faster than existing slider-based approaches, enabling substantially more responsive interactive editing.
Read the full abstract ▾
All authors Arman Zarei, Sayan Nag, Bryan Wang, Franck Dernoncourt, Varun Manjunatha, Nedim Lipka, Soheil Feizi, Samyadeep Basu
Computer Vision
Latent World Action Model for GUI Agents
Seungjun Yu, Hojun Choi, Seojeong Park et al. (+2 more)
Pixel-space world models have recently emerged as a promising approach for GUI
agents, predicting future screens prior to execution and subsequently re-perceiving them for action selection. However, this two-stage paradigm faces two key challenges in GUI environments: (L1) the high computational cost of pixel reconstruction and (L2) the need to reconstruct large and diverse appearance-level details beyond what is required for action selection. In practice, this screen-generation objective can induce an identity shortcut problem, where future-state prediction largely preserves the current state rather than capturing action-induced changes. In this work, we reformulate GUI world modeling by shifting the prediction target from full-screen reconstruction to action-induced future information in latent space. Concretely, we leverage a large
multimodal model as the latent world model, whose shared representations support two dedicated heads for both action selection and predictive transition modeling. The action head directly translates these latent representations into action selection, while the transition head captures action-conditioned future information. To mitigate the identity shortcut, we further introduce an action-aware contrastive objective that encourages the transition representations to capture discriminative action-induced changes. Extensive experiments across offline and online GUI settings show that our method achieves up to 31.9% and 40.6% relative gains in success rate over pixel-space world models, respectively, while substantially reducing inference latency by up to 98.9%. Our code is available at https://github.com/sjyu001/latent2world.
Read the full abstract ▾
All authors Seungjun Yu, Hojun Choi, Seojeong Park, Jaeyo Shin, Hyunjung Shim
Computer Vision
CAPTOR: a vision-language foundation model for Chest-Abdomen-Pelvis Oncologic CT
Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali et al. (+3 more)
Purpose : To develop and evaluate CAPTOR, a deep-learning
vision-
language foundation model for oncologic chest-abdomen-pelvis (CAP) CT interpretation. Materials & Methods : CAPTOR (Chest-Abdomen-Pelvis foundaTion model for Oncologic Radiology) is a contrastive
vision-
language model, trained using a pretraining strategy for oncologic CAP CT (in single- or multiple-series acquisitions), on a multicenter retrospective
dataset from North America, Europe and Asia, predominantly from tumor lesion examinations. CAPTOR was compared with state-of-the-art CT foundation models on three downstream tasks: disease detection, survival prediction using TumorImagingBench and image
retrieval. Classification and
retrieval were both evaluated on held-out internal and public test sets, stratified by anatomical coverage and indication. Bootstrap resampling was used to estimate confidence intervals, and pairwise comparisons were performed using permutation tests. Results : CAPTOR was trained and tested on 25,512 and 10,794 CT-report pairs respectively. TumorImagingBench
datasets comprised between 134 and 421 studies each. CAPTOR outperformed all baseline models, on the classification task, on the CAP subset (AUC 0.767, 95%CI: 0.734-0.793, vs. 0.695, 95%CI: 0.645-0.736 for the next-best, p<0.001) and oncologic subset (AUC 0.784, 95%CI: 0.759-0.805, vs. 0.734, 95%CI: 0.703-0.767 p<0.001), ranked first on three of the four oncologic prognostic
benchmarks, and achieved superior
retrieval performance on CAP and oncologic subsets. Conclusion : A foundation model trained on oncology-enriched CAP CT data outperformed relevant state-of-the-art models in its target clinical context, underscoring the importance of clinically aligned pretraining. We release our preprocessing, training and inference code : https://github.com/Guerbet-AI/CAPTOR and model weights : https://huggingface.co/Guerbet-AI/CAPTOR
Read the full abstract ▾
All authors Séléna Jean-Mactoux, Sophie Loizillon, Ghina Jardali, Amandine Crombé, Nathalie Lassau, Alexandre Bône
Computer Vision · Cross-listed from cs.LG
Agreement versus Softmax under Spatial Disambiguation
Michał Zakrzewski, Mariia Shpir, Taras Rumezhak et al. (+1 more)
ScanRefer Multiple queries require spatial disambiguation: several instances of the target class are present, and the referencer must pick one. When per-query logits were never logged, a common substitute is cross-epoch agreement. On the full ScanRefer validation set (9,508 queries; GeoClassifier with ground-truth proposals), object-level softmax is similarly miscalibrated on Unique and Multiple (ECE 0.068 vs. 0.102; the scene-bootstrap gap interval includes zero). Fixed-final agreement on the same checkpoints reports ECE 0.041 vs. 0.272 (gap +0.231). Modal-vote agreement is 0.045 vs. 0.318. Most Multiple errors are same-class wrong-instance (45.6%);
geometric miss (H2) is 59.6%, and wrong-category (H1) is 16.7%. A non-monotone H1 pattern does not appear under the default fixed-final protocol, and we retract it. Logit-free thresholds rank spatially hard queries poorly. If only agreement was logged, calibrate abstention on Multiple and report wrong-instance rates with H1.
Read the full abstract ▾
All authors Michał Zakrzewski, Mariia Shpir, Taras Rumezhak, Volodymyr Karpiv
Computer Vision
Token-Budgeted Candidate Retrieval for 3D Scene-Grounded VLMs: Category Names vs. Pairwise Geometry
Michał Zakrzewski, Mariia Shpir and Taras Rumezhak
Under a fixed context budget, which objects and relations should a 3D scene-grounded VLM see first? On ScanNet indoor referring
benchmarks (Dai et al., 2017), matching the query to object category names outperforms pairwise geometry scoring at every reported K. The gaps are largest at high recall. Analytic pairwise cues beat per-object cue scoring and query-agnostic kNN graphs, including on paraphrases. MiniLM and BGE-small still score higher on those splits. We evaluate
token-budgeted candidate
retrieval with axis-aligned boxes, a
token-ratio metric, training-free baselines, two small encoders that match q to each object's category name, and QCRR (Query-Conditioned Relation
Retrieval), a deterministic pairwise-geometry scorer. Across ScanRefer gt_val plus a custom 20% scan-id split of Nr3D and Sr3D (35,504 queries; not official ReferIt3D val), additive noun matching reaches R@10 0.864 on ScanRefer against QCRR's 0.626; MiniLM and BGE-small reach 0.911 to 0.913 on ScanRefer and 0.941 to 0.947 on Nr3D. A corrected 3DGraphLLM harness (native Acc@0.25 0.67 on the full graph, n=300 ScanRefer queries) shows that context selection changes downstream accuracy under a matched object budget (Random 0.44, QCRR 0.56, MiniLM 0.67, oracle 0.70). Semantic
retrieval beats hand-designed geometry on that harness. We will release the protocol, retriever implementations, and per-query outputs so new selectors can be swapped in without rewriting the analysis.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
ClothCode: Generative Modeling for Garment State Estimation and Dynamics Prediction
Yingdong Ru, Lipeng Zhuang, Shiyu Fan et al. (+3 more)
We introduce ClothCode, a unified generative framework for garment state estimation and action-conditioned dynamics prediction in a shared discrete configuration space. A material-aligned tokenizer encodes garment configuration as residual vector-quantized patch
tokens, while task-specific Masked Generative
Transformers predict these
tokens through parallel confidence-guided refinement. Across three garment templates, ClothCode reduces mean displacement error by 26.6% for state estimation and 37.0% for dynamics prediction relative to the strongest baselines. Dynamics prediction requires only 16.7 ms per window, compared with 556.2 ms for
diffusion. Integrated into the same model-predictive controller, ClothCode improves real-world folding and flattening success from 56.7% to 78.9%. Further results and
videos are available at: https://anonymous.4open.science/r/ClothCode-DE32
Read the full abstract ▾
All authors Yingdong Ru, Lipeng Zhuang, Shiyu Fan, Zhuo He, Florent P. Audonnet, Gerardo Aragon Camarasa
Computer Vision · Cross-listed from cs.NI
Semantic Edge-Cloud Communication for Real-Time Urban Traffic Digital Twins
Murat Arda Onsu, Arman Dogru, Poonam Lohan et al. (+2 more)
Urban traffic systems are becoming increasingly complex, demanding real-time traffic perception, proactive collision prediction, and intelligent management to improve safety and reduce congestion. Although AI-driven traffic management has shown strong promise, its success relies on diverse, high-quality data spanning critical scenarios such as collisions, near-collisions, and traffic rule violations. Digital twins offer a powerful solution by enabling synchronized virtual replicas of physical traffic environments for real-time monitoring, collision prediction, traffic-flow optimization, and diverse realistic scenario generation. However, continuously synchronizing a high-fidelity urban traffic digital twin through edge surveillance cameras requires substantial edge-cloud communication, and transmitting raw
video streams leads to severe bandwidth and latency constraints. To overcome this challenge, this paper proposes a semantic communication-based edge-cloud framework for real-time urban traffic digital twin applications. By leveraging edge intelligence to extract and transmit only task-relevant semantic traffic information instead of full
video feeds, the proposed framework significantly reduces communication overhead while preserving the essential dynamics needed for accurate real-time digital twin reconstruction. Experimental results demonstrate more than 99% transmission savings and an average trajectory reconstruction error of only 0.12 m, confirming the proposed framework as an effective, scalable, and low-latency solution for AI-enabled urban traffic digital twin applications.
Read the full abstract ▾
All authors Murat Arda Onsu, Arman Dogru, Poonam Lohan, Burak Kantarci, Aisha Syed
Computer Vision · Cross-listed from cs.RO
Disentangling Task-Relevant Latent Dynamics for Reward Modeling
Dayou Li, Jionghao Wang, Peihao Wang et al. (+9 more)
Reward design remains a major bottleneck for reinforcement learning in robotic manipulation. Handcrafted dense rewards can compensate for the limited guidance provided by sparse rewards, but require substantial task-specific reward engineering. Learning rewards from
videos provides a scalable, data-driven alternative to manual reward design. However, existing
video-based rewards are often computed in entangled visual spaces where task-relevant state changes are confounded with extraneous visual variation. As a result, the learned reward may respond to visual distractors instead of actual task execution. To address this limitation, we propose DiTaR (Disentangling Task-Relevant Latent Dynamics for Reward Modeling), a framework for learning dense rewards from heterogeneous
videos. DiTaR learns a continuous task-relevant representation from multi-frame
video latents through an asymmetric two-stage procedure that suppresses task-irrelevant visual variation. In the task-relevant latent space, we train a history- and
language-conditioned rectified-flow model to capture task dynamics and derive a directional dense reward from temporal changes in dynamics compatibility. Together, these components enable reward modeling directly over task-relevant dynamics rather than generic visual variation. DiTaR substantially improves policy learning on MetaWorld-v3 and ManiSkill over
video-based reward baselines, and transfers to real-world offline RL through reward relabeling. Reward analyses further show that DiTaR provides more reliable rewards.
Read the full abstract ▾
All authors Dayou Li, Jionghao Wang, Peihao Wang, Jiuzhou Lei, Hao Wang, Jichen Chai, Hezhen Hu, Minghui Zheng, Zhangyang Wang, Xin Li, Wenping Wang, Zhiwen Fan