Computer Vision
Graph-Based Recognition of Simulated Train-Driver States From Facial and Upper-Body Keypoints
Olivia Nocentini, Marta Lagomarsino, Gokhan Solak et al. (+6 more)
Driver fatigue poses a significant challenge to railway safety, with traditional systems like the dead-man switch offering limited and basic alertness checks. This study presents a
vision-based monitoring system that relies solely on a single front-facing RGB camera and a graph neural network to classify simulated train-driver states into alert, not-alert, and an emergency class comprising acted emergency-like behaviours. To optimize input representations for the model, an ablation study was performed, comparing three feature configurations: skeletal-only, facial-only, and a combination of both. Experimental results show that combining facial and skeletal features yields the highest accuracy (81%) for the three-class model under the light condition, outperforming models that use only facial or skeletal features. Furthermore, the combination of facial and skeletal features achieves 99% accuracy in the alert/not alert classification in light condition. Additionally, we introduced a controlled RGB
video dataset containing alert, not alert, and acted emergency-like behaviours recorded under three illumination conditions. These contributions represent a step toward passive and non-contact train-driver state recognition based on facial and upper-body dynamics.
Read the full abstract ▾
All authors Olivia Nocentini, Marta Lagomarsino, Gokhan Solak, Younggeol Cho, Qiyi Tong, Sara Zeynalpour, Marta Lorenzini, Alessandro Ledda, Arash Ajoudani
Computer Vision
On Color Alignment in VAE Latent Spaces and Its Applications
Julian D. Santamaria, Kai Wang, Jesús Malo et al. (+2 more)
Variational autoencoders (VAEs) are a key part of modern text-to-image models, which generate images within their latent space. VAEs are known to disentangle the main factors of variation in the data, and color is known to be one of the most structured of these in natural images: decorrelating it yields one luminance axis and two opponent-color axes. Color should therefore be expected to emerge as a distinct factor in the VAE latent space. Yet how these latent spaces represent color remains largely unexplored. In this work, we show that the VAEs of text-to-image models share a color subspace aligned with brightness and opponent-colors. Through a linear approximation of the encoder and targeted latent steering, we find this subspace consistently across a broad range of VAEs, from SD1.5 to FLUX.2 and Z-Image. Building on this characterization, we propose three applications: ColorTuning, which achieves state-of-the-art in precise numerical color generation on the fine-grained CSS3/X11 system of GenColorBench, saturation control, to adjust the global chromatic intensity, and color transfer, to change the palette to match a reference. The code and models are publicly available at https://julian075.github.io/Color_Subspace/
Read the full abstract ▾
All authors Julian D. Santamaria, Kai Wang, Jesús Malo, Javier Vazquez-Corral, Alexandra Gómez-Villa
Computer Vision
A BEMD-Based Quaternion Filtering Approach Sharp-to-Soft Kernel CT Image Conversion
Mahmoud Nasr, Jan K. Argasinski, Krzysztof Brzostowski et al. (+1 more)
The quality of computed
tomography (CT) images is significantly affected by the selection of reconstruction kernels: sharp kernels improve spatial resolution but increase noise, whereas soft kernels diminish noise at the expense of edge clarity. This study presents an innovative enhancement framework utilising Bidimensional Empirical Mode Decomposition in conjunction with Quaternion Bilateral Filtering (BEMD–QBF) to convert sharp-kernel CT images into representations resembling soft-kernels, while maintaining critical anatomical structures. The technique disaggregates each image into intrinsic mode functions via BEMD and analyzes them inside a cohesive quaternion framework to attain efficient noise reduction and structural integrity. The proposed methodology is evaluated using several reconstruction kernels (B50, B46, B41, B36, B35, B31) and compared with recognised filtering strategies, including Non-Local Means, Anisotropic
Diffusion, Bilateral Filtering, and Quaternion Bilateral Filtering. Quantitative evaluations of the Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR) indicate that BEMD-QBF consistently attains superior structural fidelity and competitive noise reduction across all evaluated kernels. The results underscore the efficacy of the proposed strategy as a viable approach to enhancing post-reconstruction CT images, yielding superior image quality without requiring access to raw projection data.
Read the full abstract ▾
All authors Mahmoud Nasr, Jan K. Argasinski, Krzysztof Brzostowski, Adam Piorkowski
Computer Vision
Decomposition-Guided Curvelet Thresholding for Sharp-to-Soft CT Kernel Conversion
Mahmoud Nasr, Jan K. Argasinski, Krzysztof Brzostowski et al. (+1 more)
Image denoising is a crucial task in image processing, focused on improving image quality by minimizing noise while maintaining essential structural elements. This study presents a hybrid denoising framework that combines several decomposition techniques, including empirical mode decomposition (EMD), variational mode decomposition (VMD), multichannel EMD (MEMD), and bidimensional EMD (BEMD), with curvelet transform thresholding. Each decomposition mode undergoes processing through both soft and hard thresholding, and the denoised modes are combined to rebuild the final image. Comprehensive evaluations of standard CT image
datasets reconstructed with various kernels (B50, B46, B41, B36) reveal substantial enhancements in denoising efficacy. VMD consistently achieves the highest peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), signifying exceptional noise reduction and feature preservation. The study analyses the trade-offs between soft and hard thresholding: soft thresholding maintains intricate visual details, whilst harsh thresholding provides enhanced noise reduction. The suggested method surpasses traditional techniques in both reference and non-reference quality criteria, indicating its potential for broader application in medical imaging and future incorporation with adaptive thresholding
algorithms.
Read the full abstract ▾
All authors Mahmoud Nasr, Jan K. Argasinski, Krzysztof Brzostowski, Adam Piorkowski
Computer Vision
Crop Yield Prediction for Punjab, Pakistan: A Tree-Ensemble and Leaf-Health Prototype, and What Random Validation Hides
Amina Asif, Qurat Ul ain Asif, Noor Bakhat Asif et al. (+1 more)
Yield forecasts help planners and farmers decide on inputs, storage and imports, but small agricultural tables can make reported accuracy fail on a new season. We built a crop-yield prototype for Punjab, Pakistan that combines Random Forest, XGBoost, support vector regression and a Ridge-stacked ensemble with a MobileNetV2 leaf-health classifier, and deployed it as a web application with SHAP explanations. A random 80/20 split of a merged Kaggle-derived table (414 rows) gives the ensemble an R2 of 0.991. An audit showed that the table contains only 46 independent observations: a join with nine temperature records per year copied every crop-year nine times. Holding out whole years takes XGBoost on the same rows from R2 = 0.994 to -0.20. On deduplicated data, and on a longer FAOSTAT table (1990-2024, 70 observations), a per-crop linear trend (leave-one-year-out RMSE 0.29 Ton/Ha) beats every model not given the year (0.84-1.06). Neither pesticide use nor national temperature change explains the trend residuals. An apparent pesticide gain in the Kaggle table disappears on FAOSTAT, where the pesticide series is mostly imputed and the two releases disagree. An independent district-level wheat panel (36 districts, 13 seasons) shows that most variation in Punjab is spatial and that a district mean with a common trend matches the learned models. The leaf classifier reaches 99.87% accuracy on held-out PlantVillage images and recalls 96.4% of 336 unseen diseased leaves after near-duplicates were removed. However, no leaf image is paired with a yield record, so the health score used in the yield models had to be constructed and adds nothing. We report these negative findings together with the prototype.
Read the full abstract ▾
All authors Amina Asif, Qurat Ul ain Asif, Noor Bakhat Asif, Kasa-e-Fatima
Computer Vision
One Figure, Every Canvas: Editable Flowchart Relayout via Agentic Pipeline
Shih-Chen Tseng, Chih-Hsuan Chen, Ryan Yang et al. (+3 more)
Pipeline figures in ML papers must be repurposed across many canvases, including paper columns, 16:9 slides, portrait posters, 1:1 social teasers, 9:16 phone previews. Each format imposes a different aspect ratio on the same computational graph, where any silently broken connection misrepresents the method. We formulate aspect-ratio-adaptive flowchart relayout as a distinct task: given a raster flowchart and a target ratio, produce a structurally faithful,
hallucination-free, editable layout. Existing methods fail characteristically: image-to-image models stretch blocks and reject extreme ratios, text-to-image agentic systems hallucinate content, and parse-then-render systems mis-route edges. We propose an agentic pipeline factored into Parse, Style, and Layout stages, each pairing a main
agent with a critic that combines deterministic constraint checks with VLM visual feedback so connectivity is explicitly checked and prevented from being silently broken. Outputs are draw.io-editable mxGraph XML. On a curated
benchmark of 100 flowcharts at five aspect ratios, evaluated by Gemini 3.1 Pro and validated against human judgments, our method reaches 68.6% Content Fidelity versus 11.2-41.4% for prior work. Project page: https://onefigureeverycanvas.vercel.app/
Read the full abstract ▾
All authors Shih-Chen Tseng, Chih-Hsuan Chen, Ryan Yang, Hsi-An Chen, Chun-Wei Tuan Mu, Yu-Lun Liu
Computer Vision · Cross-listed from cs.RO
InterMimicGen: Scaling Humanoid Loco-Manipulation through Self-Evolving Motion Imitation
Yucheng Zhang, Sirui Xu, Jinhong Li et al. (+8 more)
Captured human-object interactions provide rich supervision for humanoid loco-manipulation, but they are sparse, heterogeneous, and not directly executable by
robots. We introduce InterMimicGen, a self-evolving motion-imitation framework in which
robot motion data and a tracking policy improve each other. First, we consolidate motion-captured human-object interaction
datasets and retarget them into humanoid
robot references while preserving whole-body coordination and dexterous hand-object relationships. This produces a large and diverse humanoid
robot reference collection for dexterous whole-body loco-manipulation. Second, we train a physics-based generalist tracker that executes these references in simulation on a humanoid with dexterous hands, covering a scale and diversity beyond prior humanoid tracking systems for loco-manipulation. Third, we close a data flywheel: each round makes small, task-preserving changes to where an interaction takes place and how the body performs it, fine-tunes the tracker on them, and keeps only the variants whose simulated execution completes the task, which seed the next round. With more iterations, these small edits compound into broader coverage around the sparse original demonstrations while preserving task semantics and motion quality. Experiments show contact-preserving retargeting across
robot configurations, broad tracking with a single generalist policy, executable motions that keep growing over augmentation rounds, and transfer to real
robots. InterMimicGen provides a unified path from heterogeneous human demonstrations to a continually expanding motion resource for humanoid
robot learning.
Read the full abstract ▾
All authors Yucheng Zhang, Sirui Xu, Jinhong Li, Liuyu Bian, Anatulya Nandi, Derek Zhang, Xiangchen Liu, Xueting Li, Umar Iqbal, Yu-Xiong Wang, Liang-Yan Gui
Computer Vision
S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation
Jeffrey Hu, Daniel Olmeda Reino and Ayush Tewari
Bidirectional
video diffusion models denoise entire
videos in parallel, yet when trained on effectively unlimited in-distribution data from procedural generators, continue to violate physical laws and simple symbolic rules. We introduce Serial-to-Parallel
Diffusion (S2PD), which performs autoregressive
diffusion at high noise before switching to parallel
diffusion at low noise. The autoregressive phase provides the serial computation needed to coordinate interdependent events and produce valid state transitions while the parallel phase jointly refines the entire
video and reduces sampling time relative to fully serial generation. We implement S2PD with two architectures: a pixel-space
diffusion transformer trained from scratch and a pretrained
video model adapted through LoRA fine-tuning with causal
attention. Across games, physical simulations, and real
video, S2PD follows rules more reliably than matched bidirectional baselines and generates
videos with greater temporal stability and sampling efficiency than other serial methods.
Read the full abstract ▾
Computer Vision
Omer Dahary, Etai Sella, Hadar Averbuch-Elor et al. (+2 more)
Multimodal Diffusion Transformers (MM-DiTs) jointly process visual and textual representations throughout generation. These models repeatedly update the text
tokens through
multimodal attention, forming dynamic contextual
tokens whose function is not well understood. In this work, we introduce a framework for reading this contextual space through natural-
language interrogation. We train a lightweight bottleneck network that maps intermediate contextual
tokens into the input space of a frozen Large
Language Model (LLM), allowing the LLM to answer questions about the emerging image directly from these hidden representations. Our reader reveals that contextual
tokens encode a rich, global representation of the emerging scene: generation-specific semantics, including attributes left underspecified by the
prompt, are accessible surprisingly early in denoising, while increasingly fine-grained details become readable over time. Remarkably, this information remains decodable even when the MM-DiT receives an empty
prompt, showing that contextual
tokens accumulate substantial image-specific information from the evolving visual representation itself. We further find that generations with more readable contextual representations tend to receive higher human-preference scores. Building on these observations, we introduce Contextual
Alignment, a training technique that explicitly reinforces the visual-semantic information encoded in the contextual
tokens, improving generation quality and distributional coverage. Together, our results establish contextual
tokens as both an interpretable view into the internal dynamics of MM-DiTs and an effective target for improving generative models.
Read the full abstract ▾
All authors Omer Dahary, Etai Sella, Hadar Averbuch-Elor, Daniel Cohen-Or, Or Patashnik
Computer Vision
Anatomy-aware Fine-grained Multimodal Fusion for Laryngopharyngeal Cancer T-Staging Prediction Using CT and Radiology Report
Xingyue Zhao, Yanzhou Su, Fang Zhang et al. (+12 more)
Accurate T-staging is crucial for guiding personalized treatment strategies for laryngopharyngeal cancer. However, current clinical practice relies on invasive biopsy procedures, whereas CT-based staging remains challenging due to the complex patterns of tumor invasion. Recent computer-aided approaches face two key challenges: 1) Structural relationship modeling: existing methods underrepresent anatomically structured patterns of tumor invasion, as they either process whole CT volumes without tumor-specific anatomical constraints or rely on labor-intensive tumor segmentation. 2) Fine-grained cross-modal
alignment: while radiology reports contain organ-specific invasion details, current methods that apply global feature fusion struggle to accurately align individual anatomical structures with their corresponding textual descriptions. To address these issues, we propose an anatomy-aware
multimodal framework that integrates organ-level CT context and radiology reports into a unified representation for laryngopharyngeal T-staging. The framework first constructs an Anatomy-Structured Organ Graph (AOG) that captures invasion patterns between primary sites and surrounding organs, then performs Organ-Anchored Cross-Modal
Alignment (OCA) so that each organ node aggregates textual evidence from the radiology report, and finally refines this graph representation by injecting organ-specific invasion cues extracted from the report via Report-Enhanced Graph-Refinement (REG), yielding a
multimodal organ graph that combines spatial and textual evidence. Extensive experiments demonstrate that the proposed framework achieves superior performance in T-staging of laryngopharyngeal cancer.
Read the full abstract ▾
All authors Xingyue Zhao, Yanzhou Su, Fang Zhang, Zhanghexuan Ji, Yirui Wang, Dazhou Guo, Sibo Ju, Yuehua Cheng, Yuzhen Chen, Ming Feng, Le Lu, Tsung-Ying Ho, Jian Wang, Dakai Jin, Na Shen
Computer Vision
UniSlider: Perceptually Uniform Sliders for Continuous Image Editing
David Serrano-Lozano, Duygu Ceylan, Yannick Hold-Geoffroy et al. (+3 more)
Sliders provide an intuitive interface for continuous image editing. In current generative approaches, however, the slider is simply a rescaling of the method's strength parameter, such as an adapter coefficient, a
prompt weight, or an interpolation factor. This strength relates poorly to perceptual change. The image can partially revert as the slider moves, long stretches of the range produce no visible difference, and short intervals transform the image abruptly. Remapping the strength could fix this uneven pace, but only if the trajectory is monotone, which current methods do not enforce. We therefore distinguish the slider from the strength, and require perceptual distance from the input to grow linearly with the slider value. We introduce UniSlider, a lightweight LoRA trained on a few-step editing backbone so that its strength approximates this ideal slider. Few-step sampling lets us impose this objective in pixel space without intermediate ground truth, and the backbone's output is preserved at full strength. However, a low-rank adapter cannot make the strength fully uniform. Our slider is thus an inference-time remapping of the strength, obtained by adaptive sampling. Since training optmizes to make the trajectory monotone, this remapping closes the remaining gap without extra training or parameters. On a new
benchmark of 300 continuous edits evaluating uniformity, monotonicity, edit fidelity, and identity preservation, UniSlider outperforms all prior methods and is preferred in a user study.
Read the full abstract ▾
All authors David Serrano-Lozano, Duygu Ceylan, Yannick Hold-Geoffroy, Iliyan Georgiev, Javier Vazquez-Corral, Anna Frühstück
Computer Vision · Cross-listed from cs.CL
PlotGround: Grounding Plot Digitization in Real Scientific Figures and Their Source Data
Yaohui Zhang, Binxu Li, Haoyi Duan et al. (+7 more)
Scientific figures often encode quantitative results that are not readily available in machine-readable form, making accurate plot digitization important for verifying and reusing published findings. Yet it remains unclear how accurately current models recover plotted values from real scientific figures, as existing
benchmarks rely largely on synthetic charts or cover only a limited range of chart types. We introduce PlotGround, an automated pipeline for building plot digitization
benchmarks from real scientific figures and their author-released source data. PlotGround maps figures to source tables, identifies reconstructable panels, and generates quantitative questions with source-grounded reference values. We use PlotGround to construct PlotGround-1k, a human-verified
benchmark of 1,119 questions from 1,066 bioRxiv preprints. Across sixteen
multimodal models, the best reaches 87.5% accuracy at a $\pm 5\%$ relative-error tolerance. Tightening the tolerance to $\pm 2\%$ lowers every model's accuracy by 11-24 percentage points, revealing a gap between approximate visual reading and precise quantitative recovery. PlotGround's paired figure-source structure lets us compare how accurately the same values are recovered from figures and from source tables. Providing source tables instead of figures raises a coding
agent's accuracy from 90.0% to 97.4% while cutting cost by 72%.
Read the full abstract ▾
All authors Yaohui Zhang, Binxu Li, Haoyi Duan, Jiacheng Miao, Yixin Wang, Xinran Du, Chenyue Li, Shilong Liu, Kevin Wu, James Zou
Computer Vision
TAPDreamer: Transferable Adversarial Patches for World Action Models
Xuanyu Lu, Fengqing Jiang, Kaiyuan Zheng et al. (+8 more)
World models learn to predict how their environment will evolve, making them an important foundation for general-purpose robotic control. Yet world action models depend on camera inputs whose manipulation can corrupt the visual representations used across tasks and action policies. Existing attacks on these models optimize against the victim's actions or predicted futures and therefore require access to target-model outputs. In this paper, we propose an attack, TAPDreamer, against world action models that instead uses a public encoder alone to construct a fixed local perturbation that transfers across tasks and action architectures. TAPDreamer requires no target-policy queries. Our key insight is that interactions between patch-induced changes in
attention weights and value vectors broadcast a nearly identical representation shift far beyond the patch footprint, and this shift remains stable across task observations. Guided by this insight, TAPDreamer uses six frames from one source task to maximize the global L1 distance between clean and patched encoder representations. In closed-loop evaluation, one frozen patch per
benchmark, covering about 6.5% of the input, reduces FastWAM's success rate from 97.7% to 0.0% across 40 LIBERO tasks and from 90.86% to 0.0% across 50 RoboTwin tasks; matched random patches retain 81.5% and 79.2% success. The same patches reduce success to 1.45% and 1.00% on two DreamWAM configurations and to 10.60% on Motus. These results show that protecting downstream action generation alone is insufficient: defenses for world action models must also secure shared visual encoders against persistent local perturbations.
Read the full abstract ▾
All authors Xuanyu Lu, Fengqing Jiang, Kaiyuan Zheng, Yichen Feng, Yaorui Ding, Yuetai Li, Zhen Xiang, Bhaskar Ramasubramanian, Basel Alomair, Luyao Niu, Radha Poovendran
Computer Vision
Less Context, Better Geometry: Masked Geometric Encoder for Robust 3D Foundation Models
Zhimin Shao, Xijun Liu, Zhaoliang Zhang et al. (+4 more)
Recent progress in 3D foundation models has enabled rapid 3D reconstruction and camera calibration by leveraging learned 3D priors from vast amount of spatial data. However, the all-to-all global
attention design leads to quadratic complexity and limits long-sequence inference; unconstrained cross-view interactions also can propagate unreliable evidence from occluded or visually similar but geometrically distant views. In this paper, We introduce a Masked
Geometric Encoder (MGE), which promotes the learning of robust
geometric representations under incomplete cross-view context. During training, MGE strategically drops frame
tokens from global
attention and distills from a pretrained full-context teacher model. This allows the model to learn an intrinsically richer per-frame representation while providing sufficient intermediate supervision to avoid performance degradation. Through extensive experiments, we show that MGE leads to much stronger performance under occlusion and doppelganger views while retaining high performance on standard
benchmarks. Such a richer frame representation also leads to more effective
token reduction during inference. To this end, we develop a novel Anchor-Guided Adaptive
token merging technique that preserves representative anchor frames while jointly merging redundant
tokens from the remaining views. Compared to other efficient inference approaches, we can achieve inference speedup while consistently maintaining higher reconstruction quality, particularly in limited-view settings.
Read the full abstract ▾
All authors Zhimin Shao, Xijun Liu, Zhaoliang Zhang, Yutao Tang, Abhay Yadav, Rama Chellappa, Cheng Peng
Computer Vision
MC-Sparse: Deconstructing and Closing the Dense-Sparse Attention Gap in Diffusion Transformers
Jiarui Chen, Zeqiang Lai, Jiangshan Wang et al. (+5 more)
Sparse
attention is a primary approach to reducing the latency of
diffusion transformers in long-sequence generation tasks, such as
video and high-resolution 3D asset generation. However, existing methods can degrade generation quality and fidelity at high sparsity levels. Through controlled oracle comparisons, we trace this degradation to three sources: constraints imposed by
token grouping, inaccurate interaction selection, and the
attention contributions lost when
tokens are discarded. Guided by this analysis, we propose Meta-Cached Sparse
Attention (MC-Sparse), a training-free framework that selects individual key-value (KV)
tokens while organizing similar queries into tile-aligned groups for efficient GPU execution. MC-Sparse caches metadata comprising query groups, KV indices selected using exact
attention probabilities, and residuals between dense and sparse
attention outputs, and reuses them across subsequent denoising steps. Across
video and 3D generation models, MC-Sparse achieves higher fidelity to dense-
attention outputs and larger denoising speedups than existing sparse-
attention baselines, without visible quality degradation. Relative to dense
attention, it delivers a $1.80\times$ denoising speedup on Minimax-H3-Base and a $2.32\times$ speedup on 3D asset generation, both with negligible quality loss.
Read the full abstract ▾
All authors Jiarui Chen, Zeqiang Lai, Jiangshan Wang, Ziheng Ouyang, Ye Huang, Xiangyu Yue, Cewu Lu, Chunchao Guo
Computer Vision
Extending Dynamic World Surface Water Mapping to Sentinel-1 with AlphaEarth Embeddings
Rohit Mukherjee, Frederick Policelli, Beth Tellman et al. (+4 more)
Dynamic World (DW) maps land use and land cover globally at 10 m from Sentinel-2 (S2) imagery, but only for cloud-free observations, which limits where and when surface water can be mapped. We use the DW water class as weak supervision for a Sentinel-1 (S1) synthetic aperture radar (SAR) model so that DW-like water maps can be produced for every S1 acquisition. Google's AlphaEarth Foundations (AEF) annual
embedding supplies spatial context, while S1 backscatter supplies the acquisition-time observation. On 53 globally distributed scenes with independent annotations of 3 m PlanetScope imagery acquired within 48 h of the S1 overpass, the S1-only model already reaches a pooled water intersection over union (IoU) of 0.77, comparable to 0.75 for the operational OPERA DSWx-S1 product, and adding AEF raises it to 0.85. The fused model improves on the S1-only model on 44 of 53 scenes and exceeds OPERA on 48, and on the independent S1S2-Water
benchmark it reaches 0.94, compared with 0.87 for OPERA. Optical land-cover products can thus provide scalable training labels for SAR surface water mapping.
Read the full abstract ▾
All authors Rohit Mukherjee, Frederick Policelli, Beth Tellman, TC Chakraborty, Jonathan Giezendanner, Jonathan A. Sullivan, Ning Sun
Computer Vision
GS-Pool: Object-Level Change Detection in 3D Gaussian Splatting
Boaz Keren-Gil, James Gain and Patrick Marais
Factories, museums and surveyors photograph the same space months apart and need to know which objects changed. When each visit is reconstructed with 3D Gaussian Splatting (3DGS), a direct comparison of the two reconstructions does not answer this. Training is
stochastic, so two reconstructions of an unchanged space never coincide, and the second visit is often a quick re-scan with far fewer photographs. We propose GS-Pool, which takes two independently reconstructed Gaussian fields of the same space and returns the changed objects in each, together with their masks. SAM2 masks of each visit's photographs are lifted onto the Gaussians that render them and merged into an object pool, so every decision is taken once per object in 3D. We introduce a photographic carrier, the 3DGS training loss of each input reconstruction against the other visit's photographs, backpropagated to the Gaussians that rendered each pixel. We combine it with GS-Diff's geometry and colour terms and our distilled DINOv3 features. This evidence is compared with that of the objects present in both visits, which sets a change threshold for each scene. On PASLCD, GS-Pool reaches mIoU/F1 scores of 0.751/0.846 against 0.644/0.758 for GS-Diff, the strongest prior method, a gain of 17%/12%. Its mIoU is also 36%, 40% and 57% above that of O-SCD, PlenoCI and MV-3DCD, and it reaches 0.855 mIoU on CL-Splats, 33% above MV-3DCD. Each changed object is returned as a set of Gaussians with the evidence behind its decision, which an inspector can review in 3D.
Read the full abstract ▾
Computer Vision
ChronoWorld: Camera-Controlled Consistent 4D World Generation via Spatiotemporal Cues and Geometric Reflections
Xiaoyu Zhou, Dingwei Xian, Zhenyu Wang et al. (+3 more)
While existing camera-controllable
video generation models can produce visually compelling sequences, preserving intrinsic 4D spatiotemporal coherence remains challenging. To address this limitation, we propose ChronoWorld, an "Observation–State–Reflection" framework that leverages spatiotemporal causal cues and reconstruction priors to generate globally consistent, free-view 4D scenes. Given a context
video, we introduce a Spatiotemporal Epipolar Causal
Attention mechanism that enforces multi-view epipolar constraints and temporal causality throughout the generation process. In addition, we develop a reconstruction-driven
geometric reflection pipeline with a 4D
retrieval strategy to enable dynamic self-assessment and correction of generated outputs, improving consistency and accuracy. Extensive experiments show that ChronoWorld achieves state-of-the-art performance in spatiotemporally consistent, cinematic-quality 4D scene generation, with strong generalization and high-fidelity geometry across diverse scenarios.
Read the full abstract ▾
All authors Xiaoyu Zhou, Dingwei Xian, Zhenyu Wang, Yajiao Xiong, Yongtao Wang, Ming-Hsuan Yang
Computer Vision · Cross-listed from cs.LG
Detecting Nighttime Anomalies from NASA Black Marble Using a Generalized Spatio-Temporally Robust Framework of Machine Leaning Ensembles
Srija Chakraborty
Nighttime lights from NASA's Black Marble product suite capture thermal and light emission signals from anomalous events including fires, volcanic eruptions, and gas flaring. Existing detection approaches rely primarily on thermal bands, limiting sensitivity to weaker signals. We propose a novel machine learning framework that jointly models Black Marble M-band and Day/Night Band (DNB) signals to derive a generalized, spatio-temporally robust ensemble of anomaly detectors. The framework iteratively builds detectors that scale across regions, seasons, anomaly classes, and extends over land and ocean. Detection sets at varying confidence levels are derived based on relevant bands and detector agreement. The approach improves true detection rate while reducing spurious detections and results demonstrate strong generalizability with applications in natural hazard monitoring and energy extraction.
Read the full abstract ▾
Computer Vision
VideoTapestry: Query-Adaptive Memory Refinement for Multi-Agent Long-Video Understanding
Yucheng Liu, Yufei Yin, Mingxiao Feng et al. (+3 more)
Long-
video understanding places substantial demands on
memory, as answering questions often requires retrieving information distributed across extended temporal spans. Existing approaches broadly follow two paradigms: query-driven exploration, which is sensitive to localization errors, and query-independent
memory construction, which may omit question-specific details. We introduce VideoTapestry, a training-free multi-
agent framework that adapts a preconstructed hierarchical
video memory through coarse-to-fine, query-driven refinement. The preconstructed
memory organizes
video content into three levels, capturing global narrative context, event-level temporal structure, and fine-grained relational evidence, respectively. To support coarse-to-fine localization and observation, we assign a specialized
agent to each level, keeping
retrieval and refinement within a scale-specific context. Guided by the query, these
agents revisit relevant
video regions and enrich layer-wise
memories with targeted
multimodal observations. Their refinements are assembled according to the original hierarchy into a composite query-adaptive
memory, preserving global context in a compact form while retaining fine-grained evidence along query-relevant branches for final
reasoning. Compared with direct GPT-5.5 inference, VideoTapestry achieves absolute accuracy gains of 17.2%, 14.9%, 9.8%, and 7.0% on LVBench, LongVideoBench (Long),
Video-MME (Long), and EgoSchema, respectively, achieving the state-of-the-art results among all competitors.
Read the full abstract ▾
All authors Yucheng Liu, Yufei Yin, Mingxiao Feng, Jiajun Deng, Wengang Zhou, Houqiang Li
Computer Vision
Cross-dataset harmonization for robust endoscopic image analysis
Romil Imtiaz and Dimitris K. Iakovidis
A significant problem in endoscopic image analysis is that the machine learning (ML) models used for this purpose usually underperform when applied on images acquired from endoscopes that are different from those used to acquire the images of their training set. The main difference of the images originating from different endoscopes is their color distributions, which depend both on the image sensors and the light sources used. Although previous studies have highlighted this challenge, to the best of our knowledge it has not been previously explicitly tackled. This study focuses on this problem and proposes very simple but impactful method. It implements a reference-based image harmonization that reduces global appearance differences between endoscopic
datasets. Specifically, it extracts global color statistics from a chosen reference
dataset in the CIE-Lab color space and applies a statistical channel-wise transformation to map each target image toward the appearance of the images of the reference
dataset. The method is evaluated in the context of polyp detection in both flexible colonoscopy and capsule endoscopy
datasets using a
dataset-level cross validation protocol. The results indicate that the proposed harmonization consistently improves cross-
dataset performance up to 30.7%, outperforming relevant baseline and state-of-the-art methods. The results indicate that a substantial part of the generalization gap is driven by low-level appearance variation that can be mitigated without retraining.
Read the full abstract ▾
Computer Vision
Talk Like You: Imitating How You Speak in Real-Time Talking Head Generation
Baiqin Wang, Zhixing Ding, Jijie Li et al. (+3 more)
In daily life, each person exhibits unique speaking habits, leading to subtle yet consistent lip-shape variations even when pronouncing the same word. Although recent talking head generation methods have achieved impressive visual fidelity and lip synchronization, they largely overlook user-specific customization, especially the motion patterns that characterize individual speaking habits. These habits are difficult to model and capture, as their motion patterns are highly fine-grained and often similar across individuals. As a result, many approaches produce overly uniform facial motions and fail to capture diverse, person-specific articulation patterns. To address this, we propose TalkLikeYou, an efficient framework that imitates how a target person speaks in talking head generation. Our method models habit in motion-space and achieves real-time performance through Flow Matching with only one sampling step during inference. We further adopt a two-stage imitation learning strategy to capture subtle distinctions between habits, allowing users to specify a target habit through either a preset style from the
dataset or a reference
video. In addition, we introduce a new metric PLAD that projects mouth motions onto representative articulation axes to evaluate imitation accuracy and generation diversity. Extensive experiments demonstrate that TalkLikeYou generates high-quality talking heads in real-time and significantly improves speaking habit imitation compared with prior methods. The code is available at: https://github.com/BQ-Wang0511/TalkLikeYou
Read the full abstract ▾
All authors Baiqin Wang, Zhixing Ding, Jijie Li, Jiankuo Zhao, Zhen Lei, Xiangyu Zhu
Computer Vision · Cross-listed from cs.RO
AffordCraft: Scalable Construction of Task-Ready Simulation Assets from Single Images
Haoyun Yang, Xueyang Zhou, Ziyi Xie et al. (+1 more)
Robot learning in simulation depends on the objects the simulator offers. Many tasks need objects with separate parts, joints that allow the required motion, and physical properties that remain valid under contact. Existing methods recover this structure anew for every image: generative models predict parts and joints that mostly fail to settle or move in simulation, and general-purpose
agents need a long session of model calls for each photograph. AffordCraft builds such an asset from a single RGB image and a task instruction by
retrieval instead of generation: it locates the object and the part to operate, selects a matching entry from a library of articulated assets, and fits it to the image while keeping its parts and joints intact. Without any box or mask marking the object, AffordCraft produces a physically valid asset for 1,703 of 2,000 photographs from 31 categories. Five generative methods pass on at most 45% of the same photographs and, at the median, need 10 to 78 times our GPU time per valid asset. On 50 cluttered images, 162 of 237 annotated objects pass the same physical test after automatic detection. Growing the library from 141 to 11,372 entries needs no change to the method and raises category coverage from 46% to 100% and the share of selections with the requested label from 18% to 51%. We also build manipulation tasks from the constructed assets, both with single objects and in composed scenes; policies trained on scripted demonstrations complete both kinds of tasks from initial states unseen in training.
Read the full abstract ▾
All authors Haoyun Yang, Xueyang Zhou, Ziyi Xie, Yongchao Chen
Computer Vision
RealtimeWAM: One-Step Asynchronous World Action Models
Chengtao Lv, Jinyang Du, Shuyi Feng et al. (+7 more)
World Action Models (WAMs) incorporate visual representations from
video generation backbones to guide action prediction. Recent efficient WAMs adopt Mixture-of-
Transformers (MoT) architectures and compute
video representations once for reuse by the action expert. However, intra-expert iteration (\ie, multi-step action denoising) and inter-expert waiting (\ie, sequential execution of the
video and action experts) still limit inference efficiency. To this end, we present RealtimeWAM, an extremely efficient WAM variant with one-step action generation and asynchronous inference, addressing these two bottlenecks. To reduce intra-expert iteration, we propose Teacher-Anchored Consistency
Distillation (TACD) to address a local-global error gap: low local consistency error alone does not guarantee accurate final actions. TACD supplements local consistency with explicit supervision from the frozen teacher's multi-step rollout endpoint, enabling accurate one-step action generation. Additionally, we propose Cross-Expert Wavefront Pipelining (CEWP) to eliminate unnecessary expert-level waiting. It overlaps the two experts through block-wise sharing of the
video KV cache, synchronizing only immediately before the corresponding action
attention consumes it. Extensive experiments across diverse
benchmarks (\eg, LIBERO, LIBERO-Plus and RoboTwin) and model variants (\eg, Fast-WAM and Faster-WAM) demonstrate the superiority of RealtimeWAM. Notably, RealtimeWAM maintains near-lossless performance (\ie, $<1\%$ drop) across these
benchmarks while delivering significant end-to-end speedup (\eg, $\sim25\times$ on H100). Our code and checkpoints are available via this \href{https://github.com/ModelTC/LightX2V/tree/main/examples/realtimewam}{link}.
Read the full abstract ▾
All authors Chengtao Lv, Jinyang Du, Shuyi Feng, Yang Yong, Shiqiao Gu, Shunzi Yang, Ruihao Gong, Shen Ren, Tianwei Zhang, Wenya Wang
Computer Vision
Video Encoders Built on Image Representations
Jusheng Zhang, Wenhao Wang, Longqi Cai et al. (+3 more)
The design of a
video encoder determines when frames begin to interact and which frame-specific visual evidence remains accessible to the
language model. Native
video pathways couple neighboring frames during visual encoding, whereas image pathways preserve independently computed frame representations but incur a much larger visual-
token cost when all image
tokens are forwarded. We ask a basic question: whether a compact
video encoder can instead be built on image representations. To answer this question, we separate three operations that are often coupled: per-frame representation, cross-frame
token allocation, and temporal interaction. A frozen image encoder first produces frame-specific candidates. A question-aware selector then allocates a fixed
token budget across frames using relevance, diversity, and cross-frame correspondence, after which a lightweight learned refiner reads neighboring-frame context and writes residual updates only to the retained anchors. This preserves source positions and keeps the visual output at the fixed budget. Across 13
benchmarks and three
vision-
language backbones, the resulting pathway matches full-image aggregate performance while using only about 28%-35% of its visual
tokens. Specifically, on Qwen3-VL-8B, it achieves a 13-
benchmark macro-average of 62.75 with 1,535 visual
tokens, compared with 62.58 for the full Image pathway at 4,424
tokens and 59.49 for native Conv3D at 2,212
tokens. On Qwen3-VL-32B, it reaches a 13-
benchmark macro-average of 66.28, compared with 66.09 for Image, while providing a 2.16x end-to-end speedup. These results show that compact
video encoding does not require early temporal mixing: frame-specific evidence can be preserved first, allocated jointly, and temporally contextualized after selection.
Read the full abstract ▾
All authors Jusheng Zhang, Wenhao Wang, Longqi Cai, Liangzhe Yuan, Yuxiao Wang, Ming-Hsuan Yang
Computer Vision
Lens3D: Target-Conditioned Visual Foveation for Fine-Grained 3D Understanding
Junming Huang, Zini Chen, Shuaiying Hou et al. (+3 more)
Existing 3D large
language models often overlook fine-grained attributes and less visually salient objects and parts, even when relevant evidence is present in scene
videos. We introduce Lens3D to improve fine-grained object understanding through external visual assistance and knowledge transfer. Its LensUnd pipeline adopts 3D localization to select informative, complementary views for an external 2D
vision-
language model, supporting fine-grained object captioning, small-object grounding, and fine-grained object question answering. LensDistill transfers the resulting fine-grained knowledge to 3D LLMs through detailed caption supervision, enabling captioning from native inputs without external VLM calls. We also construct LensBench, a held-out evaluation set of 2,068 objects with three silver-standard reference descriptions per object. Experiments with
Video-3D LLM and 3DRS demonstrate that LensDistill substantially improves fine-grained object captioning while preserving existing grounding and scene-level QA performance. These results establish the feasibility of transferring externally acquired fine-grained knowledge into native 3D LLMs.
Read the full abstract ▾
All authors Junming Huang, Zini Chen, Shuaiying Hou, Chi Wang, Qiang Dai, Weiwei Xu
Computer Vision
Multitask Conditional Generative Adversarial Network Enables Automatic Whole Knee Cartilage and Menisci Segmentation and Reliable $T_{1\rho}$ and $T_2$ Quantification Without High-Resolution Morphological Images
Ahmed Tahseen Minhaz, Richard Lartey, Zhiyuan Zhang et al. (+8 more)
Early osteoarthritis detection through quantitative MRI (qMRI) requires accurate cartilage and meniscus segmentation, traditionally necessitating time-consuming, costly 3D high-resolution Double Echo Steady-State (DESS) MRI scans. This study developed a multi-task conditional generative
adversarial network (MT-cGAN) to simultaneously synthesize DESS-like images and segment tissues directly from qMRI echo images. This retrospective study evaluated 508 knee MRI volumes from 361 subjects (mean age: $40.4 \pm 12.2$ years; 179 female) across three cohorts. Ground truth segmentation masks were generated from DESS images using a pretrained model with manual correction, and $T_{1\rho}$ and $T_2$ maps were computed from magnetization-prepared angle-modulated partitioned $k$-space spoiled gradient echo snapshots (MAPSS) echo images. MT-cGAN was trained to jointly synthesize DESS-like images and segment cartilage and meniscus directly from echo images. Model performance was evaluated using Dice score for segmentation accuracy and coefficient of variation (CV) for $T_{1\rho}$ and $T_2$ quantification. MT-cGAN achieved the highest segmentation performance, mean Dice score 0.84 (range: 0.80–0.86) across all cartilage and meniscus compartments and significantly outperformed the state-of-the-art conditional GAN model with transfer learning (mean Dice, 0.82; $p < 0.001$, Wilcoxon signed-rank test). For relaxometry quantification, MT-cGAN demonstrated the highest consistency with the reference DESS protocol, yielding the lowest CV ($T_{1\rho}$: 1.84%, $T_2$: 1.81%). The proposed MT-cGAN accurately segmented cartilage and menisci while providing reliable $T_{1\rho}$ and $T_2$ quantification directly from echo images. By eliminating the need for separate morphological DESS scans, this workflow reduces required scan times to facilitate the clinical translation of qMRI.
Read the full abstract ▾
All authors Ahmed Tahseen Minhaz, Richard Lartey, Zhiyuan Zhang, Jeehun Kim, Kunio Nakamura, Mingrui Yang, Jiasen Zhang, Weihong Guo, Naveen Subhas, Carl S. Winalski, Xiaojuan Li
Computer Vision · Cross-listed from cs.RO
SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
Xiaodong Wang, Tianle Li, Chuanxin Song et al. (+4 more)
Action-conditioned
robot world models must respond precisely to
robot trajectories while preserving realistic visual dynamics, yet learning both from heterogeneous
robot videos remains challenging. Simulation offers structured motion supervision, but appearance differences hinder direct transfer, and inaccurate simulation predictions can misguide real-
video generation. We present SimForcing, a simulation-guided framework that uses simulation both as a source of transferable motion knowledge and as a controllable reference for prediction. First, we transfer motion knowledge from a simulation teacher through latent-motion
distillation, aligning temporal changes in latent space to internalize motion priors while mitigating the influence of appearance differences. Second, we introduce multi-block simulation conditioning with condition dropout to exploit predicted simulation trajectories without relying excessively on their accuracy. Our simulation-conditioning classifier-free guidance scheme unifies these two ideas by balancing predictions based on internalized motion knowledge with those additionally guided by simulation latents. The jointly trained student generates both simulation conditions and real-domain
videos, requiring no additional world model at inference. On Bridge, SimForcing achieves the best PSNR, SSIM, LPIPS, and FVD among the compared methods without external embodied pretraining. Evaluation on InternData-A1 further supports its applicability across
robot datasets. Moreover, using our trained world model to initialize a
vision-
language-action model improves LIBERO success, suggesting its utility for downstream policy learning. \url{https://github.com/Wang-Xiaodong1899/SimForcing}
Read the full abstract ▾
All authors Xiaodong Wang, Tianle Li, Chuanxin Song, Junliang Xie, Zhanmi Zhong, Suiying Wu, Peixi Peng
Computer Vision
Analysis of SWIR Imaging Detection Performance Under Adverse Environmental Conditions for Autonomous Driving Systems
Rohan Mehra, Alexandre Riffard, Yannis Loumouamou et al. (+1 more)
Short-wave infrared (SWIR) imaging has emerged as a promising modality for autonomous driving, yet its practical benefits over RGB remain poorly characterized across diverse conditions. This paper presents a systematic comparative study of paired RGB and SWIR object detection on the RASMD
dataset, covering four weather conditions and two real-time detection architectures, with various fine-tunings evaluated against a unified ground truth. Overall, RGB demonstrates comparable or superior performance in most scenarios, while RF-DETR exhibits greater robustness across varying conditions. Beyond aggregate metrics, we propose a sensor-dominance mining framework that combines multi-model agreement with targeted manual inspection to identify scenarios where one sensing modality provides more reliable detections using largely unannotated paired data. This analysis reveals that SWIR offers clear advantages in four safety-critical situations, including windshield glare, water droplets on the windshield, low-contrast object visibility, and long-range vehicle detection. The findings suggest that SWIR should be viewed as a complementary modality that enhances perception in rare but challenging conditions. The
datasets will be available upon request, and all code and trained model weights are publicly released at https://github.com/comsee-research/swir-adverse-env-analysis.
Read the full abstract ▾
All authors Rohan Mehra, Alexandre Riffard, Yannis Loumouamou, Mathieu Labussière
Computer Vision
VGGT-Bridge: Beyond Sequential Pose Graphs via Coarse-Stride Skip Edges
Sungjae Choi, Hanna Bae, Sunghyun Baek et al. (+1 more)
Feed-forward visual geometry
transformers such as VGGT reconstruct dense 3D structure from images in a single forward pass, simplifying multi-view 3D reconstruction. However, their quadratic
attention complexity makes them difficult to scale to long sequences with thousands of frames. Chunk-and-align frameworks address this by splitting a long sequence into overlapping chunks and stitching their local reconstructions into a pose graph. Yet existing methods connect only sequentially adjacent chunks, so small per-frame errors accumulate along the chain into large-scale drift. To move beyond sequential edges, we propose VGGT-Bridge, which adds long-range skip edges that directly constrain non-adjacent chunks without retraining. By running VGGT on sparsely sampled coarse chunks, each coarse chunk bridges distant fine chunks into a single direct constraint. We further turn VGGT's first-frame scale bias into a drift correction by feeding selected coarse chunks in reverse, and a loop-aware policy keeps this reversal compatible with existing loop closures. VGGT-Bridge reduces ATE by 28.3% on KITTI Odometry, 18.8% on Virtual KITTI, and 10.0% on Waymo Open over the SwiftVGGT baseline, achieving the best performance among all chunk-and-align methods.
Read the full abstract ▾
All authors Sungjae Choi, Hanna Bae, Sunghyun Baek, Junmo Kim