Computer Vision
Towards automatic digital garment assembly
Alejandro Rodriguez
Creating digital garments from existing two-dimensional patterns remains a time-consuming process. Although patterns are widely available in digital form, their assembly in garment CAD software involves manual definition of sewing operations and arrangement of the pattern panels for drape simulation. We present two learning-based approaches that automate both tasks using the 2D vector pattern as their sole input. Our automatic sewing model operates at the segment level, leveraging a
Transformer encoder with multiple prediction heads. Our novel architecture supports many-to-many, ordered segment correspondences, as well as hems, overlapping seams needed for multilayer garments, and construction properties such as seam allowances and stitch types, required for the finished virtual garment but neglected by previous methods. A complementary panel-level model predicts body-region assignments, layers, and placement coordinates to initialize the drape simulation. We evaluate both models on a curated
dataset of 4,035 professionally assembled garment instances. The sewing model exactly recovers all seam and hem
topologies of 63.3% of test garments, while the arrangement model meets the specified placement criteria for all panels in 66.2% of test garments. We also integrate both methods into garment CAD software, demonstrating automatic assembly of successfully predicted garments.
Read the full abstract ▾
Computer Vision
Efficient Fréchet Distributional Decoder Alignment for Latent Generative Models
Xianghong Fang, Jiajun Zhu, Esma Aimeur et al. (+2 more)
Fréchet Distributional (FD) objectives provide an effective distribution-level signal for post-training generative models. For latent generative models, however, existing FD-based post-training methods optimize only the latent generator while keeping the decoder frozen: the decoder is trained on reconstruction latents obtained from real images but deployed on generator-produced latents at generation time. When these latent distributions differ, this decoder train-generation mismatch means that the decoder is deployed on inputs drawn from a distribution it was not directly trained on. We introduce Fréchet Distributional Decoder
Alignment (FDDA), an efficient post-training method that freezes the latent generator and directly optimizes the decoder on generation-time latents using an FD objective. A single epoch of FDDA consistently improves gFID and gFDr$^6$ across diverse flow-based and autoregressive latent generative models. Across all 11 evaluated models, FDDA reduces gFDr$^6$ by 34.0-59.6% from the original pretrained models and provides a further 10.2-57.2% reduction after generator-side FD post-training. FDDA also reduces total wall-clock time by factors of $3.6\times$-$328.7\times$ relative to generator-side FD post-training across these models. These results show that generator-side post-training alone does not exhaust the benefits of FD optimization, and that further gains can be obtained by adapting the decoder at substantially lower cost.
Read the full abstract ▾
All authors Xianghong Fang, Jiajun Zhu, Esma Aimeur, Dehan Kong, Tim G. J. Rudner
Computer Vision
TableVision: Recovering Verified Structure from Historical Tables
Todd Y. Zhou and James M. Snyder Jr
Gaps, skips, and mistakes are the normal state of a historical register: rows go missing, tables cross page breaks, columns slip under wrong names, and column sums disagree with the corresponding row entries. Readers read digits correctly but lose the table structure. On seven clean panels of election returns, the best open-source reader reproduces 99.0% of the digits but files only 83.7% of the figures in the correct
cells and gets 11 of 30 columns correct. TableVision treats table structure recovery as the problem. On three degraded 1888 pages, Amazon Textract files only 1 of 1494 figures correctly; TableVision correctly files 99.2%, and all 51 columns its exact-total check releases are correct. A dynamic program cuts a volume into blocks by how many columns sum to their last line; witnesses read the
cells; the gate classifies each column as exactly reconciling, flagged as faithful to a page whose arithmetic is wrong, or withheld. A theory of the sum check says which errors it cannot see and why its optimal block is about $\theta^{-1/2}$ rows at defect rate $\theta$. On 2201 columns of Wisconsin electoral data, the gate accepts 2042, flags 118, and withholds 41; failure rises with column length $R$ as $1-(1-\theta)^R$ at 1.1 defects per 1000
cells in general elections; official returns confirm 933 of 934 testable accepted sums and all 35 testable flagged totals; and the deployment added 491\,359 gated precinct rows to a 9.3-million-row
dataset. One implication is that the design of the verification block is critical.
Read the full abstract ▾
Computer Vision
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
DoYoung Kim, Changkyu Choi, Adín Ramírez Rivera et al. (+2 more)
Dual-encoder
vision-
language models systematically fail at negation, scoring ”a cat without a dog” nearly identically to ”a cat with a dog.” We show that this failure is inherent to the single-
embedding framework. Under contrastive training, single-
embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first
benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.
Read the full abstract ▾
All authors DoYoung Kim, Changkyu Choi, Adín Ramírez Rivera, YunSeok Choi, Jee-Hyong Lee
Computer Vision
World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models
Junyi Wu, Zekun Li, Chenhongyi Yang et al. (+9 more)
Recent advances in
video world models have enabled plausible interactive environments, yet long-horizon generation places a fundamental demand on visual
memory: the previously observed world must remain consistent as the model continues to generate. Existing autoregressive approaches often implicitly rely on the generative model itself to retain such information, leading to spatial
memory decay beyond the active context. We introduce World-T3, a camera-controlled
video world model with a test-time training
memory that consolidates past observations into dedicated model parameters. To robustly update this
memory, we propose a recursive least-squares rule that incrementally absorbs novel visual features via online pose-keyed
memory updates. Our TTT
memory reads out previously observed scene features from the fast-weight state by camera pose, enabling consistent generation with constant storage and per-step computation regardless of rollout length. To facilitate real-time interactive generation with this robust
memory, we further distill the autoregressive model into a few-step generator via dual-teacher distribution matching. On a long-horizon revisit
benchmark, World-T3 raises revisit PSNR from 13.52 to 18.31 dB and reduces absolute rotation error from 3.344 to 1.075 relative to the strongest baseline on both metrics, while sustaining 20.6 FPS at 1280 x 704 on a single NVIDIA B200 GPU.
Read the full abstract ▾
All authors Junyi Wu, Zekun Li, Chenhongyi Yang, Chen Guo, Lingchen Yang, Yongan Zhang, Xinyi Zhang, Hongbing Hu, Bartek Rymkowski, Srinath Sridhar, Yan Yan, Xijie Huang
Computer Vision
Bottlenecked semantic compression by representation reconstruction
Xiaoya Tang, Shilong Liu, Daniel Smolyak et al. (+3 more)
Vision Transformers incur high computational costs when processing long visual
token sequences. Existing reduction methods typically rank
tokens independently and discard those deemed unimportant or redundant. We propose a reconstruction-guided framework for visual
token condensation (ReConV), which summarizes the complete visual sequence with a compact Summary Bank (SMB) and retains
tokens whose information is not adequately captured by the SMB, as indicated by their reconstruction residuals. At each compression layer, a lightweight position-conditioned decoder reconstructs the current
tokens, leaving shared information compressed in the SMB while explicitly preserving poorly reconstructed details. Experiments on image classification and
multimodal understanding demonstrate favorable performance-efficiency trade-offs: ReConV surpasses the uncompressed baseline with 48% fewer GMACs on ImageNet and enables reusable image-level representations for efficient multi-turn inference. Code is available at TokenDrop.
Read the full abstract ▾
All authors Xiaoya Tang, Shilong Liu, Daniel Smolyak, Xiwen Li, Quynh Nguyen, Tolga Tasdizen
Computer Vision · Cross-listed from cs.AI
What Makes a Latent Space Shared? Direct and Recoverable Compatibility in Motion VAEs
Iago Brito, Walcy Rios, Lucas Fernandes et al. (+2 more)
A shared latent space is often assumed to make representations from different schemas directly interchangeable, yet sharing parameters, dimensionality, or a prior does not determine how corresponding instances are expressed in latent coordinates. We show that this assumption conflates recoverable correspondence, direct compatibility, and decoder reuse. In a controlled multi-schema motion VAE on HumanML3D, even identical inputs yield only $5.9$–$17.7\%$ raw Recall@1 and $0.221$–$0.484$ paired-
token cosine, despite correspondence being almost perfectly recoverable after linear calibration. Cycle consistency strengthens recoverability but still leaves direct
retrieval and
token agreement near zero. In contrast, only $64$ paired clips raise raw Recall@1 to $88.4$–$90.3\%$ and
token cosine to approximately $0.82$, making most post-hoc
alignment unnecessary, although crossed decoding remains substantially worse than native reconstruction. We further find that reducing measured marginal discrepancy does not imply schema invariance. These results show that a shared backbone or prior does not ensure directly compatible latent coordinates, while limited paired correspondence can substantially improve direct compatibility in the tested setting.
Read the full abstract ▾
All authors Iago Brito, Walcy Rios, Lucas Fernandes, Kaiki de Oliveira, Arlindo Galvão Filho
Computer Vision
Brain2Face: A Study on the Relation of EEG Signals and Spatio-temporal Head Avatars
Camilo Andrés Martínez Martínez, Olaf Dünkel, Kartik Teotia et al. (+3 more)
Driving digital head avatars is classically done by hand-crafted artist design or by image-based facial performance capture. In this work, we ask whether spatio-temporal tracking of the humans's head geometry and appearance can be performed solely from
brain signals, i.e., Electroencephalography (
EEG). Since prior work mostly focused on activity and emotion recognition, this task is relatively unexplored without even having publicly available high-quality
datasets and
benchmarks. Thus, we contribute a
dataset featuring paired
EEG and human head geometry as well as appearance annotations. We synchronize 64-channel
EEG with per-frame expression latent codes of a personalized Gaussian head avatar. Second, we introduce a baseline to the task of expression recovery of a head avatar from
EEG signals. To this end, we propose a dedicated
EEG-to-expression model that takes a temporal sequence of
EEG signals as input and regresses the latent expression parameters of the 3D Gaussian head avatar. Lastly, we validate our capture and algorithmic design choices through a
benchmark protocol that we introduce to this task. We believe this work paves the way towards better understanding of the relation between
EEG signals and facial surface as well as appearance changes.
Read the full abstract ▾
All authors Camilo Andrés Martínez Martínez, Olaf Dünkel, Kartik Teotia, Maurice Rekrut, Christian Theobalt, Marc Habermann
Computer Vision
WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMs
Ameen Ali, Tamim Zoabi, Lior Wolf et al. (+1 more)
Video language models represent a
video as a long sequence of visual
embeddings in a decoder's input. Every visual
embedding is processed by every decoder layer during prefill, stored in the key value cache, and attended by every generated
token, so a long
video raises prefill,
memory, and generation cost at once. We present WIRE (Weight Space Implicit Representation Encoding), which reduces this recurring cost by moving part of the
video conditioned computation of a frozen
video language model from its input activations into a temporary low rank update of its weights. WIRE keeps a short explicit visual context, so the update acts as a correction rather than a replacement of the
video. WIRE first compresses the full resolution
embeddings of each frame into a small set of carrier
tokens using a question conditioned bidirectional selective scan, and physically contracts the decoder input so every decoder layer, cache entry, and decoding step operates on the shorter sequence. Hypernetworks at selected decoder layers read the compressed forward pass together with a readout of the full resolution features removed by contraction and generate LoRA updates to the
attention and feedforward projections. The updates are merged after prefill, so autoregressive decoding runs the unmodified frozen backbone. On Qwen3-VL-8B, WIRE with 16 carrier
tokens per frame contracts the visual stream by $3.5\times$, reduces decoder prefill time by $3.35\times$ at 256 frames and $4.14\times$ at 1024 frames, shrinks the key value cache proportionally, and keeps per
token decoding latency approximately constant. Across nine
benchmark settings at 256 frames, it obtains the highest average accuracy among efficient
video language models we evaluate under a common 256 frame budget.
Read the full abstract ▾
All authors Ameen Ali, Tamim Zoabi, Lior Wolf, Chaim Baskin
Computer Vision
MID: Mask-Image Distributional Divergence for Evaluating Medical Image Segmentation
Vincenzo Marcianò, Xiaoming Zhang, Gianluca Guglielmo et al. (+3 more)
Current evaluation protocols for medical image segmentation remain largely anchored in sample-wise comparisons, e.g., Dice and Hausdorff distance, which require each prediction to be paired with a ground-truth mask. Yet two limitations arise in practice: they are inapplicable when predictions are made on unlabeled data; and even when annotations are available, they cannot assess whether a collection of predictions matches the distributional structure of the ground-truth population. We introduce MID (Mask-Image Distributional Divergence), a distribution-level evaluation metric for medical image segmentation. MID builds upon the observation that segmentation quality is a property of the joint image-mask distribution: a mask may be anatomically plausible in isolation yet incorrect for the image it accompanies. MID compares the joint distribution of images and masks between a reference set and a set of predictions, without requiring sample-wise correspondence. This captures both mask realism and image-mask consistency. We evaluate MID across 13 publicly available
datasets spanning CT, MRI, and MRA. Under anatomically realistic corruptions across 56 organ groups, MID tracks mean Dice and Hausdorff distance without per-sample ground-truth pairing. MID also detects distributional shifts across modalities and anatomies, identifies image-mask misalignment through controlled shuffling experiments, and produces reliable model rankings across real segmentation models. Official implementation available at github.com/erc-caravel/pytorch-mid
Read the full abstract ▾
All authors Vincenzo Marcianò, Xiaoming Zhang, Gianluca Guglielmo, Michela Antonelli, Sebastien Ourselin, Maria A. Zuluaga
Computer Vision
HairFlow: Modeling 3D Hair Strands with Multi-Modal Conditioned Diffusion
Yating Wang, Kui Wu, Ran Yi et al. (+5 more)
Modeling 3D hair strands is challenging due to complex hairstyle geometry, scarce high-quality training data, and incomplete or noisy input observations. Foundation models offer broader hairstyle priors, but their outputs are not directly usable for strand-based rendering and simulation. Existing approaches require costly per-instance optimization to fit hair priors to meshes from foundation models and 2D orientations from specialized predictors trained on limited hair data, restricting efficiency and generalization. We present HairFlow, a
multimodal conditioned
diffusion framework for efficient and controllable hair strands modeling. HairFlow conditions strand generation on two complementary representations: a coarse mesh that constrains global shape and multi-view sketches that specify local strand flow. HairFlow combines an Orientation VAE with a conditioned
diffusion transformer to generate volumetric fields that are traced into explicit strands. To overcome the lack of paired sketch-strand training data, we introduce an automatic synthesis pipeline that generates geometrically aligned multi-view sketches with diverse styles representative of inference-time inputs. Experiments show that HairFlow generates plausible 3D strands that faithfully follow the input conditions across diverse hairstyles, substantially reduces inference time, and supports flexible
multimodal control.
Read the full abstract ▾
All authors Yating Wang, Kui Wu, Ran Yi, Zixuan Lu, Tongtong Wang, Zhongtian Zheng, Lizhuang Ma, Yuefan Shen
Computer Vision
SSM-Attack: Learning Where to Perturb for Black-Box Face Privacy
Tunahan Parlay{\i}c{\i}, Erchan Aptoula and Yücel Sayg{\i}n
Face recognition systems are deployed widely enough that a photograph shared online can be matched without its subject's consent. Generator-based protection methods rely on heavy pretrained backbones, multi-stage inference or visible semantic edits. We introduce SSM-Attack, a black-box face
privacy framework built on Selective State Space Models. Rather than synthesizing the perturbation, the state-space model predicts in one linear-time pass a spatial allocation of the perturbation budget, marking which facial locations carry identity evidence; a short projected refinement then sets the values within this per-pixel budget. Against published
benchmark results, SSM-Attack attains the highest average black-box attack success rate on all three evaluation sets: 87.10 percent on CelebA-HQ, 86.90 on LADN and 81.15 on FFHQ, the last from a network that never saw that
dataset. The allocation network holds 5.73 million parameters, or 16.13 million with the frozen networks used by its quality terms. Under the same cap of 0.10 on the maximum per-pixel change, allocation improves structural similarity by 0.044, peak signal-to-noise ratio by 1.62 decibels and Frechet Inception Distance by 11.2 points over a uniform map while spending 37 percent of its capacity. Strong facial
privacy protection therefore does not require a large generative prior; it requires knowing where to spend a small one.
Read the full abstract ▾
Computer Vision
HybridDualPathNet: Cross-Fusion Attention of EfficientNetV2 and Swin Transformer for Brain MRI Classification of Alzheimer's and Parkinson's Disease
Jarpula Nirjala and Rahul Patel
Separating Alzheimer's Disease (AD), Parkinson's Disease (PD), and Healthy Controls (HC) using structural MRI remains challenging because disease signatures overlap and model performance can be strongly affected by data partitioning. We present HybridDualPathNet, a hybrid architecture combining EfficientNetV2-S for local features and Swin-Tiny for global context through a Cross-Fusion
Attention (CFA) module. We evaluate the model in two experiments. First, on the Kaggle APD-3-Class
dataset using an image-level split, the model achieves 98.25% accuracy. Second, on a combined Kaggle and Alzheimer's Disease Neuroimaging Initiative (ADNI)
dataset containing 17,859 slices, we use a per-source split with patient-wise separation for ADNI subjects. In this setting, the model achieves 73.37% standard accuracy and a macro AUC of 0.880. Source-stratified evaluation reveals a substantial domain gap, with 91.2% accuracy on Kaggle test images and 59.2% on ADNI test images. Temperature scaling reduces expected calibration error from 0.130 to 0.015, while split-conformal prediction achieves 89.5% coverage. Saliency analyses using Grad-CAM++, Integrated Gradients, and occlusion indicate that model activations concentrate on
brain tissue. These results highlight the importance of patient-level separation, cross-
dataset evaluation, and uncertainty estimation when assessing MRI classification systems. The findings also show that strong performance on a single
dataset does not necessarily translate to reliable generalization across imaging cohorts.
Read the full abstract ▾
Computer Vision
When Left Is Right? Camera-Motion Understanding beyond Screen
Chong Cheng, Ran Zhang, Xinyuan Li et al. (+3 more)
Can a
vision-
language model tell left from right? Modern VLMs solve visual puzzles and follow hour-long
videos, yet when asked whether the camera moved left or right, most do no better than a coin flip. Adding camera-motion data or
geometric representations has been tried, separately, and neither closes the gap. We argue that camera-motion understanding needs two conditions at once: the representation must couple off-screen camera motion with on-screen flow dynamics, and it must be interpreted by the
language model rather than trained as a standalone pose or depth
estimator. We propose CameraVLM, which pairs a motion-state representation with state-
language alignment. Camera
tokens capture how the camera moves off screen, flow
tokens capture how the subject and background move on screen, and a two-stage training strategy keeps
language in the loop so that both are learned together with camera-motion semantics. With only 9B parameters, CameraVLM outperforms all open-source and camera-motion-specific models by a wide margin across three
benchmarks and more than 20 baselines, and matches or surpasses GPT-6 Astra as well as human viewers in a 48-participant study. Project page: https://3dagentworld.github.io/CameraVLM/
Read the full abstract ▾
All authors Chong Cheng, Ran Zhang, Xinyuan Li, Alan Zhao, Hao Wang, Xi Wang
Computer Vision · Cross-listed from cs.AI
IsoVRBench: Video Reward Modeling Without Human Feedback
Shuyue Stella Li, Jacqueline He, Jiahui Chen et al. (+5 more)
Video reward models are trained and evaluated on human preferences between
videos that often differ simultaneously on properties such as
prompt alignment, visual fidelity, motion, and generator identity. As a result, aggregate accuracy does not reveal which
video properties a reward model can reliably distinguish. We construct preference pairs by controlled interventions that change one property of a generated
video while holding the rest fixed. The resulting pairs, by construction, provide preferences for how the
videos differ along a specific dimension, without requiring any human annotation. We propose IsoVRBench, a
video generation reward modeling
benchmark of 8,000 such pairs across eight interventions, and show that state-of-the-art models separate broad quality differences between generators but miss important features such as temporal order. The best reward model reaches only 60.8% on reordered frames against 100% for humans, and rankings on existing
benchmarks do not predict this failure. Using the synthetic isolated intervation pairs as training data leads to 1) IsoVRM, a reward model outperforming its counterpart trained with a larger human labeled preference
dataset by 3.8 points on VBench-2.0, 2) IsoWan, the
video generator post-trained with IsoVRM, outperforming its human-data counterpart in human preferences, and 3) gains that stack when combined with human data. Because intervention pairs are automatically generated rather than collected, the
benchmark and training data can be easily extended as generators evolve.
Read the full abstract ▾
All authors Shuyue Stella Li, Jacqueline He, Jiahui Chen, Sreya Dutta Roy, Yulia Tsvetkov, Luke Zettlemoyer, Marjan Ghazvininejad, Xiaochuang Han
Computer Vision
Beyond Sparse Captions: Aligning Slide-Level Text and Patch-Level Vision in Pathology
Daniel Shao, Dongmin Bang, Luca L. Weishaupt et al. (+6 more)
Contrastive
vision-
language (VL) pretraining in computational pathology has generally relied on curated image-caption
datasets sourced from social media, educational
videos, and research publications. These corpora are sparse relative to the diversity of tissue morphology and difficult to scale further, as pathologists do not routinely produce descriptive captions in clinical practice. We propose to instead leverage slide-level pathology reports which can be generated during routine diagnostic workflows in a scalable manner. A key challenge in this weakly-supervised setting is that pathology reports describe macroscopic, diagnostic findings while patch-level visual features describe microscopic findings. We introduce Locked Image Multiple Instance PreTraining (LIMIT), which addresses this gap by freezing a pretrained
vision encoder and fine-tuning only the text encoder: report
embeddings serve as cross-
attention queries over bags of patch features, producing report-conditioned slide representations optimized via InfoNCE. Using LIMIT, we establish CONCH-Z, which aligns the text encoder of CONCH v1.5 to 335k clinical reports. Evaluated across 22 tasks spanning patch-level classification, tumor detection, and slide-level subtyping, CONCH-Z establishes state-of-the-art zero-shot performance over VL encoders trained on curated caption
datasets, while simultaneously closing the gap with slide-level foundation models trained with substantially more parameters and compute. The code and model is available at https://github.com/mahmoodlab/LIMIT.
Read the full abstract ▾
All authors Daniel Shao, Dongmin Bang, Luca L. Weishaupt, Sophia J. Wagner, Nic G. Reitsam, Ming Y. Lu, Long P. Le, Richard J. Chen, Faisal Mahmood
Computer Vision
AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers
Javad Rajabi, Koorosh Roohi, David B. Lindell et al. (+1 more)
Generating images beyond the training resolution of
Diffusion Transformers (DiTs) remains challenging, as the number of
tokens grows rapidly with resolution, full
attention becomes expensive, and outputs often suffer from structural distortions and degraded detail. We identify uniform full
attention as the common source of both problems. Attending to all
tokens at every step causes quadratic cost and exposes the model to more
tokens and relative distances beyond the training range, pushing the model out of distribution. We observe that
attention's effective receptive field narrows during denoising, from global in early steps that form structure to local in later steps that refine details. We propose AReA, a training-free
attention mechanism whose receptive field evolves from coarse to fine, following the denoising progression. In early steps, AReA applies cross-scale coarse
attention over downsampled keys and values, capturing global context, while in later steps it applies local
attention within progressively shrinking windows. Without additional parameters or training, experiments show that AReA consistently outperforms state-of-the-art training-free baselines across target resolutions and different models, while also being faster and more efficient.
Read the full abstract ▾
All authors Javad Rajabi, Koorosh Roohi, David B. Lindell, Babak Taati
Computer Vision · Cross-listed from cs.RO
DexNinja: Learning Robust Dexterous Cutting Policy with a Real-to-Sim-to-Real Data Engine
Haozhe Lou, Runyi Yang, Wanqi Zhong et al. (+10 more)
Cutting a piece of food looks effortless in human hands. For a
robot, however, every cut is a high-contact interaction with a deformable, elasto-plastic object whose behavior changes with pose, contact mode, and cutting stage. Collecting real-world data is expensive and wasteful: each trial typically contaminates or damages the food, so obtaining scalable real world data for dexterous cutting is expensive and difficult. Simulation is an appealing alternative, but a large sim-to-real gap exists in both physics and perception. This gap is especially severe for
topology-changing interactions like cutting, where small errors in contact, friction, or material and elasticity responses quickly compound. We propose DexNinja, a real2sim2real framework that turns a handful of real demonstrations into a dense and realistic training distribution. DexNinja (i) reconstructs object instances from real captures, (ii) randomizes physically meaningful parameters under category-level constraints, and (iii) produces augmented dexterous manipulation episodes using a custom differentiable simulator that couples
robot dynamics, tactile contact, and deformable cutting. We evaluate DexNinja on a dexterous food-slicing task and show that (1) with tens of real demonstrations and hundreds of augmented simulator data, the success rate could reach more than 60%. (2) the augmented data consistently improves success rates when only using real data with more than 20% via sim-to-real, and (3) the resulting policies generalize to out-of-distribution objects with varying shapes and sizes.
Read the full abstract ▾
All authors Haozhe Lou, Runyi Yang, Wanqi Zhong, Caiyun Liu, Yicheng Liu, Wenhao Liu, Wenlong Ma, Jiawei Xia, Kanzhi Wu, Danda Pani Paudel, Luc Van Gool, Hang Zhao, Yiming Li
Computer Vision
Multics: Generative Multi-Player World Model with Hierarchical Communication
Kaiyi Huang, Wangbo Yu, Yukun Huang et al. (+6 more)
Multi-player interactive world models aim to simulate shared environments from multiple players' perspectives, but achieving robust generation while scaling to more players remains challenging. Existing approaches that rely on fully-connected communication (e.g., Solaris) incur quadratic
attention costs as the number of players grows, while centralized approaches (e.g., $\gamma$-World) suffer from interference between unrelated events in a shared global representation. To address these challenges, we present Multics, a multi-player
video generation framework with hierarchical communication that structures information exchange across player, group, and global levels. This hierarchy introduces group
tokens into self-
attention as intermediate group-level representations that aggregate locally relevant player information. These
tokens connect to global
tokens for cross-group communication, while the hierarchical
topology limits direct mixing between unrelated local events. Group-
token parameters are shared across group slots, allowing the same communication module to accommodate varying numbers of players without introducing new group-specific parameters. We further propose budget-constrained dynamic grouping based on recent camera trajectories, which adaptively groups relevant players while limiting unnecessary communication. Experiments demonstrate that Multics achieves state-of-the-art performance in 4- and 6-player settings, with improved cross-player consistency, player-specific action and camera controllability, and interference robustness. Moreover, Multics generalizes from 4-player training to 6-, 8-, and 10-player inference without retraining, while reducing self-
attention computation compared with fully-connected communication.
Read the full abstract ▾
All authors Kaiyi Huang, Wangbo Yu, Yukun Huang, Jiwen Yu, Kunhao Liu, Donglin Yang, Wenbo Hu, Ying Shan, Xihui Liu
Computer Vision · Cross-listed from cs.AI
Aotian Chen, Frazier N. Baker, Reza Averly et al. (+5 more)
Current chemistry
benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an observed chemical effect occurs. We introduce ChemExplain, a
benchmark of 1,180 multiple-choice chemistry questions drawn from 383 peer-reviewed papers. Each question pairs a molecular or reaction image with an observed chemical effect and four candidate explanations, each associated with a different structural feature visible in the image. Answering requires the model to perceive the depicted structure, apply chemical knowledge, and identify which feature most directly explains the effect, a capability we term explanatory attribution. The correct feature-mechanism relation is derived from supporting evidence in the source paper rather than invented by the question-generation LLM. We construct the
benchmark automatically with a multi-
agent framework that applies progressive quality control at three stages and validate it with eight chemistry domain experts. The strongest evaluated model reaches 78.6% accuracy, leaving considerable room for improvement. On a 600-question expert-reviewed subset,
reasoning-trace analysis classifies 71.4% of the strongest model's errors as attribution failures. Attribution is also the largest error category for the open-source and chemistry-specific models included in our analysis.
Read the full abstract ▾
All authors Aotian Chen, Frazier N. Baker, Reza Averly, Shan Zhang, Tianao Chen, Daniel Adu-Ampratwum, Xia Ning, Yuan Xue
Computer Vision
MARS: Systematic Multi-Level Acceleration for World Action Models
Zengyuan Zuo, Yongchang Zhang, Guangyu Zhao et al. (+4 more)
World Action Models (WAMs) have emerged as a promising alternative for embodied intelligence by jointly predicting future world states and
robot actions. However, their heavy visual encoding and iterative generative inference incur substantial latency, limiting their applicability to high-frequency closed-loop control. Existing training-free acceleration methods typically optimize a single stage, yielding limited speedups or degraded control under aggressive reuse. In this work, we systematically profile the inference overhead of WAMs and observe abundant redundancy across
Transformer computation,
diffusion denoising, and consecutive policy queries. However, the safe removal of such redundancy depends strongly on its impact on action prediction and control sensitivity. Based on these observations, we introduce MARS, a training-free and plug-and-play framework for multi-level WAM acceleration. MARS selectively removes low-impact
attention computation, adaptively accelerates
diffusion inference according to motion granularity, and speculatively reuses predicted future visual representations across consecutive queries with online verification. Together, these complementary mechanisms provide remarkable acceleration while limiting approximation error. Extensive experiments across LIBERO, LIBERO-PRO, and RoboCasa show that MARS achieves up to $4.00\times$, $3.94\times$, and $2.10\times$ end-to-end speedups, respectively, while maintaining task success rate close to the original policy. Moreover, MARS generalizes across different model backbones and across simulation and real-
robot experiments, substantially improving the efficiency–performance trade-off of WAM inference. Our code is available at https://github.com/PKU-VaLuE-Lab/MARS.
Read the full abstract ▾
All authors Zengyuan Zuo, Yongchang Zhang, Guangyu Zhao, Tianze Zhang, Biao Wu, Meng Fang, Yiwu Zhong
Computer Vision
EgoProbe: Benchmarking Long-Range and Exact Memory in Streaming Egocentric Videos
Haokai Pang, Yuqian Fu, Yifei Wang et al. (+4 more)
Egocentric
video provides a continuous first-person record of everyday activities, making it a natural foundation for wearable AI assistants. Unlike offline
video understanding, streaming systems must process observations as they arrive, while queries may refer to experiences from the distant past. Recent work has begun to explore whether information can be recalled across long temporal gaps. Yet remembering how far is only part of the challenge: recurring objects, actions, and locations also require models to remember exactly which detail belongs to which event. We introduce EgoProbe, a
benchmark that evaluates streaming egocentric
memory along these two complementary dimensions: long-range retention, i.e., how far, and event-specific recall under interference, i.e., how exact. Built on long-form egocentric
videos, EgoProbe uses an event-grounded data engine to extend recall intervals, identify related events, and construct event-grounded distractors, followed by human verification. It contains two subsets: EgoProbe-Base, which evaluates long-range recall without intervening related events, and EgoProbe-Echo, which challenges models with similar intervening events and event-grounded distractors. EgoProbe-Echo adopts coarse-to-fine multi-turn probing: the first turn evaluates coarse-grained event recall, while the second tests whether the model can retrieve the target-specific detail associated with the designated event. Extensive evaluation across diverse models reveals substantial limitations in long-range recall, with further challenges in recovering target-event details among related experiences. Even when models correctly answer coarse-grained questions, they often fail to recall the specific details of the designated event. EgoProbe exposes this underexplored failure mode and provides a testbed for advancing long-range and exact
memory in streaming egocentric
video understanding.
Read the full abstract ▾
All authors Haokai Pang, Yuqian Fu, Yifei Wang, Yujiao Shen, Tianwen Qian, Mohamed Elhoseiny, Hongbo Fu
Computer Vision
Contrastive learning for video reuse detection
Tristan J.B. Cann and Arjun Biswas
The incorporation of footage from other sources has long been a concern from a copyright perspective, as such usage can harm the interests of rights holders. Recent trends in the “remixing” of social media content and the development of bespoke stock libraries for specialist causes mean that methods to identify content reuse will have a range of applications. In this paper, we develop a framework based on a Siamese network fine tuned using contrastive learning to identify potential sources for key frames in a given
video. This framework makes two contributions to analysis of large
video datasets. Firstly, we demonstrate its utility at reducing complex
video data to a set of key visuals typically less than 10% of the size of the full set. Secondly, we show that these reduced sets allow for efficient and effective identification of component footage. We test this method over a set of
videos compiled from a stock footage library and demonstrate its effectiveness for constituent
video identification with average accuracy greater than 0.95.
Read the full abstract ▾
Computer Vision
Scaling Unified Multimodal Models to Long-Horizon Interleaved Generation
Ziqiao Peng, Yutao Cui, Wang Lin et al. (+12 more)
Unified
multimodal models (UMMs) provide an interface for interleaved text-image generation, where generated visual states can directly inform subsequent
reasoning and synthesis. However, existing systems are still primarily demonstrated on short trajectories or simplified visual content. We develop a large UMM natively trained on interleaved
multimodal data, with strong capabilities in image synthesis, text rendering, complex composition, and
multimodal planning, and study how to extend these native capabilities to substantially longer autoregressive trajectories. The main challenge is that generated images continuously accumulate in context: irrelevant or imperfect visual states interfere with future synthesis, while previously established characters and scenes must still be recovered after long intervals. We introduce a visual-
memory framework that maintains persistent visual references, selectively retrieves scene-relevant historical states while preserving the complete semantic history, and uses few-step self-rollout to adapt the model to imperfect generated histories. We further introduce LongMIG-Bench for evaluating long-horizon text-to-multi-image and image-to-multi-image generation. Our model achieves high-fidelity long-horizon interleaved generation with strong character and scene consistency, demonstrating how visual
memory can scale native interleaved generation to substantially longer trajectories.
Read the full abstract ▾
All authors Ziqiao Peng, Yutao Cui, Wang Lin, Zhong-Yu Li, Qingwen Gu, Tianhui Song, Chengyou Jia, Guozhen Zhang, Xuerui Qiu, Qixun Wang, Miles Yang, Zhao Zhong, Yonglong Tian, Hongyan Liu, Jun He
Computer Vision
Rethinking LLM-Guided Visual Question Answering: Does Deeper Knowledge Integration Really Help?
Aneerban Saha, Vijay Kumar, Sameer Madhav D. et al. (+2 more)
Knowledge-based Visual Question Answering (VQA) requires
reasoning that extends beyond visible image content, demanding external commonsense and factual knowledge. While recent work leverages Large
Language Models (LLMs) as knowledge sources, the question of how deeply such guidance should be integrated into a
vision language architecture remains underexplored. In this work, we generate structured
reasoning guidance, visual focus concepts, distractor elements and external knowledge using Llama~3.3~70B and systematically compare two integration strategies within a unified frozen ViT + FLAN-T5 framework on the OK-VQA
benchmark: decision level late fusion and
attention-
matrix injection. Contrary to the prevailing assumption that deeper integration is superior, we find that shallow late fusion achieves the strongest results (39.4% exact match, +10.0 over the no guidance baseline), outperforming deep
attention modification (31.8%) under data scarce fine-tuning. We further contribute an LLM based augmentation pipeline that expands training data more than tenfold via question paraphrasing. Our findings indicate that integration depth must be matched to the data regime.
Read the full abstract ▾
All authors Aneerban Saha, Vijay Kumar, Sameer Madhav D., Priyam Srivastava, Dhruv Bansal
Computer Vision
A unified three-branch deep learning framework for automated detection and counting of wildlife in aerial imagery
Zeyu Xu, Tiejun Wang, Andrew Skidmore et al. (+5 more)
Automating aerial wildlife surveys requires reliable animal detection, consistent delineation of observation strips, and correction for repeated coverage between successive images. We present a unified three-branch deep learning framework that integrates these tasks through WildSeg, WildMatch and WildDet. WildSeg combines rod segmentation with
geometric refinement to delineate counting strips from aircraft-mounted markers. WildMatch uses teacher-student
distillation to learn dense image correspondences and identify overlapping survey areas without relying on geolocation metadata. WildDet combines dual-scale detection with feature fusion, density supervision and empty-image gating to support animal detection and counting. We evaluated the framework using oblique aerial imagery of four antelope species from the Great Nile Migration Landscape in South Sudan. WildSeg achieved reliable rod extraction, while WildMatch approached teacher-level accuracy with a substantially lighter and faster model. Both modules maintained high accuracy on resized images, improving processing efficiency. WildDet improved detection performance across all four species and reduced detection-based count MAE, RMSE and mean bias by 21.0%, 4.6% and 27.8%, respectively, relative to YOLO11. By connecting spatial survey constraints with animal detection, the framework provides a practical and computationally efficient approach to processing large aerial image collections for wildlife monitoring and population surveys.
Read the full abstract ▾
All authors Zeyu Xu, Tiejun Wang, Andrew Skidmore, Megan Claase, Harry Wells, Isla Duporge, Ramiro D. Crego, Richard Lamprey
Computer Vision
PerturbBridge: Direct Phenotypic Supervision for Cellular Perturbation Generation
Yucheng Yang, Xiaoxiao Sun, Huangqingbo Sun et al. (+3 more)
Virtual
cell models should capture phenotypes induced by specific perturbations. However, image-based models are typically trained and evaluated to match real-image distributions, which alone does not explicitly prioritize perturbation-specific biological responses. Recent approaches add biological signals through reinforcement-learning post-training or inference-time steering, raising a central question: where should biological response information enter the generative pipeline? We introduce PerturbBridge, a
stochastic data-to-data bridge model that directly supervises perturbation-specific phenotypes in predicted perturbed images during training. PerturbBridge jointly optimizes bridge regression and differentiable biological response losses, with biological gradients passing through a frozen image encoder. We further introduce perturbation-response evaluations based on downstream classification and
retrieval. Across three microscopy
benchmarks for perturbation profiling and subcellular imaging, PerturbBridge surpasses CellFlux in biological fidelity, image quality, and sampling efficiency. On BBBC021, mechanism-of-action classification improves by 5.66 and 7.5 percentage points with morphological and learned visual features, respectively. On JUMP and AllenCell,
retrieval Hit@10 and Hit@1 improve by 1.77 and 7.15 percentage points, respectively. On BBBC021 and JUMP, PerturbBridge achieves lower FID with 80% fewer model evaluations than CellFlux. Overall, these results show that directly supervising perturbation responses during training improves phenotypic fidelity while enabling efficient cellular image generation. Code is available at https://github.com/yyc1215lll-creator/PerturbBridge.
Read the full abstract ▾
All authors Yucheng Yang, Xiaoxiao Sun, Huangqingbo Sun, Elaine Sui, Emma Lundberg, Serena Yeung-Levy
Computer Vision · Cross-listed from cs.AI
VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval
Simon Ott, Yufan Chen, Ruiping Liu et al. (+6 more)
Vision-based document
retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap between leaderboard performance and realistic long-document
retrieval, where the evidence needed to answer a query may span multiple pages or even multiple documents. We present VL-DocIR, a page-level
benchmark for evidence-complete visual long-document
retrieval. VL-DocIR is built from 29,641 HTML-born documents rendered into 388,548 page images from Wikipedia, arXiv, PubMed, and SEC proxy statements. It contains 271,760 queries across 23 domains and six evidence structures, covering single-page, within-document multi-page, and cross-document evidence configurations. Evidence annotations are grounded to rendered pages and HTML element identifiers, and queries are filtered with a cleaning pipeline targeting clarity, correctness, closedness, and the absence of explicit layout references. We evaluate leading single-vector and multi-vector visual retrievers together with diagnostic
retrieval settings such as document-context scoring, query splitting, reranking, and round-robin ensembles. The strongest evaluated configuration reaches 69.3 nDCGAll@10 and 80.6 RecallAll@10 under structure-balanced Macro evaluation, but source- and evidence-structure breakdowns show that cross-document
retrieval remains substantially harder than single-page
retrieval, and that document-context scoring is especially helpful for within-document multi-page queries. VL-DocIR exposes these failure modes and provides a more demanding target for visual long-document
retrieval. The new
dataset and code will be made publicly available at https://yufanchen96.github.io/VL-DocIR.
Read the full abstract ▾
All authors Simon Ott, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
Computer Vision
DIME: Scaling Facial Representation Learning via Differential Masked Autoencoding
Hao Yu, Haoyu Chen, Hui Wei et al. (+3 more)
Scaling up data and model capacity has driven substantial progress in visual representation learning. However, conventional pretext tasks exhibit diminishing returns on fine-grained facial analysis. We attribute this to a scaling bottleneck: these tasks model each image's appearance and are dominated by identity, pose, and illumination, so subtle within-identity facial differences contribute little to the training signal. We introduce DIME, a differential masked autoencoder for large-scale facial representation learning. DIME builds on a simple observation: two images of the same person share identity-specific appearance, so their difference exposes facial changes. DIME reconstructs both masked images in an identity-matched pair and, through an Explicit Differential Direction Supervision (EDDS) objective, aligns the direction of their reconstructed difference with the true difference, while source isolation makes each reconstruction depend only on its own image. Shared appearance cancels in this difference, and because EDDS ignores magnitude, subtle facial changes count as much as large pose or illumination changes. DIME uses identity labels only to form pretraining pairs, requires no dense annotations, and learns dense facial representations with emergent part-level semantics. We pretrain DIME on 20 million high-resolution facial images consolidated from existing face
datasets. DIME's AU detection performance continues to improve as pretraining data scales from 6M to 20M images, with its BP4D F1 increasing by 5.9 points, while MAE gains only 0.5 points on the same subsets. DIME also outperforms specialized facial methods on AU detection, AU intensity estimation, cross-domain anti-spoofing, landmark localization, head pose estimation, face parsing, and face generation.
Read the full abstract ▾
All authors Hao Yu, Haoyu Chen, Hui Wei, Yan Jiang, Nicu Sebe, Guoying Zhao