Computer Vision
MoVerse: Real-Time Video World Modeling with Panoramic Gaussian Scaffold
Yang Zhou, Ziheng Wang, Yuqin Lu et al. (+4 more)
We present MoVerse, a real-time
video world model that creates an interactively navigable scene from a single narrow-field-of-view image. This setting is challenging because the input observes only a small fraction of the environment, while interactive roaming requires a complete surrounding world, persistent geometry, controllable camera motion, and temporally coherent high-fidelity observations. MoVerse addresses this problem by separating world construction from observation rendering. It first expands the input into a
gravity-aligned 360$^\circ$ panorama with
topology-aware
diffusion, closing the missing field of view before 3D
reasoning. It then lifts the panorama into a persistent 3D Gaussian scaffold using panoramic geometry-aware residual prediction, yielding a dense and directly renderable spatial
memory. Finally, a Gaussian-conditioned
video renderer translates scaffold renderings along user-specified camera trajectories into photorealistic
video. To make this renderer practical for interaction, we train a bidirectional
diffusion teacher for high-quality conditional rendering and distill it into a causal autoregressive student for bounded-latency streaming. This design combines the controllability and long-range consistency of explicit 3D representations with the perceptual quality of generative
video models. MoVerse supports real-time scene roaming at 8~FPS on a single NVIDIA RTX~4090 GPU, demonstrating a practical path toward single-image world creation with interactive
video output.
Read the full abstract ▾
All authors Yang Zhou, Ziheng Wang, Yuqin Lu, Haofeng Liu, Jun Liang, Shengfeng He, Jing Li
Computer Vision · Cross-listed from cs.AI
IterCAD: An Iterative Multimodal Agent for Visually-Grounded CAD Generation and Editing
Tao Hu, Jiaxin Ai, Licheng Wen et al. (+12 more)
Computer-Aided Design is pivotal in modern manufacturing, yet existing automated methods predominantly rely on open-loop, one-shot generation, creating a mismatch with iterative real-world practices. In this paper, we present IterCAD, a unified
multimodal agent framework for closed-loop, interactive CAD generation and editing. We formulate the task as a multi-turn interaction between a
multimodal agent and an executable CAD sandbox, covering three tasks: Drawing-to-Code, Text-to-Code, and Interactive Editing. To support this, we develop a data synthesis pipeline incorporating advanced industrial manufacturing features to generate standard-compliant multi-view engineering drawings, complex code-editing tasks, and high-fidelity interaction trajectories. We optimize the
agent via progressive SFT followed by geometry-aware reinforcement learning with viable-prefix masking to enhance code executability and
geometric fidelity. Finally, we introduce the IterCAD-Bench evaluation suite and propose the Chamfer Distance Tolerance-Recall (CD-TR) curve alongside its AUC-TR metric, establishing a survivor-bias-free standard that unifies code validity and
geometric precision. Extensive experiments demonstrate that IterCAD achieves highly competitive performance across multiple
benchmarks, significantly outperforming existing approaches in both code executability and
geometric precision, while exhibiting superior capabilities in closed-loop iterative refinement.
Read the full abstract ▾
All authors Tao Hu, Jiaxin Ai, Licheng Wen, Xueheng Li, Shu Zou, Siqi Li, Nianchen Deng, Xinyu Cai, Hongbin Zhou, Pinlong Cai, Daocheng Fu, Yu Yang, Hairong Zhang, Botian Shi, Xuemeng Yang
Computer Vision
Dual-Constrained Diffusion Image Compression for Operational Rate-Distortion-Perception Optimization
Sanxin Jiang, Jiro Katto and Heming Sun
The rate-distortion-perception (RDP) trade-off extends classical rate–distortion theory by imposing a distributional constraint on reconstructions, providing a unified framework for neural image compression that jointly governs fidelity and perceptual realism. While prior work achieves near-optimal rate–perception trade-offs, practical frameworks explicitly realizing the full RDP surface remain scarce, primarily due to the difficulty of introducing common randomness at the decoder. We propose DCIC (Dual-Constrained
Diffusion Image Compression), which integrates a learned codec with a
diffusion-based decoder governed by joint distortion and idempotence constraints. The distortion constraint bounds reconstruction fidelity relative to the base codec output; the idempotence constraint – requiring that re-encoding the restored image recovers the base codec reconstruction – serves as a tractable surrogate for the distributional perception requirement. Together, they steer the reverse denoising process via iterative optimization with consistent noise injection, realizing common randomness without additional rate overhead. At fixed rate, dual attenuation factors $(K_D, K_P)$ jointly navigate the Pareto frontier of the distortion-perception plane, enabling continuously adjustable fidelity-realism trade-offs from a single bitstream. DCIC$_{RD}$ ($K_P{=}0$) and DCIC$_{RP}$ ($K_D{=}0$) arise as boundary curves, with DCIC$_{RDP}$ ($K_D = K_P=1$) realizing the optimal interior operating point. Experiments on CelebA-HQ, CLIC2020, and ImageNet-1K across CNN,
Transformer, and hybrid architectures confirm that DCIC$_{RDP}$ achieves superior BD-PSNR over all perceptual codecs, while DCIC$_{RP}$ matches dedicated perception-oriented methods in BD-FID, validating the practical value of full RDP surface
navigation.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
VideoMDM: Towards 3D Human Motion Generation From 2D Supervision
Amir Mann, Gal Michael Harari, Merav Keidar et al. (+1 more)
We introduce VideoMDM, a
diffusion-based framework that trains 3D human motion priors directly from accurate 2D poses extracted from
monocular videos, without any 3D ground truth. A pretrained 2D-to-3D lifter provides approximate 3D pose sequences that serve as a noisy teacher: these are diffused, denoised by the model in 3D, and supervised in 2D by reprojecting the prediction and comparing against accurate keypoints. We show that, under mild assumptions, a depth-weighted 2D reprojection loss is equivalent in expectation to direct 3D supervision, and we adapt standard 3D motion regularizers - velocity consistency and over-parameterized representation
alignment - to this 2D setting. Unlike methods that lift 2D to 3D only at inference, VideoMDM learns a coherent 3D motion
manifold during training. On HumanML3D it nearly closes the gap to fully 3D-supervised MDM (FID 0.88 vs 0.54); On real
video datasets Fit3D and NBA the method learns to generate motions consistently preferred by humans, with strong quantitative results.
Read the full abstract ▾
All authors Amir Mann, Gal Michael Harari, Merav Keidar, Or Litany
Computer Vision
JointEdit3D: Feed-Forward 3D Scene Editing in a Unified Latent Space
Xinnan Zhu, Ruijie Xu, Jiayu Ying et al. (+4 more)
Existing 3D scene editing methods typically rely on per-scene optimization over explicit 3D representations or cascaded edit-and-reconstruct pipelines, resulting in high test-time cost, limited 3D awareness, and structural inconsistencies. To couple appearance synthesis with geometry prediction, we adapt a pretrained unified RGB-geometry latent space to feed-forward scene editing. Given a source
video and an edited reference image, JointEdit3D performs asymmetric latent inpainting: it observes only the edited RGB reference latent and jointly generates the remaining RGB latents and the entire geometry latent along the source trajectory. JointEdit3D introduces a dedicated SceneAnchor Branch to inject source-scene structure without forcing direct copying, and adopts edit/background-aware losses to balance edited-region fidelity with unedited-content preservation. To address the lack of paired resources for standardized 3D scene editing evaluation, we introduce SceneEdit3D-15K, a
dataset with 15K paired editing samples and renderer-provided 3D annotations, together with SceneEdit3D-Bench, a curated 100-sample
benchmark. Experiments show that JointEdit3D improves edited-region quality and 3D structural completeness over prior baselines while maintaining competitive background preservation.
Read the full abstract ▾
All authors Xinnan Zhu, Ruijie Xu, Jiayu Ying, Daoguo Dong, Jiachen Xu, Yuan Xie, Xin Tan
Computer Vision
Dual-Domain Equivariant Generative Adversarial Network for Multimodal CT-PET Synthesis
Gabriel Steele, Alzahra Altalib and Alessandro Perelli
We present a Dual-Domain Equivariant Generative
Adversarial Network (DDE-GAN) for
multimodal CT-PET image synthesis. Traditional GAN-based approaches often operate solely in the spatial domain and ignore
geometric consistency, resulting in limited structural fidelity. DDE-GAN addresses these challenges by jointly learning from both spatial and frequency (
Fourier) domains, capturing complementary anatomical and spectral information. Furthermore, rotational equivariance embedded in the physics of the CT and PET measurements are integrated into the loss of both the generator and discriminator to ensure consistent responses under rotations, improving anatomical accuracy. A hierarchical dual-domain training strategy enforces intra- and inter-domain consistency through multi-stage loss functions. Evaluated on the HECKTOR 2022 CT-PET
dataset, DDE-GAN achieves superior synthesis quality over baseline models for CT-PET image synthesis. The results demonstrate that combining dual-domain learning with
geometric equivariance substantially enhances
multimodal image synthesis accuracy and robustness, enabling practical applications in PET completion and data augmentation.
Read the full abstract ▾
Computer Vision
OR-Action: Multi-Role Video Understanding with Fine-Grained Actions
Felix Tristram, Ege Özsoy, Christian Benz et al. (+3 more)
Fine-grained understanding of operating room (OR) activity could enable workflow-aware assistance, yet remains difficult due to clutter, occlusions, and limited sensing. The prevailing approach to model this environment is scene graphs as an interpretable representation of OR interactions. Converting their frame-wise relational predictions into temporally extended, fine-grained actions however, is challenging without explicit temporal modeling. To enable a principled temporal evaluation of current OR understanding methods, we introduce the first action-centric
benchmark built on a publicly available ego-exocentric OR
dataset by defining a fine-grained, multi-role action taxonomy and generating dense action segments via
distillation from ground-truth scene graph state changes. Experiments on this
benchmark show that current scene graph prediction methods struggle to model temporal structure, even when adding explicit modeling through Graph Neural Networks. We therefore introduce a
vision-only temporal model that outperforms graph-based methods significantly when using all available
egocentric video as input. Building on this model we also introduce a novel multi- to single-view feature
alignment strategy that improves single-view performance on multi-role action recognition, mitigating the need for extensive
egocentric video capture.
Benchmark and code will be released upon acceptance.
Read the full abstract ▾
All authors Felix Tristram, Ege Özsoy, Christian Benz, Marcel Walch, Ghazal Ghazaei, Nassir Navab
Computer Vision
Masked and Predictive Self-Supervised Pretraining for Brain MRI: Effects of Objective Design on Disease Detection
Esra Ergün, Hersh Chandarana, Dan Sodickson et al. (+1 more)
MRI self-supervised learning (SSL) has largely emphasized
segmentation and dense prediction, while controlled studies of how alternative pretraining objectives shape transferable representations for MRI-based disease detection remain limited. We compare reconstruction-based Masked Autoencoders (MAE) and latent-predictive Joint
Embedding Predictive Architectures (JEPA) for 3D structural MRI. We further introduce a spectral-domain reconstruction loss for MAE to promote sensitivity to fine-grained anatomical structure and integrate variance-covariance regularization (VCR) into JEPA to encourage broader, less redundant latent representations. Models are pretrained on heterogeneous MRI
datasets using individual single-contrast inputs. Across seven
dataset-task settings spanning four clinical problem domains, downstream transfer depends on both the pretraining paradigm and how the representational bias induced by a specific objective aligns with the downstream task. Spectral regularization provides its clearest gains for Alzheimer's disease classification and tumor grading, while its benefit is smaller for NC vs. MCI and ASD. VCR yields its strongest improvements when discrimination appears to involve multiple sources of structural variation. Overall, MAE-based pretraining generally outperforms baseline JEPA, with SL-MAE providing the strongest gains in several disease-detection settings. Representation analyses show that fine-tuning reorganizes learned representations in an objective-, depth-, and task-dependent manner. VCR-JEPA maintains broader effective dimensionality across depth and greater cross-task representational
alignment than baseline JEPA. We publicly release subject-level pretraining and downstream evaluation splits.
Read the full abstract ▾
All authors Esra Ergün, Hersh Chandarana, Dan Sodickson, Gozde Unal
Computer Vision
MEGMAG: Micro-Expression Generation via MAGnification
Sliman Jammal and Andrei Sharf
Facial micro-expressions are subtle, short-lived facial movements that reveal genuine human
emotions. Despite the large advances in facial expression synthesis and animation, realistic micro-expressions remain challenging due to scarce annotated data and the extremely subtle nature of their motion. Existing dedicated micro-expression methods therefore often struggle with generation quality, robustness, and generalization. We present MEGMAG, a transferable framework for micro-expression generation via magnification that enables pretrained facial animation models to generate realistic micro-expressions without modifying or fine-tuning their backbones. The key idea is to transform subtle micro-expression dynamics into the domain of regular facial expressions, where pretrained animation models can effectively capture and transfer facial motion. A learnable Magnification Module amplifies subtle micro-expression dynamics, enabling a frozen facial animation backbone to transfer the amplified motion to a source identity without altering its original architecture or parameters. A DeMagnification Module then attenuates the generated motion to recover realistic micro-expression characteristics. This design allows MEGMAG to leverage diverse off-the-shelf facial animation models while remaining architecture-agnostic and fully transferable across backbones.
Read the full abstract ▾
Computer Vision
ReFree: Towards Realistic Co-Speech Video Generation via Reward-Free RL and Multilevel Speech Guidance
Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral et al. (+1 more)
Speech-driven talking character animation seeks to generate life-like portrait
videos that convey natural conversation behavior, aligning facial motion with spoken audio. Although recent advances in
video generation have substantially improved realism in
video-based animation, achieving both accurate lip articulation and expressive behavior remains challenging. Existing approaches typically trade off precise phoneme-to-lip synchronization against dynamic facial expressions and head motion, yielding animations that are either accurate yet rigid, or expressive but poorly synchronized. We address this challenge by proposing ReFree-S2V, a flow-matching
speech-to-portrait animation framework that builds upon a pretrained
video generation model to achieve fine-grained
speech articulation and high-level expressive cues in
speech-driven portrait animation. This model introduces a multi-level
speech representation capturing phonetic and prosodic information at both local and global granularities. These representations are selectively injected into
transformer blocks via learnable level selectors, enabling both accurate lip synchronization and natural expressive motion. To achieve natural head movements, we further introduce a novel reward-free reinforcement learning scheme into flow-matching training to discourage perceptually implausible motion without relying on handcrafted synchronization metrics or reward models, or the high cost of human preference annotation. Extensive experiments demonstrate that ReFree-S2V achieves state-of-the-art performance, significantly outperforming existing methods in both quantitative lip-sync accuracy and qualitative human evaluations of naturalness and expressivity.
Read the full abstract ▾
All authors Salaheldin Mohamed, M. Hamza Mughal, Rishabh Dabral, Christian Theobalt
Computer Vision
DuET: Dual Expert Trajectories for Diffusion Image Editing
Lidia Troeshestova, Alexander Ustyuzhanin and Sergey Kastryulin
Recent
diffusion editors perform diverse instruction-based edits while conditioning on the source image at every denoising step. Yet persistent source-image conditioning can limit how fully an edit is executed and how natural the result appears, especially when the target scene diverges substantially from the input. We introduce DuET (Dual Expert Trajectories), a training-free inference method that temporarily relaxes source-image conditioning by transitioning through a text-to-image phase before returning to edit mode ($\mathrm{E}\to\mathrm{T2I}\to\mathrm{E}$), allowing the denoising trajectory to move toward the target distribution while retaining the structural benefits of image-conditioned editing. Without modifying model weights or increasing sampling cost, DuET consistently improves instruction relevance, semantic fidelity, and perceptual quality across diverse models and
benchmarks. Fixed switching schedules obtain these gains at a modest, predictable cost in source-image preservation; we show this cost is not fundamental. A per-edit variant, Selective DuET, routes on lightweight
attention-probe signals read from the edit trajectory and improves fidelity, naturalness, and artifact scores while keeping source preservation perceptually indistinguishable from the baseline.
Read the full abstract ▾
Computer Vision
HYDRA-X: Native Unified Multimodal Models with Holistic Visual Tokenizers
Guozhen Zhang, Xuerui Qiu, Yutao Cui et al. (+11 more)
Holistic visual tokenizers are fundamental to unified
multimodal models (UMMs) as they map diverse visual inputs into a unified representation space. In this paper, we present HYDRA-X, the first UMM that unifies image and
video tokenization within a single
Vision Transformer (ViT). Our design is driven by two core challenges: efficiently injecting spatiotemporal reconstruction capability into a native ViT, and
embedding image- and
video-level semantic awareness into the latent space. To address the first, comprehensive ablations reveal two key findings: (1) frame-level causal temporal
attention suffices for visual reconstruction, whereas full spatiotemporal
attention degrades it; and (2) hierarchical temporal compression substantially outperforms single-step alternatives. To tackle the second, we propose a lightweight decompressor that upsamples temporally compressed features under joint image-
video teacher supervision, thereby enforcing complementary semantic structures within the compact latent space. Building on this holistic tokenizer, we further propose a principled improvement of the editing pipeline: source-target interaction should occur at the latent level inside the tokenizer rather than at the semantic level inside the LLM, substantially improving editing consistency and accelerating convergence. Instantiated at the 7B dense model, HYDRA-X achieves strong performance across image and
video understanding and generation tasks, paving the way for future unified-tokenizer UMMs.
Read the full abstract ▾
All authors Guozhen Zhang, Xuerui Qiu, Yutao Cui, Tianhui Song, Changlin Li, Junzhe Li, Tao Huang, Xiao Zhang, Yang Li, Jianbing Wu, Miles Yang, Zhao Zhong, Liefeng Bo, Limin Wang
Computer Vision
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
Wei Li, Zhen Huang and Xinmei Tian
Contrastively trained
vision-
language models like CLIP, have made remarkable progress in learning joint image-text representations, but still face challenges in compositional understanding. They often exhibit a "bag-of-words" behavior–struggling to capture the object relations, attribute-object bindings, and word order dependencies. This limitation arises not only from the reliance on global, single-vector representations for optimization, but also from the insufficient exploitation and modeling of the rich compositional information inherently present in paired image text data. In this work, we propose MACCO (MAsked Compositional Concept MOdeling), a framework that masks compositional concepts in one modality and reconstructs them conditioned on the full contextual information from the other, enabling the model to capture and align cross-modal compositional structures more effectively. To facilitate this process, we introduce two auxiliary objectives that jointly align and regularize masked features both inter-modally and intra-modally. Extensive experiments on five compositional
benchmarks, along with in-depth analyses, demonstrate that our approach not only significantly enhances compositionality in VLMs but also improves their ability to capture syntactic structure and linguistic information. Additionally, the improved compositionality also benefits text-to-image generation and
multimodal large
language model. Code is available at https://github.com/hiker-lw/MACCO.
Read the full abstract ▾
Computer Vision
Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing
Anugrah Aidin Yotolembah, Novanto Yudistira and Gembong Edhi Setyawan
This paper presents Custom ZeroCLIP, a
retrieval-augmented
vision-
language framework for zero-shot captioning of Indonesian traditional garments. The
dataset contains 3,800 expert-annotated images from all 38 Indonesian provinces. Using a province-level inductive zero-shot protocol, the model is trained on 24 seen provinces, validated on 6 seen provinces, and evaluated on 8 unseen provinces. The framework combines a frozen CLIP ViT-B/32 image encoder, a CLIP text encoder, a BERT text encoder, and an LSTM caption decoder. During inference, unseen-province labels and captions are unavailable, and
retrieval uses only captions from training provinces. No unseen-province image, label, or caption is used during training, validation, or
retrieval-bank construction. Custom ZeroCLIP achieves a CLIPScore of 0.8536, BLEU-4 of 0.3342, and METEOR of 0.4859, outperforming existing baselines. Ablation results show that
retrieval improves cultural vocabulary recovery with a 19.3% METEOR gain, while human evaluation confirms stronger cultural accuracy and fluency. The results demonstrate the effectiveness of
retrieval-augmented domain adaptation for culturally grounded caption generation in low-resource heritage settings. The
dataset is publicly available at https://github.com/AnugrahAidinYotolembah/Traditional-Indonesian-Clothing-Captioning-
Dataset.
Read the full abstract ▾
Computer Vision
TimeLens: On-Device Artifact Recognition with Retrieval-Augmented Question Answering for the Grand Egyptian Museum
Rawan Hesham, Ali Ashraf, Amr Ahmed et al. (+3 more)
TimeLens is an AI-powered bilingual mobile guide for the Grand Egyptian Museum (GEM). Pointing a phone at an exhibit, a visitor sees the artifact recognized in real time and can ask follow-up questions answered in English or Arabic. The work addresses three problems specific to in-gallery deployment: fine-grained visual similarity among 51 catalogued artifacts (many near-identical Ramesside statues), the gap between curated training data and handheld camera conditions, and the risk of an AI guide stating unsupported historical facts. Two engineering contributions are reported. First, an on-device artifact detector was developed through a data-quality-driven iteration study – from foundation-model auto-annotation (YOLO-World), through spatial label-cleaning rules, to a fully hand-annotated
dataset – isolating label quality as the decisive factor: the final YOLOv8n model resolves every previously failing class while remaining a 5.97 MB TensorFlow Lite asset that runs in real time on a mid-range phone (mAP@0.5 = 0.995, mAP@0.5:0.95 = 0.924). Second, a bilingual
Retrieval-Augmented Generation (RAG) guide, grounded in a 108-record ChromaDB knowledge base, was benchmarked across seven candidate
language models, with Gemma 4 E2B (Q4 K M) selected; ten targeted optimizations reduce end-to-end latency from over 30 s to approximately 10 s. Both subsystems are integrated in a production Flutter application with bilingual interface, museum location gating, and text-to-
speech support.
Read the full abstract ▾
All authors Rawan Hesham, Ali Ashraf, Amr Ahmed, Malak Alaa, Omar Ahmed, Omar Wagih
Computer Vision · Cross-listed from cs.LG
Towards More General Control of Diffusion Models Using Jeffrey Guidance
Raphaël Razafindralambo, Rémy Sun, Frédéric Precioso et al. (+2 more)
A key strength of
diffusion models lies in their flexibility, since their outputs can be controlled at sampling time through guidance. However, beyond simple cases such as conditional sampling, the target distribution is often left implicit, defined only through a sampling rule or a heuristic energy function. To address this, we propose Jeffrey guidance, a principled framework that extends
diffusion-model control to applications beyond what standard guidance can express. It leverages Jeffrey's rule of conditioning to update marginal distributions towards a prescribed target, preserving the conditional structure and minimally perturbing the joint distribution. We first demonstrate Jeffrey guidance by targeting a prescribed
embedding distribution. With Inception
embeddings as the target, this leads to substantial reductions in FID on both CIFAR-10 and FFHQ. We further apply Jeffrey guidance to
fairness on CelebA-HQ, updating an unconditional
diffusion model to enforce independence between attributes.
Read the full abstract ▾
All authors Raphaël Razafindralambo, Rémy Sun, Frédéric Precioso, Jes Frellsen, Pierre-Alexandre Mattei
Computer Vision · Cross-listed from cs.SE
ComAct: Reframing Professional Software Manipulation via COM-as-Action Paradigm
Jiaxin Ai, Tao Hu, Xuemeng Yang et al. (+11 more)
Existing computer-use
agents remain fundamentally limited in professional software manipulation: GUI-based
agents suffer from fragile visual grounding and long-horizon error accumulation, while API-basedapproaches struggle with heterogeneous protocols and inaccessible commercial interfaces. In this work,we identify the Component Object Model (COM) as a unified executable abstraction, proposing COM-as-Action: a new paradigm that reframes professional software interaction as deterministic program synthesisrather than sequential visual control. To validate this paradigm in the most demanding environments, weintroduce ComCADBench, the first
benchmark for
agents operating real industrial CAD software. Ourexperiments reveal a substantial paradigm gap: frontier proprietary models achieve near-zero successunder GUI-based interaction, whereas COM-based execution yields substantial immediate gains. Tobridge the remaining gap between syntactic correctness and
geometric accuracy, we develop ComActor, aself-correcting
agent trained through a progressive three-stage framework, alongside ComForge, a scalableplatform for large-scale training in Windows containers. Extensive experiments show that ComActorachieves state-of-the-art performance on ComCADBench, with strong resilience in long-horizon taskswhere baselines collapse, and generalizes to external CAD
benchmark.
Read the full abstract ▾
All authors Jiaxin Ai, Tao Hu, Xuemeng Yang, Shu Zou, Hairong Zhang, Daocheng Fu, Yu Yang, Hongbin Zhou, Nianchen Deng, Pinlong Cai, Zhongyuan Wang, Botian Shi, Kaipeng Zhang, Licheng Wen
Computer Vision · Cross-listed from cs.LG
Distributional Loss for Robust Classification
Kathleen Anderson and Thomas Martinetz
This paper proposes a novel loss concept for supervised classification tasks. Rather than enforcing a direct mapping from each input sample to a single assigned label, we define an optimization objective over all classifier outputs as a bimodal Gaussian distribution. This softer target formulation implicitly captures class ambiguity, mitigates overfitting, and encourages the learning of more robust decision boundaries, all without requiring additional label information. Experimental results demonstrate consistent improvements in robustness, with particularly pronounced gains in low-data regimes, while requiring only minimal modifications to standard training pipelines.
Read the full abstract ▾
Computer Vision
Visual Place Recognition in Forests with Depth-Aware Distillation
Walter Nedov, Saimunur Rahman, Kavindie Katuwandeniya et al. (+3 more)
Visual place recognition in natural forest environments remains challenging due to repetitive vegetation, weak structural cues, and significant appearance variation across traversals. To address this limitation, this paper proposes a lightweight depth-aware
distillation framework that injects
geometric cues into a DINOv2-based place recognition model, while maintaining its pre-trained descriptor space. Evaluated on the recent WildCross
benchmark, the proposed approach yields gains over an appearance-only counterpart, providing robustness to appearance variations. These results demonstrate the importance of depth as a strong complementary modality for place recognition in natural environments and identify depth-aware
distillation as a promising direction for more robust forest perception.
Read the full abstract ▾
All authors Walter Nedov, Saimunur Rahman, Kavindie Katuwandeniya, David Hall, Kaushik Roy, Peyman Moghadam
Computer Vision
Transformer-Guided Graph Attention for Direct Cardiac Mesh Reconstruction: A Structural Digital Twin Framework
Abhishek H S, Akash Ganamukhi, Abhimanyu Suresh et al. (+3 more)
Building patient-specific cardiac models sits at the heart of precision cardiology, yet getting those models into clinical use keeps running into the same wall: mesh generation is slow, messy, and frustrating. The standard workflow – segmenting the image, running Marching Cubes, and then manually cleaning up the result – is time-consuming, inconsistent across operators, and demands specialist knowledge most clinical teams do not have. We take a fundamentally different approach. Instead of treating
segmentation and mesh generation as two separate problems, we train a single end-to-end network that goes directly from a raw 3D medical image to a smooth, simulation-ready cardiac surface mesh. The core is a 3D Swin
Transformer encoder-decoder that extracts volumetric features from CT or MRI volumes, paired with a Graph
Attention Network (GAT) head that iteratively deforms a template mesh to fit the patient's cardiac boundary. We tested on the MM-WHS 2017
benchmark using both CT and MRI.
Segmentation scores were competitive (Dice of 0.84 on CT, 0.83 on MRI), but the primary focus is mesh quality: mean Chamfer distance of 1.8 mm, with 95th-percentile surface distance below 5 mm. Every mesh is produced in a single forward pass – no Marching Cubes, no smoothing filters, no manual cleanup. We argue that for cardiac digital twin pipelines,
geometric fidelity and topological correctness matter more than pixel-level Dice scores. By removing the post-processing bottleneck, this approach makes patient-specific cardiac simulation substantially more accessible for clinical use.
Read the full abstract ▾
All authors Abhishek H S, Akash Ganamukhi, Abhimanyu Suresh, Aditya G Hiremath, Prasad B Honnavalli, Adithya Balasubramanyam
Computer Vision
Iterative Visual Thinking and the Self-Correction Mirage in VLM Grounding
Animesh Tripathy and Aswanth Krishnan
Letting a
vision-
language model (VLM) think longer at test time has driven much recent progress. A natural way to bring this to spatial grounding is visual self-correction: the model predicts a bounding box, sees it rendered on the image, and refines it over several steps. We build a faithful instance of this idea, Iterative Visual Thinking (IVT), with a two-phase recipe: a supervised warm-up in which the base model's own predictions serve as realistic errors that a teacher VLM turns into corrective
reasoning traces (yielding training data without human annotation), followed by GRPO with a simple IoU reward. Measured the way such systems are usually reported, it works: the trained model surpasses the single-shot base by +2.4pp Acc@0.5. We show this gain is a measurement mirage. The reported number silently keeps, per sample, the trajectory step closest to the ground-truth box: an oracle that needs the very answer it predicts. Re-scored under deployable, label-free stopping rules the improvement vanishes, and the best policy is not to iterate at all: stopping at step 0 matches the base and beats every shippable rule. The cause is a verification failure, since the model can generate a better box somewhere in its trajectory but cannot identify it. Self-verification confidence correlates only weakly with correctness (r about 0.22), and a counterfactual overlay shows the loop reacts to the presence of a rendered box rather than its correctness. We distill the lesson into an honest-trajectory evaluation protocol: accuracy under fixed label-free policies plus an explicit oracle-shippable gap.
Read the full abstract ▾
Computer Vision
An Extensible and Lightweight Unified Architecture for Demosaicing Pixel-bin Image Sensors
Saurabh Kumar and Nutan Sairam Yenneti
Pixel-bin image sensors are becoming the default choice for smartphone cameras due to their resolution vs light-gathering trade-off. However, their larger inter-color separation compared to the Bayer color filter array (CFA) makes them challenging to demosaic. Furthermore, existing deep learning-based demosaicing methods are CFA-specific, requiring multiple individual models that take up precious onboard resources and demand larger development and maintenance efforts. In this work, we propose a modular unified architecture for demosaicing various pixel-bin sensors that provides higher image quality while being extensible and lightweight. Additionally, to enable plug-and-play operation, we introduce a learning-free CFA-identification module to detect the CFA type of raw data accurately.
Read the full abstract ▾
Computer Vision
Cascade Classification of Dermoscopic Images of Skin Neoplasms with Controllable Sensitivity and External Clinical Validation
Elena S. Kozachok, Sergey S. Seregin, Aleksandr V. Kozachok et al. (+2 more)
Purpose. To compare deep learning architectures and classification schemes for dermoscopic images of skin neoplasms and assess their generalization on transfer from open international
datasets to independent clinical
datasets of Russian practice. Methods. Four architectures (ViT-B/16, Swin-S, ConvNeXt-S, EfficientNetV2-S) were compared in three schemes: binary (malignant/benign), single-stage four-class (benign, MEL, SCC, BCC), and a two-stage cascade (binary triage, then three-class differentiation MEL/SCC/BCC). All models used ImageNet-pretrained weights and a single augmentation protocol on aggregated open ISIC Archive data, and were evaluated on an internal held-out sample and two clinical
datasets (Melanoscope AI mobile system; Sechenov University). Results. Internally the binary stage attains ROC-AUC 0.952-0.966; on Sechenov University it drops to 0.797-0.893, sensitivity to 0.53-0.67, and ECE rises from 0.02 to 0.27-0.39 with underestimation of malignancy, quantifying a generalization gap in ranking and calibration. Paired tests confirm one inter-architecture result on clinical data: the deficit of ViT-B/16 at the binary stage (p<0.05); at the differentiation stage no architecture has a proven advantage. The cascade raises macro F1 over single-stage four-class classification for most architectures, but significantly only for ViT-B/16, by recovering malignant lesions assigned to the dominant benign class. On ISIC MILK10k, direct 11-class classification yields mean-class sensitivity 0.525. Conclusion. A tunable triage threshold gives sensitivity control not attainable in standard single-stage (argmax) classification and better reproduces clinical differential-diagnosis logic. The persistent generalization gap mandates external clinical validation and recalibration before deployment.
Read the full abstract ▾
All authors Elena S. Kozachok, Sergey S. Seregin, Aleksandr V. Kozachok, Ilya P. Latyshev, Oleg I. Samovarov
Computer Vision
Fully Distributed Multi-View 3D Tracking in Real-Time
Byron Hernandez, Fangyu Li, Aotian Wu et al. (+3 more)
Multi-camera tracking with overlapping fields of view typically relies on centralized fusion, which creates computational bottlenecks that prevent deployment at scale. We present MV3DT, a fully distributed framework for real-time multi-view 3D tracking that achieves accurate identity propagation and occlusion recovery through peer-to-peer coordination, eliminating the need for central aggregation. Each camera node executes a lightweight modular pipeline comprising
monocular 3D perception, distributed multi-view association, and collaborative fusion via lightweight messaging. MV3DT achieves 96.5% IDF1, 93.1% MOTA, and 94.6% MOTP on WILDTRACK, competitive with state-of-the-art centralized methods, and unprecedented 41.7% IDF1 and 50.9% MOTA on SCOUT while demonstrating superior scalability: sustaining 30 FPS on 100 cameras with <10ms inter-camera latency and only 2.2% communication overhead. MV3DT operates in a zero-shot regime given camera calibrations, requiring no scene-specific learning and making it directly deployable in new environments. These results establish MV3DT as a practical solution for real-time multi-view tracking in large-scale overlapping camera networks.
Read the full abstract ▾
All authors Byron Hernandez, Fangyu Li, Aotian Wu, Paul J. Shin, Kaustubh Purandare, Henry Medeiros
Computer Vision
PP-OCRv6: From 1.5M to 34.5M Parameters, Surpassing Billion-Scale VLMs on OCR Tasks
Yubo Zhang, Xueqing Wang, Manhui Lin et al. (+13 more)
Vision-
Language Models (VLMs) have achieved impressive results on general
vision-
language tasks, yet they suffer from
hallucination, imprecise localization, and prohibitive computational cost when applied to dedicated OCR scenarios. This paper presents PP-OCRv6, a lightweight OCR system that combines architectural innovation with data-centric optimization. PP-OCRv6 redesigns the backbone, detection neck, and recognition neck around a unified MetaFormer-style building block with structural reparameterization, decoupling spatial
token mixing from channel mixing and supporting both tasks through task-specific stride configurations. Three model tiers (medium, small, tiny) share the same block primitives, covering deployment scenarios from server to edge. On our in-house
benchmarks, PP-OCRv6_medium achieves 83.2% recognition accuracy and 86.2% detection Hmean, outperforming PP-OCRv5_server by +5.1% and +4.6% respectively while surpassing Qwen3-VL-235B, GPT-5.5, and Gemini-3.1-Pro with orders of magnitude fewer parameters. The tiny tier achieves 3.9$\times$ faster inference than PP-OCRv5_mobile on Intel Xeon CPU while maintaining comparable accuracy.
Read the full abstract ▾
All authors Yubo Zhang, Xueqing Wang, Manhui Lin, Yue Zhang, Penglongyi Deng, Ting Sun, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Changda Zhou, Hongen Liu, Suyin Liang, Cheng Cui, Yi Liu, Dianhai Yu, Yanjun Ma
Computer Vision
Unified MRI Brain Image Translation via Hierarchical Tumor Structure Comparison
Yupeng Cai, Jia Wei and Jianlong Zhou
Multi-modal MRI
brain image translation via available modalities holds significant practical importance in modern medicine, providing robust support for early diagnosis, treatment planning, and outcome assessment of diseases. For this purpose, it is important to ensure the fidelity of the tumor regions after translation. However, existing
brain image translation methods ignore the structure information of different tumor regions, which could assist translation models in enhancing the quality and clinical applicability of the translated images. In this work, we propose a novel translation model called HTSCGAN, which is a unified multi-modal
brain image translation generative
adversarial model integrating the structural information within tumor regions with the aim of improving the quality of
brain image translation. Specifically, the generator employs three Patch Contrast Module (PCM) with different patch sizes to capture the hierarchical structural information of the tumor regions. In addition, a pretrained Patch Classifier (PC) and a pretrained Structure-Aware Encoder (SAE) are employed to derive the generated image containing the same tumor region structure as the ground truth image via patch classification loss and tumor perceptual loss, respectively. The experiments on BraTS2020 and BraTS2021 demonstrate strong performance of our model in both translation tasks and down stream
segmentation tasks, highlighting its effectiveness in enhancing the quality and clinical relevance of the translated
brain images. Our code is available at https://anonymous.4open.science/r/HTSCGAN.
Read the full abstract ▾
Computer Vision
LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
Peixi Wu, Biao Yang, Feipeng Ma et al. (+7 more)
Reasoning-driven universal
multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT)
reasoning into the
embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT
reasoning incurs high computational cost, making it impractical for low-latency
retrieval; and (ii)
embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions: Is textual CoT the optimal form of
reasoning for
embedding, and can effective
embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent
Reasoning Multimodal Embedding), which formulates
embedding-oriented latent
reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason
tokens as a fixed-capacity bottleneck, completing all
reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60x faster inference than explicit CoT methods and 2x faster than latent baselines with throughput comparable to discriminative
embedding models. Code is available at https://github.com/PeppaWu/LaME.
Read the full abstract ▾
All authors Peixi Wu, Biao Yang, Feipeng Ma, Bosong Chai, Bo Lin, Wei Yuan, Fan Yang, Tingting Gao, Hebei Li, Xiaoyan Sun
Computer Vision · Cross-listed from cs.AI
Augmentation techniques for video surveillance in the visible and thermal spectral range
Vanessa Buhrmester, Ann-Kristin Grosselfinger, David Munch et al. (+1 more)
In intelligent
video surveillance, cameras record image sequences during day and night. Commonly, this demands different sensors. To achieve a better performance it is not unusual to combine them. We focus on the case that a long-wave infrared camera records continuously and in addition to this, another camera records in the visible spectral range during daytime and an intelligent
algorithm supervises the picked up imagery. More accurate, our task is multispectral CNN-based object detection. At first glance, images originating from the visible spectral range differ between thermal infrared ones in the presence of color and distinct texture information on the one hand and in not containing information about thermal radiation that emits from objects on the other hand. Although color can provide valuable information for classification tasks, effects such as varying illumination and specialties of different sensors still represent significant problems. Anyway, obtaining sufficient and practical thermal infrared
datasets for training a deep neural network poses still a challenge. That is the reason why training with the help of data from the visible spectral range could be advantageous, particularly if the data, which has to be evaluated contains both visible and infrared data. However, there is no clear evidence of how strongly variations in thermal radiation, shape, or color information influence classification accuracy. To gain deeper insight into how Convolutional Neural Networks make decisions and what they learn from different sensor input data, we investigate the suitability and robustness of different augmentation techniques...
Read the full abstract ▾
All authors Vanessa Buhrmester, Ann-Kristin Grosselfinger, David Munch, Michael Arens
Computer Vision
SeamEdit: A Black-Box VLM-Agnostic Pipeline for Large-Image Semantic Editing
Xiangyu Lyu and Dan Lei
Semantic region editing for large images must satisfy two requirements at the same time: high generative quality and natural integration with surrounding content. Some related methods rely on white-box models and leave the strong generation capability of closed-source models underexplored. Directly applying closed-source models to tiled editing, however, introduces several failure modes: semantic deformation, canvas-level
alignment drift, and visible seam artifacts. This paper presents SeamEdit, a training-free and model-agnostic pipeline that treats any VLM with inpainting capability as a black-box oracle. SeamEdit mitigates these issues through a five-stage post-hoc pipeline: overlay-based tile decomposition, black-box VLM inpainting,
geometric and color-consistency correction, seam-risk-based multi-candidate ranking, and dynamic-programming curved seam fusion. The pipeline reduces seam visibility and supports semantic modification of arbitrary tile regions.
Read the full abstract ▾
Computer Vision
TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment
Yu Meng, Xiangyang Luo, Letian Li et al. (+5 more)
Autoregressive
video diffusion models provide a natural formulation for streaming and variable-length
video generation by conditioning newly generated frames on previously generated content. However, extending these models to minute-level generation remains challenging: the limited KV-cache budget prevents the model from retaining the full history, while repeatedly conditioning on self-generated frames induces a context distribution shift that accumulates over time, leading to visual artifacts, quality degradation, and temporal drift. In this paper, we propose TetherCache, a training-free and plug-and-play cache management strategy for drift-resistant long
video generation. TetherCache organizes the cache into sink,
memory, and recent regions, and introduces two complementary mechanisms. First, GRAB (Gated Recall with
Attention-Diversity Balancing) selects long-range
memory frames using a gated score that combines
attention-based relevance with temporal diversity, preserving informative yet diverse historical context under a fixed cache budget. Second, TAME (Trusted
Alignment via
Memory Editing) lightly edits newly recalled
memory tokens by aligning their statistics to a trusted context distribution, reducing the pollution caused by drifted historical features. Built on Self-Forcing, TetherCache consistently improves long-
video generation quality on VBench-Long across 30s, 60s, and 240s settings. In particular, for 240s generation, it substantially improves overall and semantic scores while reducing quality drift from 7.84 to 1.33, demonstrating its effectiveness for stable long-horizon autoregressive
video diffusion.
Read the full abstract ▾
All authors Yu Meng, Xiangyang Luo, Letian Li, Wenyuan Jiang, Chen Gao, Xinlei Chen, Yong Li, Xiao-Ping Zhang