Computer Vision · Cross-listed from cs.GR
DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation
Ziyuan Liu, Shizhao Sun, Danqing Huang et al. (+5 more)
Graphic design generation demands a delicate balance between high visual fidelity and fine-grained structural editability. However, existing approaches typically bifurcate into either non-editable raster image synthesis or abstract layout generation devoid of visual content. Recent combinations of these two approaches attempt to bridge this gap but often suffer from rigid composition schemas and unresolvable visual dissonances (e.g., text-background conflicts) due to their inexpressive representation and open-loop nature. To address these challenges, we propose DesignAsCode, a novel framework that reimagines graphic design as a programmatic synthesis task using HTML/CSS. Specifically, we introduce a Plan-Implement-Reflect pipeline, incorporating a Semantic Planner to construct dynamic, variable-depth element hierarchies and a Visual-Aware Reflection mechanism that optimizes the code to rectify rendering artifacts. Extensive experiments demonstrate that DesignAsCode significantly outperforms baselines in both structural validity and aesthetic quality. Furthermore, our code-native representation unlocks advanced capabilities, including automatic layout retargeting, complex document generation (e.g., resumes), and CSS-based animation. Our project page is available at https://liuziyuan1109.github.io/design-as-code/.
Read the full abstract ▾
All authors Ziyuan Liu, Shizhao Sun, Danqing Huang, Yingdong Shi, Meisheng Zhang, Ji Li, Jingsong Yu, Jiang Bian
Computer Vision · Cross-listed from cs.LG
Robust Pre-Training of Medical Vision-and-Language Models with Domain-Invariant Multi-Modal Masked Reconstruction
Melika Filvantorkaman and Mohsen Piri
Medical
vision-
language models show strong potential for joint
reasoning over medical images and clinical text, but their performance often degrades under domain shift caused by variations in imaging devices, acquisition protocols, and reporting styles. Existing multi-modal pre-training methods largely overlook robustness, treating it as a downstream adaptation problem. In this work, we propose Robust Multi-Modal Masked Reconstruction (Robust-MMR), a self-supervised pre-training framework that explicitly incorporates robustness objectives into masked
vision-
language learning. Robust-MMR integrates asymmetric perturbation-aware masking, domain-consistency regularization, and modality-resilience constraints to encourage domain-invariant representations. We evaluate Robust-MMR on multiple medical
vision-
language benchmarks, including medical visual question answering (VQA-RAD, SLAKE, VQA-2019), cross-domain image-text classification (MELINDA), and robust image-caption
retrieval (ROCO). Robust-MMR achieves 78.9% cross-domain accuracy on VQA-RAD, outperforming the strongest baseline by 3.8 percentage points, and reaches 74.6% and 77.0% accuracy on SLAKE and VQA-2019, respectively. Under perturbed evaluation, Robust-MMR improves VQA-RAD accuracy from 69.1% to 75.6%. For image-text classification, cross-domain MELINDA accuracy increases from 70.3% to 75.2%, while
retrieval experiments show a reduction in mean rank degradation from over 16 to 4.1 under perturbation. Qualitative results further demonstrate improved clinical
reasoning for disease detection and structural abnormality assessment. These findings show that explicitly modeling robustness during pre-training leads to more reliable and transferable medical
vision-
language representations for real-world deployment.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
SAS-Net: Cross-Domain Image Registration as Inverse Rendering via Structure-Appearance Factorization
Jiahao Qin
Cross-domain image registration requires aligning images acquired under heterogeneous imaging physics, where the classical brightness constancy assumption is fundamentally violated. We formulate this problem through an image formation model I = R(s, a) + epsilon, where each observation is generated by a rendering function R acting on domain-invariant scene structure s and domain-specific appearance statistics a. Registration then reduces to an inverse rendering problem: given observations from two domains, recover the shared structure and re-render it under the target appearance to obtain the registered output. We instantiate this framework as SAS-Net (Scene-Appearance Separation Network), where instance normalization implements the structure-appearance decomposition and Adaptive Instance Normalization (AdaIN) realizes the differentiable forward renderer. A scene consistency loss enforces
geometric correspondence in the factorized latent space. Experiments on EuroSAT-Reg-256 (satellite remote sensing) and FIRE-Reg-256 (retinal fundus) demonstrate state-of-the-art performance across heterogeneous imaging domains. SAS-Net (3.35M parameters) achieves 89 FPS on an RTX 5090 GPU. Code: https://github.com/D-ST-Sword/SAS-Net.
Read the full abstract ▾
Computer Vision
Towards Understanding Multimodal Fine-Tuning: Spatial Features
Lachin Naghashyar, Hunar Batra, Ashkan Khakzar et al. (+4 more)
Contemporary
Vision-
Language Models (VLMs) achieve strong performance on a wide range of tasks by pairing a
vision encoder with a pre-trained
language model, fine-tuned for visual-text inputs. Yet despite these gains, it remains unclear how
language backbone representations adapt during
multimodal training and when
vision-specific capabilities emerge. In this work, we present the first mechanistic analysis of VLM adaptation. Using stage-wise model diffing, a technique that isolates representational changes introduced during
multimodal fine-tuning, we reveal how a
language model learns to "see". We first identify
vision-preferring features that emerge or reorient during fine-tuning. We then show that a selective subset of these features reliably encodes spatial relations, revealed through controlled shifts to spatial
prompts. Finally, we trace the causal activation of these features to a small group of
attention heads. Our findings show that stage-wise model diffing reveals when and where spatially grounded
multimodal features arise. It also provides a clearer view of modality fusion by showing how visual grounding reshapes features that were previously text-only. This methodology enhances the interpretability of
multimodal training and provides a foundation for understanding and refining how pretrained
language models acquire
vision-grounded capabilities.
Read the full abstract ▾
All authors Lachin Naghashyar, Hunar Batra, Ashkan Khakzar, Philip Torr, Ronald Clark, Christian Schroeder de Witt, Constantin Venhoff
Computer Vision
VideoNeuMat: Neural Material Extraction from Generative Video Models
Bowen Xue, Saeed Hadadan, Zheng Zeng et al. (+3 more)
Creating photorealistic materials for 3D rendering requires exceptional artistic skill. Generative models for materials could help, but are currently limited by the lack of high-quality training data. While recent
video generative models effortlessly produce realistic material appearances, this knowledge remains entangled with geometry and lighting. We present VideoNeuMat, a two-stage pipeline that extracts reusable neural material assets from
video diffusion models. First, we finetune a large
video model (Wan 2.1 14B) to generate material sample
videos under controlled camera and lighting trajectories, effectively creating a "virtual gonioreflectometer" that preserves the model's material realism while learning a structured measurement pattern. Second, we reconstruct compact neural materials from these
videos through a Large Reconstruction Model (LRM) finetuned from a smaller Wan 1.3B
video backbone. From 17 generated
video frames, our LRM performs single-pass inference to predict neural material parameters that generalize to novel viewing and lighting conditions. The resulting materials exhibit realism and diversity far exceeding the limited synthetic training data, demonstrating that material knowledge can be successfully transferred from internet-scale
video models into standalone, reusable neural 3D assets.
Read the full abstract ▾
All authors Bowen Xue, Saeed Hadadan, Zheng Zeng, Fabrice Rousselle, Zahra Montazeri, Milos Hasan
Computer Vision
TwistNet-2D: Learning Second-Order Channel Interactions via Spiral Twisting for Texture Recognition
Junbo Jacob Lian, Feng Xiong, Yujun Sun et al. (+7 more)
Second-order feature statistics are central to texture recognition, yet existing mechanisms exhibit a structural tension: bilinear pooling and Gram
matrices capture global channel correlations but discard spatial structure, whereas self-
attention models capture cross-position relations through weighted sums rather than explicit pairwise products. We propose TwistNet-2D, a lightweight module that computes local pairwise channel products under directional spatial displacement, jointly encoding where features co-occur and how they interact. The core component, Spiral-Twisted Channel Interaction (STCI), shifts one feature map along a prescribed direction before L2-normalized channel multiplication, capturing cross-position co-occurrence patterns that characterize structured and periodic textures. Four directional heads are aggregated through content-adaptive channel reweighting, and the result is injected via a sigmoid-gated residual path with near-zero initialization. TwistNet-2D adds only approximately 3.5% parameters and approximately 2% FLOPs over ResNet-18. To isolate the contribution of architectural inductive bias from that of transfer learning, all models in this study are trained from scratch without ImageNet pretraining. Under this protocol, TwistNet-2D consistently surpasses parameter-matched baselines and substantially larger ConvNeXt and Swin
Transformer backbones across four texture and fine-grained recognition
benchmarks, while the multi-head structure produces interpretable, orientation-selective representations that align with classical texture analysis.
Read the full abstract ▾
All authors Junbo Jacob Lian, Feng Xiong, Yujun Sun, Kaichen Ouyang, Zong Ke, Mingyang Yu, Shengwei Fu, Zhong Rui, Zhang Yujun, Huiling Chen
Computer Vision
3D Transport-based Morphometry (3D-TBM) for medical image analysis
Hongyu Kan, Kristofor Pas, Ivan Medri et al. (+5 more)
Transport-Based Morphometry (TBM) has emerged as a new framework for 3D medical image analysis. By
embedding images into a transport domain via invertible transformations, TBM facilitates effective classification, regression, and other tasks using transport-domain features. Crucially, the inverse mapping enables the projection of analytic results back into the original image space, allowing researchers to directly interpret clinical features associated with model outputs in a spatially meaningful way. To facilitate broader adoption of TBM in clinical imaging research, we present 3D-TBM, a tool designed for morphological analysis of 3D medical images. The framework includes data preprocessing, computation of optimal transport
embeddings, and analytical methods such as visualization of main transport directions, together with techniques for discerning discriminating directions and related analysis methods. We also provide comprehensive documentation and practical tutorials to support researchers interested in applying 3D-TBM in their own medical imaging studies. The source code is publicly available through PyTransKit.
Read the full abstract ▾
All authors Hongyu Kan, Kristofor Pas, Ivan Medri, Naqib Sad Pathan, Natasha Ironside, Shinjini Kundu, Jingjia He, Gustavo Kunde Rohde
Computer Vision
The Double-Edged Sword of Data-Driven Super-Resolution: Adversarial Super-Resolution Models
Haley Duba-Sullivan, Steven R. Young and Emma J. Reid
Data-driven super-resolution (SR) methods are often integrated into imaging pipelines as preprocessing steps to improve downstream tasks such as classification and detection. However, these SR models introduce a previously unexplored attack surface into imaging pipelines. In this paper, we present AdvSR, a framework demonstrating that
adversarial behavior can be embedded directly into SR model weights during training, requiring no access to inputs at inference time. Unlike prior attacks that perturb inputs or rely on
backdoor triggers, AdvSR operates entirely at the model level. By jointly optimizing for reconstruction quality and targeted
adversarial outcomes, AdvSR produces models that appear benign under standard image quality metrics while inducing downstream misclassification. We evaluate AdvSR on three SR architectures (SRCNN, EDSR, SwinIR) paired with a YOLOv11 classifier and demonstrate that AdvSR models can achieve high attack success rates with minimal quality degradation. These findings highlight a new model-level threat for imaging pipelines, with implications for how practitioners source and validate models in safety-critical applications.
Read the full abstract ▾
Computer Vision · Cross-listed from eess.IV
Extracting Root-Causal Brain Activity Driving Psychopathology from Resting State fMRI
Eric V. Strobl
Neuroimaging studies of psychiatric disorders often correlate imaging patterns with diagnostic labels or composite symptom scores, yielding diffuse associations that obscure underlying mechanisms. We instead seek to identify root-causal maps – localized BOLD disturbances that initiate pathological cascades – and to link them selectively to symptom dimensions. We introduce a bilevel structural causal model that connects between-subject symptom structure to within-subject resting-state fMRI via independent latent sources with localized direct effects. Based on this model, we develop SOURCE (Symptom-Oriented Uncovering of Root-Causal Elements), a procedure that links interpretable symptom axes to a parsimonious set of localized drivers. Experiments show that SOURCE recovers localized maps consistent with root-causal BOLD drivers and increases interpretability and anatomical specificity relative to existing comparators.
Read the full abstract ▾
Computer Vision
Understanding Real-World Traffic Safety through RoadSafe365 Benchmark
Xinyu Liu, Darryl C. Jacob, Yuxin Liu et al. (+4 more)
Although recent traffic
benchmarks have advanced
multimodal data analysis, they generally lack systematic evaluation aligned with official safety standards. To fill this gap, we introduce RoadSafe365, a large-scale
vision-
language benchmark that supports fine-grained analysis of traffic safety from extensive and diverse real-world
video data collections. Unlike prior works that focus primarily on coarse accident identification, RoadSafe365 is independently curated and systematically organized using a hierarchical taxonomy that refines and extends foundational definitions of crash, incident, and violation to bridge official traffic safety standards with data-driven traffic understanding systems. RoadSafe365 provides rich attribute annotations across diverse traffic event types, environmental contexts, and interaction scenarios, yielding 36,196 annotated clips from both dashcam and surveillance cameras. Each clip is paired with multiple-choice question-answer sets, comprising 864K candidate options, 8.4K unique answers, and 36K detailed scene descriptions collectively designed for
vision-
language understanding and
reasoning. We establish strong baselines and observe consistent gains when fine-tuning on RoadSafe365. Cross-domain experiments on both real and synthetic
datasets further validate its effectiveness. Designed for large-scale training and standardized evaluation, RoadSafe365 provides a comprehensive
benchmark to advance reproducible research in real-world traffic safety analysis.
Read the full abstract ▾
All authors Xinyu Liu, Darryl C. Jacob, Yuxin Liu, Xinsong Du, Muchao Ye, Bolei Zhou, Pan He
Computer Vision
Condition Matters in Full-head 3D GANs
Heyuan Li, Huimin Zhang, Yuda Qiu et al. (+10 more)
Conditioning is crucial for stable training of full-head 3D GANs. Without any conditioning signal, the model suffers from severe mode collapse, making it impractical to training. However, a series of previous full-head 3D GANs conventionally choose the view angle as the conditioning input, which leads to a bias in the learned 3D full-head space along the conditional view direction. This is evident in the significant differences in generation quality and diversity between the conditional view and non-conditional views of the generated 3D heads, resulting in global incoherence across different head regions. In this work, we propose to use view-invariant semantic feature as the conditioning input, thereby decoupling the generative capability of 3D heads from the viewing direction. To construct a view-invariant semantic condition for each training image, we create a novel synthesized head image
dataset. We leverage FLUX.1 Kontext to extend existing high-quality frontal face
datasets to a wide range of view angles. The image clip feature extracted from the frontal view is then used as a shared semantic condition across all views in the extended images, ensuring semantic
alignment while eliminating directional bias. This also allows supervision from different views of the same subject to be consolidated under a shared semantic condition, which accelerates training and enhances the global coherence of the generated 3D heads. Moreover, as GANs often experience slower improvements in diversity once the generator learns a few modes that successfully fool the discriminator, our semantic conditioning encourages the generator to follow the true semantic distribution, thereby promoting continuous learning and diverse generation. Extensive experiments on full-head synthesis and single-view GAN inversion demonstrate that our method achieves significantly higher fidelity, diversity, and generalizability.
Read the full abstract ▾
All authors Heyuan Li, Huimin Zhang, Yuda Qiu, Zhengwentai Sun, Keru Zheng, Lingteng Qiu, Peihao Li, Qi Zuo, Ce Chen, Yujian Zheng, Yuming Gu, Zilong Dong, Xiaoguang Han
Computer Vision
DuMeta++: Spatiotemporal Dual Meta-Learning for Generalizable Few-Shot Brain Tissue Segmentation Across Diverse Ages
Yongheng Sun, Jun Shu, Jianhua Ma et al. (+1 more)
Accurate
segmentation of
brain tissues from MRI scans is critical for neuroscience and clinical applications, but achieving consistent performance across the human lifespan remains challenging due to dynamic, age-related changes in
brain appearance and morphology. While prior work has sought to mitigate these shifts by using self-supervised regularization with paired longitudinal data, such data are often unavailable in practice. To address this, we propose DuMeta++, a dual meta-learning framework that operates without paired longitudinal data. Our approach integrates: (1) meta-feature learning to extract age-agnostic semantic representations of spatiotemporally evolving
brain structures, and (2) meta-initialization learning to enable data-efficient adaptation of the
segmentation model. Furthermore, we propose a
memory-bank-based class-aware regularization strategy to enforce longitudinal consistency without explicit longitudinal supervision. We theoretically prove the convergence of our DuMeta++, ensuring stability. Experiments on diverse
datasets (iSeg-2019, IBIS, OASIS, ADNI) under few-shot settings demonstrate that DuMeta++ outperforms existing methods in cross-age generalization. Code will be available at https://github.com/ladderlab-xjtu/DuMeta++.
Read the full abstract ▾
All authors Yongheng Sun, Jun Shu, Jianhua Ma, Fan Wang
Computer Vision · Cross-listed from cs.LG
Mimetic Initialization of MLPs
Asher Trockman and J. Zico Kolter
Mimetic initialization uses pretrained models as case studies of good initialization, using observations of structures in trained weights to inspire new, simple initialization techniques. So far, it has been applied only to spatial mixing layers, such convolutional, self-
attention, and state space layers. In this work, we present the first attempt to apply the method to channel mixing layers, namely multilayer perceptrons (MLPs). Our extremely simple technique for MLPs – to give the first layer a nonzero mean – speeds up training on small-scale
vision tasks like CIFAR-10 and ImageNet-1k. Though its effect is much smaller than spatial mixing initializations, it can be used in conjunction with them for an additional positive effect.
Read the full abstract ▾
Computer Vision
Privacy in Image Datasets: A Case Study on Pregnancy Ultrasounds
Rawisara Lohanimit, Yankun Wu, Amelia Katirai et al. (+2 more)
The rise of generative models has led to increased use of large-scale
datasets collected from the internet, often with minimal or no data curation. This raises concerns about the inclusion of sensitive or private information. In this work, we explore the presence of pregnancy ultrasound images, which contain sensitive personal information and are often shared online. Through a systematic examination of LAION-400M
dataset using CLIP
embedding similarity, we retrieve images containing pregnancy ultrasound and detect thousands of entities of private information such as names and locations. Our findings reveal that multiple images have high-risk information that could enable re-identification or impersonation. We conclude with recommended practices for
dataset curation, data
privacy, and ethical use of public image
datasets.
Read the full abstract ▾
All authors Rawisara Lohanimit, Yankun Wu, Amelia Katirai, Yuta Nakashima, Noa Garcia
Computer Vision · Cross-listed from cs.IR
Jianrui Zhang, Anirudh Sundara Rajan, Brandon Han et al. (+3 more)
Universal
Multimodal Retrieval (UMR) seeks any-to-any search across text and
vision, yet modern
embedding models remain brittle when queries require latent
reasoning (e.g., resolving underspecified references or matching compositional constraints). We argue this brittleness is often data-induced: when images carry "silent" evidence and queries leave key semantics implicit, a single
embedding pass must both reason and compress, encouraging spurious feature matching. We propose a data-centric framework that decouples these roles by externalizing
reasoning before
retrieval. Using a strong
Vision–
Language Model, we make implicit semantics explicit by densely captioning visual evidence in corpus entries, resolving ambiguous
multimodal references in queries, and rewriting verbose instructions into concise
retrieval constraints. Inference-time enhancement alone is insufficient; the retriever must be trained on these semantically dense representations to avoid distribution shift and fully exploit the added signal. Across M-BEIR, our
reasoning-augmented training method yields consistent gains over strong baselines, with ablations showing that corpus enhancement chiefly benefits knowledge-intensive queries while query enhancement is critical for compositional modification requests. We publicly release our code at https://github.com/AugmentedRetrieval/ReasoningAugmentedRetrieval.
Read the full abstract ▾
All authors Jianrui Zhang, Anirudh Sundara Rajan, Brandon Han, Soochahn Lee, Sukanta Ganguly, Yong Jae Lee
Computer Vision
Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models
Haoyu Zhang, Zhipeng Li, Yiwen Guo et al. (+1 more)
Omni-modal large
language models (OLLMs) aim to unify
multimodal understanding and generation, yet extending them to jointly produce
speech and 3D facial animation remains largely underexplored. A key challenge is the mismatch between the discrete semantic
reasoning of LLMs and the dense temporal dynamics required for 3D facial motion. We propose Expressive Omni (Ex-Omni), a framework that augments OLLMs with
speech-accompanied 3D facial animation. Ex-Omni decouples semantic
reasoning from temporal generation through a
speech-unit generator with blendshape co-supervision and a non-autoregressive blendshape decoder, where
speech units provide temporal scaffolding and hidden
speech representations carry facially relevant cues. We further introduce a
token-as-query gated fusion (TQGF) interface for controlled semantic injection, as well as InstructS2SF-1200K, a 1.2M-sample weakly supervised
dataset for
speech-accompanied facial animation. Extensive experiments show that Ex-Omni retains competitive
speech QA capability while natively generating coordinated text,
speech, and 3D facial animation, and approaches the Audio2Face-3D teacher cascade in synchronization and human preference.
Read the full abstract ▾
All authors Haoyu Zhang, Zhipeng Li, Yiwen Guo, Tianshu Yu
Computer Vision
Extended to Reality: Prompt Injection in 3D Environments
Zhuoheng Li and Ying Chen
Multimodal large
language models (MLLMs) have advanced the capabilities to interpret and act on visual input in 3D environments, empowering diverse applications such as robotics and situated conversational
agents. When MLLMs reason over camera-captured views of the physical world, a new attack surface emerges: an attacker can place text-bearing physical objects in the environment to override MLLMs' intended task. While prior work has studied
prompt injection in the text domain and through digitally edited 2D images, limited
attention has been paid to how these attacks function in 3D environments. To bridge the gap, we introduce PI3D, a
prompt injection attack against MLLMs in 3D environments, realized through text-bearing object placement rather than digital image edits. We formulate and solve the problem of identifying an effective pose (position and orientation) for a 3D object with injected text, where the attacker's goal is to induce the MLLM to perform the injected task while ensuring that the object placement remains physically plausible. Experiment results demonstrate that PI3D is an effective attack against multiple MLLMs under diverse camera trajectories. We further evaluate a range of defenses and show that they are not sufficient to reliably defend against PI3D.
Read the full abstract ▾
Computer Vision
Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting
Zinan Lv, Yeqian Qian, Chen Sang et al. (+3 more)
UAV navigation in unstructured outdoor environments using passive
monocular vision is hindered by the substantial visual domain gap between simulation and reality. While 3D Gaussian Splatting enables photorealistic scene reconstruction from real-world data, existing methods inherently couple static lighting with geometry, severely limiting policy generalization to dynamic real-world illumination. In this paper, we propose a novel end-to-end reinforcement learning framework designed for effective zero-shot transfer to unstructured outdoors. Within a high-fidelity simulation grounded in real-world data, our policy is trained to map raw
monocular RGB observations directly to continuous control commands. To overcome photometric limitations, we introduce Relightable 3D Gaussian Splatting, which decomposes scene components to enable explicit, physically grounded editing of environmental lighting within the neural representation. By augmenting training with diverse synthesized lighting conditions ranging from strong directional sunlight to diffuse overcast skies, we compel the policy to learn robust, illumination-invariant visual features. Extensive real-world experiments demonstrate that a lightweight quadrotor achieves robust, collision-free
navigation in complex forest environments at speeds up to 10 m/s, exhibiting significant resilience to drastic lighting variations without fine-tuning.
Read the full abstract ▾
All authors Zinan Lv, Yeqian Qian, Chen Sang, Hao Liu, Danping Zou, Ming Yang
Computer Vision
TLC-Plan: A Two-Level Codebook Based Network for End-to-End Vector Floorplan Generation
Biao Xiong, Zhen Peng, Ping Wang et al. (+2 more)
Automated floorplan generation aims to improve design quality, architectural efficiency, and sustainability by jointly modeling global spatial organization and precise
geometric detail. However, existing approaches operate in raster space and rely on post hoc vectorization, which introduces structural inconsistencies and hinders end-to-end learning. Motivated by compositional spatial
reasoning, we propose TLC-Plan, a hierarchical generative model that directly synthesizes vector floorplans from input boundaries, aligning with human architectural workflows based on modular and reusable patterns. TLC-Plan employs a two-level VQ-VAE to encode global layouts as semantically labeled room bounding boxes and to refine local geometries using polygon-level codes. This hierarchy is unified in a CodeTree representation, while an autoregressive
transformer samples codes conditioned on the boundary to generate diverse and topologically valid designs, without requiring explicit room
topology or dimensional priors. Extensive experiments show state-of-the-art performance on RPLAN
dataset (FID = 1.84, MSE = 2.06) and leading results on LIFULL
dataset. The proposed framework advances constraint-aware and scalable vector floorplan generation for real-world architectural applications. Source code and trained models are released at https://github.com/rosolose/TLC-PLAN.
Read the full abstract ▾
All authors Biao Xiong, Zhen Peng, Ping Wang, Qiegen Liu, Xian Zhong
Computer Vision
WorldEdit: Towards Open-World Image Editing with a Knowledge-Informed Benchmark
Wang Lin, Feng Wang, Majun Zhang et al. (+7 more)
Recent advances in image editing models have demonstrated remarkable capabilities in executing explicit instructions, such as attribute manipulation, style transfer, and pose synthesis. However, these models often face challenges when dealing with implicit editing instructions, which describe the cause of a visual change without explicitly detailing the resulting outcome. These limitations arise because existing models rely on uniform editing strategies that are not equipped to handle the complex world knowledge and
reasoning required for implicit instructions. To address this gap, we introduce WorldEdit, a
dataset specifically designed to enable world-driven image editing. WorldEdit consists of high-quality editing samples, guided by paraphrased instructions that align with real-world causal logic. Furthermore, we provide WorldEdit-Test for evaluating the existing model's performance on causal editing scenarios. With WorldEdit, we use a two-stage training framework for fine-tuning models like Bagel, integrating with a causal verification reward. Our results show that the proposed
dataset and methods significantly narrow the gap with GPT-4o and Nano-Banana, demonstrating competitive performance not only in instruction following but also in knowledge plausibility, where many open-source systems typically struggle.
Read the full abstract ▾
All authors Wang Lin, Feng Wang, Majun Zhang, Wentao Hu, Tao Jin, Zhou Zhao, Fei Wu, Jingyuan Chen, Alan Yuille, Sucheng Ren