Computer Vision · Cross-listed from cs.CL
Haoxiang Sun, Tao Wang, Li Yuan et al. (+2 more)
Multimodal Large
Language Models (MLLMs) have recently made remarkable progress in unifying
vision-
language understanding and
reasoning, especially following the introduction of models such as OpenAI's O-series and DeepSeek's R-series, which have driven a paradigm shift toward perception-centric intelligence. However, there remains a lack of systematic surveys that examine perception from a truly unified
vision-
language perspective – one that treats
vision and
language as an inseparable modality. Existing reviews are often fragmented, focusing separately on either
vision or
language, and thus rarely capture the cross-modal
evolution of perception as an integrated capability. To bridge this gap, we present the first systematic survey of unified
vision-
language perception in MLLMs. Specifically, we (1) formalize MLLM perception as an intrinsic, unified
vision-
language capability analogous to human innate perception, (2) introduce a five-stage taxonomy tracing the paradigm
evolution of MLLM perception and survey representative methods and milestones at each phase, and (3) identify open challenges and outline promising research directions toward truly general, unified
multimodal intelligence. We hope our study will provide both a foundational understanding and an actionable roadmap to foster further innovation on the path toward artificial general intelligence (AGI).
Read the full abstract ▾
All authors Haoxiang Sun, Tao Wang, Li Yuan, Jian Zhao, Jiancheng Lv
Computer Vision
Self-Supervised Tree-level Biomass Estimation in Urban Environments From Airborne LiDAR and Optical Observations
Jose Bermudez, Zilong Zhong, Dominic Cyr et al. (+2 more)
Urban tree biomass remains less spatially explicitly quantified than biomass in managed forests because many estimates rely on inventories or coarse products that cannot resolve individual crowns or fine-scale heterogeneity. We present a crown-level above-ground biomass (AGB) framework for an 810~km$^2$ landscape in Ontario, Canada, using leaf-off airborne
LiDAR (8–10~pulses~m$^{-2}$) and near-infrared RGB orthophotography (0.16–0.20~m) from 2018 and 2023. A dual-stream cross-
attention network trained on rule-based pseudo-labels produced semantic marks for buildings, needleleaf trees, and deciduous trees, supporting crown delineation and functional-type assignment. On independently annotated withheld tiles, global/mean precision, recall, and Dice scores were 0.86, 0.83, and 0.84. Crowns were delineated with multiscale watershed
segmentation in mapped tree areas, and AGB was estimated from a crown area–height power-law proxy calibrated to species-specific allometry (Lambert et al., 2005) for 21,921 inventory trees. For 18,713 inventory–segment matched pairs from a 90,726-tree held-out test set, AGB prediction achieved $R^2=0.609$ using inventory crown geometry and $R^2=0.570$ under operational
segmentation, identifying crown delineation as the remaining uncertainty source. Aggregated to 30~m, estimates yielded total AGB stocks of 1.73~Tg in 2018 and 1.81~Tg in 2023 (811–850~Gg~C), local densities up to ${\sim}140$~Mg~ha$^{-1}$ along the Niagara Escarpment, and a net carbon gain of 39~Gg~C over five years. Deep-ensemble uncertainty maps highlighted high-epistemic-uncertainty areas linked to underrepresented land covers and guided assignment of uncertain crowns to a pooled allometric equation. The framework uses standard provincial data, requires no manual annotation, and produces a public bitemporal crown-level AGB database for trees outside forests at management-relevant resolution.
Read the full abstract ▾
All authors Jose Bermudez, Zilong Zhong, Dominic Cyr, Camile Sothe, Alemu Gonsamo
Computer Vision
LCG: Long-Context Consistent Image Generation with Sparse Relational Attention
Zihao Wang, Yijia Xu, Haoze Zheng et al. (+3 more)
Recent image generation models achieve impressive quality in single-image synthesis, but often fail to maintain consistency across sequential outputs, as required in comics, storyboards, and visual narratives. We propose Long-Context Generation (LCG), a framework for long-context multi-image text-to-image generation, to improve consistency and scalability in long-context multi-image generation. LCG employs the Sparse Relational
Attention (SRA) mechanism to selectively attend to core features across extended visual contexts, ensuring that the propagation of semantic and layout information remains computationally tractable. To enforce semantic
alignment, we introduce the Routing Consistency Constraint (RCC), which leverages identity-aware masks to align structural patterns across generation branches, effectively mitigating drift in appearance even in complex multi-character scenes. To support training and evaluation in this setting, we construct the Long-Context Consistency
Dataset (LCCD), a large-scale synthetic
dataset comprising character-centric multi-image sequences spanning varied situational contexts. LCCD contains 600K training sequences and a separate 1K test set, with each sequence containing 6 to 20 images. The experiments demonstrate that LCG outperforms the compared baselines in
prompt alignment and character consistency for long-context image generation, including multi-character scenes.
Read the full abstract ▾
All authors Zihao Wang, Yijia Xu, Haoze Zheng, Xuran Ma, Haokun Gui, Harry Yang
Computer Vision
Predicting Fruit Quality with a Hybrid Machine Learning and Image Processing Approach
Amir Reza Hashemi and Shahram Amiri
Fruit spoilage is a significant issue in agriculture, leading to substantial economic losses. Addressing this, our study introduces a hybrid approach combining image processing and deep learning to assess fruit freshness. We developed an image processing
algorithm that quantifies spoilage on a scale from 0 (fully fresh) to 100 (fully rotten). Alongside, we trained a convolutional neural network (CNN) to perform binary classification (fresh or rotten) using a large
dataset of fruit images. The outcomes of both methods were synthesized using logistic regression to enhance the accuracy of freshness predictions. Subsequently, this logistic regression model was utilized to enable the image processing
algorithm to provide binary classification based on its percentage output, thus eliminating the need for the CNN in real-time applications. Our approach, which does not require high computational resources, achieved real-time performance and was validated with over 90% accuracy on a
dataset comprising apples and oranges. The primary limitation lies in the requirement for fruits to be isolated on a background that must be either white or transparent, suggesting future improvements could include advanced
segmentation models to automate background removal. This study's results highlight the potential of integrating simple image processing techniques with machine learning to provide practical solutions in the agricultural sector.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
Learning Action Priors for Cross-embodiment Robot Manipulation
Dong Jing, Tianqi Zhang, Jiaqi Liu et al. (+5 more)
Most
Vision-
Language-Action (VLA) models build on a
Vision-
Language Model (VLM) backbone by attaching an action module and optimizing the full policy jointly. This design inherits strong visual and linguistic priors from the VLM, but leaves the action module to learn physical motion almost from scratch. As a result, the policy lacks an explicit motion prior, forcing early optimization to simultaneously discover temporal action dynamics and cross-modal
alignment, a challenge further amplified in cross-embodiment settings. In this work, we propose to pretrain the action module with motion priors before cross-modal VLA
alignment. Specifically, we introduce a two-stage training framework that equips the action module with cross-embodiment temporal motion structure before VLA training begins. In Stage~1, a lightweight flow-matching-based encoder-decoder action module efficiently learns temporal motion structure solely from unconditioned action trajectories, without processing visual or
language tokens. In Stage~2, this learned prior is transferred to VLA training through decoder reuse and early-stage latent
distillation, aligning visual-
language features with the action
embedding space while still allowing end-to-end policy refinement. In addition, the trained encoder serves as a compact history compressor, summarizing state-action histories into a single temporal context
token for history-aware modeling at negligible cost. Extensive experiments across 13 diverse cross-embodiment tasks on both simulated and real-world platforms validate the effectiveness of our approach. Compared with VLA training without action priors, our model achieves faster convergence, higher success rates, and substantially stronger performance on data-scarce real-world tasks. Moreover, scaling up the action data in Stage~1 yields a more generalizable action prior that directly improves downstream VLA performance.
Read the full abstract ▾
All authors Dong Jing, Tianqi Zhang, Jiaqi Liu, Jinman Zhao, Zelong Sun, Li Erran Li, Zhiwu Lu, Mingyu Ding
Computer Vision
TryOnCrafter: Unleashing Camera Trajectories for Realistic Video Virtual Try-on via a Renderable 4D Try-on Proxy
Hao Sun, Hao Yan, Mengting Chen et al. (+7 more)
While
Video Virtual Try-on (VVT) has achieved remarkable progress in synthesizing realistic garment overlays on dynamic subjects, existing paradigms remains fundamentally constrained by a passive dependency on source camera trajectories, failing to accommodate the requisite interactive freedom for omnidirectional viewpoint exploration. To address this limitation, we define a pioneering research frontier: Camera-controllable
Video Virtual Try-on (CaM-VVT). Unlike conventional VVT, CaM-VVT not only necessitates viewpoint-agnostic texture
hallucination but also strict structural synchronization between non-rigid human dynamics and background contexts under arbitrary, unconstrained camera movements. To tackle these challenges, we present TryOnCrafter, the first unified DiT-based framework specifically architected for the CaM-VVT task. Departing from implicit pixel-space manipulation, we introduce a Renderable 4D Try-on Proxy that explicitly decouples the human subject from the environment. This is achieved by distilling high-fidelity 2D try-on priors into a clothed 3DGS-based avatar, which is subsequently animated via SMPL-X sequences and metric-aligned into a reconstructed background point cloud. This proxy establishes a robust structural foundation with superior texture density and motion integrity. Our Proxy-Anchored
Video DiT leverages this robust structural foundation as a primary
geometric anchor, ensuring that the synthesized photorealistic
videos are strictly constrained by prescribed trajectories and physically plausible deformations. Benefiting from the inherent editability of the 4D proxy, TryOnCrafter facilitates diverse downstream applications, including human relocalization, “bullet time” effects, and $360$-degree orbital viewing.
Read the full abstract ▾
All authors Hao Sun, Hao Yan, Mengting Chen, Quanjian Song, Yu Li, Juan Cao, Jinsong Lan, Xiaoyong Zhu, Bo Zheng, Sheng Tang
Computer Vision
MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation
JoungBin Lee, Jaewoo Jung, Jongmin Lee et al. (+8 more)
Synthesizing a novel-view
video from a
monocular reference
video along a target camera trajectory requires both
geometric consistency and motion fidelity with respect to the reference
video. Existing methods based on explicit 3D representations are limited by the accuracy of off-the-shelf reconstruction modules, which often produce inaccurate geometry for dynamic objects in
monocular videos. In contrast, camera-conditioning-only methods can achieve high visual quality but often struggle to preserve
geometric and motion consistency. In this work, we introduce MVTrack4Gen (Multi-View point Tracking for Novel-View Generation), a motion-aware training framework that leverages multi-view point tracking as an additional
geometric and motion supervision signal for camera-conditioning-only novel-view
video diffusion models. Our key finding is that specific
attention layers encode strong correspondence cues, where query features attend to key features at geometrically corresponding locations across views and over time, and the misalignment of these correspondences causes motion inconsistency. Based on this observation, we route these features into an auxiliary multi-view tracking head and jointly train the
diffusion model with a point-tracking objective. By explicitly strengthening these motion-aware correspondences, MVTrack4Gen improves existing models to better follow the motion in the reference view and maintain cross-view
geometric consistency. Across diverse
benchmarks, our method achieves state-of-the-art
geometric consistency and competitive camera accuracy.
Read the full abstract ▾
All authors JoungBin Lee, Jaewoo Jung, Jongmin Lee, Tongmin Kim, Hyunsung Kim, Takuya Narihira, Kazumi Fukuda, Jahyeok Koo, Jisang Han, Yuki Mitsufuji, Seungryong Kim
Computer Vision · Cross-listed from cs.CL
Same Evidence, Different Answer: Auditing Order Sensitivity in Multimodal Large Language Models
Akshay Paruchuri, Sanmi Koyejo and Ehsan Adeli
Standard
benchmarks for
multimodal large
language models (MLLMs) score each item on one canonical ordering and miss whether order-irrelevant shuffling changes the answer, a baseline reliability property called for by emerging AI evaluation guidelines. We introduce Facet-Probe, a five-facet audit (option, evidence-chunk, document-rank, image-set, and mixed-modality ordering) of 18 frontier and open-weight MLLMs. A
Bayesian item-response model separates ordering noise from per-facet bias, and a same-ordering control estimates the decoder-
stochastic floor for observed flips. We find that none of the 18 MLLMs we audit are order-invariant: screened per-facet panel-mean flip rates span 24-50%. A Gemini same-ordering control at temperature 0 estimates a substantial ordering excess over a same-input decoder-noise floor in verified
cells. Capability predicts but does not eliminate flips; the best model still flips on 13.4% of trials. In our Gemini mitigation tests, training-free
prompt changes are modality-conditional and do not transfer from text to visual
reasoning. These results suggest that
prompt-level mitigation alone is unlikely to provide general order robustness, motivating future work on training-time and architectural approaches. We propose cross-ordering flip rate as a standard reporting axis for MLLMs.
Read the full abstract ▾
Computer Vision
A cross-process welding penetration status prediction algorithm based on unsupervised domain adaptation in laser and TIG welding
Sen Li, Haichao Cui, Chendong Shao et al. (+2 more)
Supervised deep learning has been widely used for weld penetration state classification; however, its performance often degrades significantly under domain shift, such as when transferring models between welding processes with distinct physical mechanisms:for instance, from arc-dominated tungsten inert gas (TIG) welding to keyhole-based
laser welding. To overcome this limitation, we propose an unsupervised domain adaptation (UDA) framework integrated with a gradual source domain expansion (GSDE) strategy. Evaluated on dedicated TIG and
laser welding
datasets, our approach achieves high accuracy in both same-process and cross-process transfer tasks. Specifically, it attains average accuracies of 90.65% on TIGFH and 90.72% on LSPS in same-process settings, surpassing a supervised baseline by 35.83% and 38.87%, respectively. More notably, in cross-process scenarios, it reaches 80.48% for TIG to
Laser and 81.13% for
Laser to TIG, improving upon the baseline by 43.39% and 43.40%. UMAP visualizations verify that the model learns domain-invariant features while maintaining discriminative class boundaries. This method considerably lowers the relabeling cost for new welding processes and enhances the versatility of intelligent monitoring across different welding systems.
Read the full abstract ▾
All authors Sen Li, Haichao Cui, Chendong Shao, Yaqi Wang, Xinhua Tang
Computer Vision
A welding penetration prediction model for laser welding process based on self-supervised learning using physics-informed neural networks
Sen Li, Xiaoying Liu, Xiaojian Xu et al. (+5 more)
The
laser welding full-penetration is of critical importance, as it constitutes one of the fundamental factors in achieving defect-free welded joints. Accurate prediction of the penetration state is therefore essential for ensuring weld quality. To this end, this paper introduces SimPhysNet, a novel
algorithm that achieves high classification accuracy in
laser welding penetration prediction using only a limited number of labelled images. This approach effectively overcomes the limitations of supervised learning classification
algorithms, which are hindered in industrial applications by their dependence on extensive, high-quality labelled data. The core of SimPhysNet is a unique self-supervised learning paradigm that embeds physical priors into a contrastive learning framework. By incorporating a physics-informed neural network (PINN), the model is guided to extract physically meaningful features of the molten pool and keyhole from a large set of unlabelled data, while three image augmentation tasks further enhance its generalization capabilities. Subsequently, a few-shot learning strategy, based on prototypical networks, enables robust classification by constructing class representations from a minimal set of labelled images. Experimental results demonstrate that SimPhysNet achieves a classification accuracy of 96.06% using only 200 labelled images (approximately 5% of the total labelled
dataset), which is comparable to the performance of conventional supervised learning
algorithms that utilize the entire labelled
dataset. This work presents a new, efficient, and highly accurate method, providing the way for the intelligent automation of
laser welding.
Read the full abstract ▾
All authors Sen Li, Xiaoying Liu, Xiaojian Xu, Chendong Shao, Yaqi Wang, Ling Lan, Xinhua Tang, Haichao Cui
Computer Vision
DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
Nan Chen, Yiyang Cai, Rongchang Xie et al. (+7 more)
Open domain subject-driven text-to-
video (S2V) generation has drawn significant interest in academia and industry. Open domain S2V mainly involves two scenarios: in-domain, which requires retaining the reference subject features as much as possible, and cross-domain, which preserves the intrinsic features of the subject while allowing subject-irrelevant properties to vary flexibly according to the text
prompt. Existing methods primarily focus on maximizing subject fidelity in in-domain scenarios, which limits their editability and adaptability in cross-domain scenarios, such as novel styles, semantic combinations, or domain attributes. In this study, we propose that an ideal S2V method should flexibly shuttle between different domains, achieving strong performance in both in-domain and cross-domain scenarios. To this end, we propose DomainShuttle, which could achieve high fidelity and generative flexibility for open domain
video personalization. Specifically, we introduce Domain-MoT, which decouples
videos and reference features and introduces the domain-aware AdaLN for domain-specific modeling of reference images. We then introduce the
Video-Reference DualRoPE scheme, which places reference image
tokens and
video tokens in separate RoPE spaces to enable precise subject-level spatial modeling, and Cross-Pair Consistent Loss, which aims to extract intrinsic subject features unaffected by irrelevant features. Extensive experiments demonstrate that DomainShuttle achieves significant performance improvements over existing methods, exhibiting high subject fidelity and generative flexibility across diverse open domain application scenarios.
Read the full abstract ▾
All authors Nan Chen, Yiyang Cai, Rongchang Xie, Junwen Pan, Cheng Chen, Weinan Jia, Zhuowei Chen, Wen Zhou, Zhenbang Sun, Wenhan Luo
Computer Vision · Cross-listed from cs.RO
RoboAtlas: Contextual Active SLAM
Alexander Schperberg, Shivam K. Panda, Abraham P. Vinod et al. (+3 more)
We present RoboAtlas, a contextual Active SLAM framework that adaptively balances
geometric exploration and semantic
reasoning using a scalable 3D semantic mapping system, OpenRoboVox. RoboAtlas integrates frontier exploration, global semantic-map
reasoning, and
egocentric VLM-based
reasoning through a contextual multi-armed bandit that transitions from exploration to semantically guided
navigation as scene understanding improves. We evaluate the system in simulation and on a Unitree Go2
robot in large-scale real-world environments exceeding 1800 m2 with approx. 30k mapped semantic instances, achieving a 100% task success rate. On the GOAT-Bench "Val Unseen"
benchmark, RoboAtlas achieves state-of-the-art performance with highest reported success rate (SR) of 90.6%, using GPT-4o, improving over the strongest prior baseline by 17.8 percentage points in SR. Using the much smaller Qwen2.5-VL-7B model, it still achieves 88.8% SR, outperforming all baselines using GPT-4o in SR, and revealing the importance of the information gained by our semantic mapping framework over simply replacing the underlying foundation model. The results demonstrate that grounding foundation models with large-scale 3D semantic maps enables robust and efficient contextual Active SLAM.
Read the full abstract ▾
All authors Alexander Schperberg, Shivam K. Panda, Abraham P. Vinod, Bhagawan Rokaha, M. K. Jawed, Stefano Di Cairano
Computer Vision
How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations
Yuxing Cheng, Yuan Wu and Yi Chang
Vision-
language models (VLMs) have achieved strong performance on OCR-based
benchmarks and increasingly focused on text-rich understanding, but their robustness under controlled visual degradation remains insufficiently understood. This gap is critical for OCR
reasoning, where visual corruption can induce OCR errors and structural distortions, thereby introducing uncertainty into the
reasoning task. To systematically study this problem, we introduce OCR-Robust, a
benchmark designed for evaluating OCR
reasoning robustness under visual perturbations. It contains 812 samples across two complementary subsets: OCR1.0, covering documents, scene text, receipts, handwriting, and mathematical content, and OCR2.0, focusing on charts, geometry diagrams, and tables. To enable efficient yet informative evaluation, we conduct a pilot study over 18 candidate perturbations and select 5 representative types at 3 severity levels each based on their impact and cross-model discriminability. We evaluate robustness using clean accuracy, Relative Corruption Retention (RCR), Worst-Case Retention (WCR), and a composite Corruption Robustness Index (CRI), and
benchmark 18 models spanning proprietary systems, open-source VLMs, and OCR+LLM pipelines. Our results show that higher clean accuracy does not necessarily imply stronger robustness, and that models can suffer pronounced degradation in the worst case on OCR tasks that are sensitive to structure, and charts and tables are substantially more fragile than document-like inputs under perturbation.
Read the full abstract ▾
Computer Vision · Cross-listed from stat.ML
FedReLa: Imbalanced Federated Learning via Re-Labeling
Guangzheng Hu, Patricia Menéndez, Feng Liu et al. (+3 more)
Federated learning has emerged as the foremost approach for decentralized model training with
privacy preservation. The global class imbalance and cross-client data heterogeneity naturally coexist, and the mismatch between local and global imbalances exacerbates the performance degradation of the aggregated model. The agnosticism of global class distribution poses significant challenges for data-level methods, especially under extreme conditions with severe class absence across clients. In this paper, we propose FedReLa, a novel data-level approach that tackles the coexistence of data heterogeneity and class imbalance in
federated learning. By re-labeling samples with a feature-dependent label re-allocator, FedReLa corrects biased global decision boundaries without requiring knowledge of the global class distribution. This modular, model-agnostic approach can be integrated with algorithmic methods to deliver consistent improvements without additional communication overhead. Through extensive experiments, our method significantly improves the accuracy of minority classes and the overall accuracy on stepwise-imbalanced and long-tailed
datasets, outperforming the previous state of the art.
Read the full abstract ▾
All authors Guangzheng Hu, Patricia Menéndez, Feng Liu, Mingming Gong, Guanghui Wang, Liuhua Peng
Computer Vision
TriViewBench: Controlled Complexity Scaling for Multi-View Structural Reasoning in MLLMs
Yu-Yang Chen and Lan-Zhe Guo
Multimodal Large
Language Models (MLLMs) demonstrate strong performance on standard visual question answering
benchmarks, yet their scalability under controlled structural complexity remains poorly understood. We introduce TriViewBench, a controlled three-view visual
reasoning benchmark constructed from synthetic 3D scenes with explicitly parameterized object count and occlusion. The
benchmark contains 1,923 scenes and over 14K Question-Answer (QA) pairs organized into four complexity levels and three
reasoning categories: Local Decision, Object Counting, and Global Recovery. We evaluate 18 open- and closed-source MLLMs under a unified prompting protocol. All 18 models exhibit an identical capability hierarchy without exception (Local Decision > Object Counting > Global Recovery), and performance degrades monotonically with complexity: Local Decision tasks decline modestly (12.11% relative drop), while Object Counting degrades substantially (59.14%) and Global Recovery collapses severely (80.02%). Error analysis on Object Counting reveals two mechanistically independent failure modes: single-view tasks are dominated by undercounting due to occlusion blindness, whereas the multi-view task reverses to overcounting due to cross-view identity confusion. Chain-of-Thought (CoT) prompting yields near-zero overall benefit ($\Delta = -0.16\%$) and its effect on Global Recovery is strongly capability-gated, suggesting that the bottleneck lies in cross-view spatial representation rather than
reasoning strategy. These findings reveal fundamental scalability limitations in current MLLMs and position TriViewBench as a controlled diagnostic framework for analyzing structural
reasoning failures.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.RO
In-Context World Modeling for Robotic Control
Siyin Wang, Junhao Shi, Senyu Fei et al. (+4 more)
Modern
Vision-
Language-Action (VLA) models often fail to generalize to novel setups, such as altered camera viewpoints or
robot morphologies, because they are typically conditioned only on current observations and
language instructions. By ignoring the underlying system configuration as a variable, these models implicitly assume a fixed execution context encountered during training, necessitating data-intensive fine-tuning for any new environment. In this work, we introduce In-Context World Modeling (ICWM), a framework that treats system identification as an in-context adaptation problem. ICWM enables
robot policies to autonomously infer essential system variables from a short history of self-generated, task-agnostic interactions. Unlike traditional In-Context Learning that uses demonstrations to specify what task to perform, ICWM leverages the context window to understand how the system operates. By processing these interactions before task execution, the model implicitly captures the world dynamics of the current system, enabling adaptation to novel configurations without parameter updates. Extensive experiments in simulation and on real-world
robot platforms demonstrate that ICWM significantly outperforms standard VLA baselines on novel camera viewpoints.
Read the full abstract ▾
All authors Siyin Wang, Junhao Shi, Senyu Fei, Zhaoyang Fu, Li Ji, Jingjing Gong, Xipeng Qiu
Computer Vision
MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation
Yang Chen, Xiaowei Xu, Shuai Wang et al. (+4 more)
Normalizing Flows (NFs) are powerful generative models capable of exact density estimation and sampling. However, their strict invertibility often forces the model to exhaust its capacity on low-level pixel details, hindering the capture of high-level semantic structures. While Masked Image Modeling (MIM) has excelled in representation learning, its integration into generative pipelines has remained largely modular and disjointed. In this paper, we propose MIMFlow, a unified end-to-end framework that jointly optimizes latent semantics, pixel reconstruction, and generative flow. By employing a VAE encoder to infer semantic latent from masked images, MIMFlow achieves a principled decoupling of the generative task: the Normalizing Flow focuses on modeling a simplified, low-frequency semantic
manifold, while a specialized decoder handles high-frequency synthesis. This design effectively resolves the inherent capacity bottleneck of NFs, allowing the model to prioritize global structural coherence over redundant noise. Empirical results on ImageNet 256$\times$256 show that MIMFlow-L reaches 71.3% linear probing accuracy and an FID of 2.50. Despite using only 128
tokens (50% fewer than standard models), it yields a 32.8% performance gain over similar-scale NF baselines. Our code is available at https://github.com/MCG-NJU/MIMFlow.
Read the full abstract ▾
All authors Yang Chen, Xiaowei Xu, Shuai Wang, Xinwen Zhang, Qiushi Guo, Tiezheng Ge, Limin Wang
Computer Vision
From Sparse and Imperfect 2D Anchors to Consistent 3D Gaussian Street Scenes: Support-Aware Appearance
Long Cao, Zhongquan Wang, Jie Li et al. (+4 more)
Image priors can synthesize target conditions for 3D Gaussian street scenes, but independently edited views do not define a coherent 3D target. Direct fitting can propagate view-specific noise, while existing pipelines do not jointly handle imperfect sparse anchors and standard-rasterizer deployment. To address this gap, teacher-relative appearance residual
distillation is introduced for appearance baking. A structured space for frequency decomposition, confidence estimation, and primitive-level lifting is formed by residuals between teacher anchors and original renders. The direct optimization signal is supplied by renderer-space matching, while primitive assignment is regularized by support-aware Gaussian-space aggregation. Supported detail is admitted and unsupported noise is suppressed through confidence-gated coarse-to-fine optimization, after which all residuals are baked into fixed-geometry spherical-harmonic coefficients. The teacher and auxiliary training modules are discarded at inference. Evaluation across Waymo street assets, Tanks and Temples scenes, and multiple target conditions shows a favorable overall balance of target
alignment, content preservation, artifact suppression, and cross-view consistency over editing-based baselines. Ablations confirm the effectiveness of the main components. Code will be released at https://github.com/Cagares/Baking-for-3D-Gaussian.
Read the full abstract ▾
All authors Long Cao, Zhongquan Wang, Jie Li, Yuhan Chen, Kefei Qian, Xiangfei Huang, Guofa Li
Computer Vision
Taxonomy-aware deep learning for hierarchical marine species classification in underwater imagery
Dan Zimmerman, Dimitris A. Pados and George Sklivanitis
Automated classification of marine species from
underwater imagery is essential for scalable ocean biodiversity monitoring and conservation policy. Existing approaches struggle with severe domain shift across collection platforms, fine-grained visual similarity between closely related species, and uneven annotation granularity, where many specimens can only be identified to genus or a coarser taxonomic rank. We present a taxonomy-aware deep learning framework that aligns both the training loss and the inference rule with the hierarchical structure of biological classification, combining a taxonomy-weighted loss, minimum-risk
Bayesian inference, multi-scale feature encoding, and independent per-rank classification heads. Evaluated on the FathomNet 2025
dataset1 (79 marine classes across seven taxonomic ranks), the system achieves a mean taxonomic distance of 1.581, within 3% of the 1st-place solution (1.535), with the largest gains from metric-aligned inference and simple, decoupled components that generalize better than learned dependencies under distribution shift.
Read the full abstract ▾
Computer Vision · Cross-listed from cs.LG
Tensorion: A Tensor-Aware Generalization of the Muon Optimizer
Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko et al. (+2 more)
Common first-order optimizers, such as Adam, implicitly treat each parameter block as an unstructured vector, which disregards the multilinear weight structure present in many modern machine learning models. Recent work has shown that exploiting
matrix structure can improve optimization dynamics. A notable example is
Muon, which performs steepest descent under the spectral norm constraint. We take the next step and introduce Tensorion, a tensor-aware optimizer that extends
Muon's constrained optimization perspective from
matrices to higher-order tensors. Tensorion is built around a linear minimization oracle (LMO) over a tensor norm ball. The norm is carefully chosen to balance two objectives: tightly bounding the tensor spectral norm, while still keeping the LMO tractable. This LMO becomes computable because it reduces to operations on adaptively selected unfolding
matrices. Notably, when restricted to order-2 tensors (i.e.,
matrices), Tensorion recovers
Muon exactly. Experiments on tensor-based computer
vision problems suggest that Tensorion can offer improved convergence behavior and more stable gradient updates compared with Adam-based and existing tensor-aware baselines in the evaluated settings.
Read the full abstract ▾
All authors Vladimir Bogachev, Vladimir Aletov, Alexander Molozhavenko, Sergei Kudriashov, Maxim Rakhuba
Computer Vision
A Benchmark for Heterogeneous Stereo Deblurring with Physically- and Epipolar-constrained Cross Attention
Hoju Shin, Jiah Kim, Seung-Wook Kim et al. (+1 more)
Modern stereo-capable smartphones enable immersive XR content capture. However, hardware heterogeneity across camera modules often causes severe asymmetric blur artifacts. Existing methods and
benchmarks largely assume homogeneous stereo setups and therefore do not explicitly address such asymmetric degradation. To bridge this gap, we present a dedicated framework for heterogeneous stereo deblurring. First, we introduce the heterogeneous stereo deblurring (HSD)
dataset, constructed from real smartphone stereo captures via multi-frame integration. Second, we propose physically- and epipolar-constrained cross
attention (PECA), a lightweight module that restricts cross-view matching to an epipolar search window bounded by a optics-derived disparity upper bound. By enforcing physically valid disparity constraints, PECA enables efficient and reliable cross-view feature fusion. Moreover, our confidence-weighted
attention with residual fusion emphasizes cross-guided deblurring when correspondences are reliable, while naturally falling back to self-deblurring in occluded or unreliable regions. PECA is architecture-agnostic and consistently improves CNN-,
Transformer-, and NAFNet-based baselines. Extensive experiments on HSD show that PECA-enhanced models achieve improved restoration performance with favorable efficiency.
Read the full abstract ▾
All authors Hoju Shin, Jiah Kim, Seung-Wook Kim, Seowon Ji
Computer Vision
Pulmonary Embolism Risk Stratification from CTPA and Medical Records: Vascular Graphs Are Not All You Need
Nathan Painchaud, Tristan Habémont, Morgane des Ligneris et al. (+6 more)
Risk stratification for pulmonary embolism (PE) is critical for clinical decision-making. Stratification guidelines are based on patient medical records, parameters measured from computed
tomography pulmonary angiography (CTPA), and blood tests. However, blood tests are often missing in routine practice. This work studies whether state-of-the-art models can accurately classify risk stratification from only medical records and biomarkers extracted from CTPA images. We
benchmark different approaches to combine medical records and cardiac biomarkers with rich pulmonary vascular information; we add vascular biomarkers to tabular models and apply graph neural networks (GNNs) on the vascular tree's intrinsic graph representation. We use a private
dataset (n=353) with uniquely complete data for PE risk stratification. Our results show that, among global features, medical records and cardiac biomarkers are the most significant predictors, while vascular biomarkers do not further improve stratification. Even more surprising, even GNNs on vascular graphs fail to outperform strong tabular baseline on global features. We consider hypotheses, on both models and data, that could explain this suboptimal performance. Our investigation suggests that, counter-intuitively, vascular graphs might hold no discriminative information for PE risk stratification. Code is available from https://github.com/creatis-myriad/GENESIS.
Read the full abstract ▾
All authors Nathan Painchaud, Tristan Habémont, Morgane des Ligneris, Allan Serva, Pierre Croisille, Laurent Bertoletti, Thomas Lampert, Johannes F. Lutzeyer, Odyssée Merveille
Computer Vision · Cross-listed from cs.RO
DSP-SLAM++: A Unified Framework for Multi-Class, High-Fidelity Object SLAM in the Wild
Ahmad Kourani, Ghina Daoud, Daniel Asmar et al. (+1 more)
Existing object-aware SLAM systems force a trade-off between real-time performance, multi-class support, and the generation of high-fidelity, semantically coherent object models. To address this trade-off, we present DSP-SLAM++, which extends the DSP-SLAM framework with an asynchronous mapping pipeline for real-time performance and dedicated sensor fusion adaptations for a
monocular fisheye-
LiDAR suite. Experiments demonstrate that our system generates fine-grained, geometrically-complete shapes for multiple object classes while eliminating severe mapping thread bottlenecks by reducing maximum object processing latency by up to 70% compared to the state-of-the-art baseline, enabling robust, real-time performance on a challenging 25 Hz multi-class
datasets. This work makes high-fidelity, multi-class object SLAM more practical for real-world applications like autonomous driving and robotic manipulation by enabling its use on platforms with common fisheye-
LiDAR sensor setups. The open-source code is available at: [github.com/AUBVRL/DSP-SLAMpp].
Read the full abstract ▾
All authors Ahmad Kourani, Ghina Daoud, Daniel Asmar, Imad Elhajj
Computer Vision
FunPiQ: A New Benchmark for Pixel-Level Quality Assessment in Fundus Images
Pengwei Wang, José Morano, Virginia Mares et al. (+1 more)
Color fundus photography (CFP) is the most common ophthalmic imaging modality for large-scale screening. However, it is highly susceptible to degradations, making robust fundus image quality assessment (FIQA) crucial. The criteria for what constitutes high-quality at the image level vary across clinical tasks, making FIQA dependent on expert knowledge. This motivated the development of automated methods and
datasets. While existing
datasets aim to standardize image-level quality, their criteria often differ. Furthermore, image-level labels preclude the quantitative evaluation of localized degradations, which is essential for trustworthy FIQA. We argue that pixel-level FIQA based on anatomical visibility represents a more task-agnostic, explainable approach. In this work, we introduce FunPiQ, the first FIQA
benchmark to provide pixel-level quality annotations. In addition, we propose EFIQA-CP, an explainable-by-design (EBD) method that uses quality pseudo-labels based on anatomical visibility to train a CNN via Non-Negative Positive-Unlabeled learning. Extensive evaluations of classification methods with post-hoc explanations, anomaly detection methods, and EBD methods demonstrate the superior performance of the last and, particularly, of EFIQA-CP.
Read the full abstract ▾
All authors Pengwei Wang, José Morano, Virginia Mares, Hrvoje Bogunović
Computer Vision
In-context Region-based Drag: Drag Any Region to Any Shape
Jiacheng Sui, Tianyu Hao, Bingjie Gao et al. (+2 more)
Diffusion models have shown promise in drag-style editing. Previous works mainly focus on point-based drag, which is inherently ambiguous. This paper focuses on region-based drag and introduces a novel In-Context Region-based Drag (ICRDrag) method. Under the in-context learning framework, ICRDrag consumes a source image, a source region mask, and a target region mask, producing the target dragged image. Built upon the basic in-context learning model, we introduce two novel
attention regularization: 1) image-mask
attention consistency to ensure that a target region attends to similar source regions for image and mask modalities; 2) source-target
attention correspondence to ensure the mutual correspondence between source and target regions. To facilitate region-based drag, we also construct Paired Region
Dataset (PRD), a large-scale
dataset with paired masks and images. Extensive experiments show that ICRDrag significantly outperforms existing methods in both quantitative metrics and user studies, achieving superior editing accuracy and visual fidelity. The
dataset, code, and model are available at https://github.com/bcmi/ICRDrag-Region-Drag-Editing.
Read the full abstract ▾
All authors Jiacheng Sui, Tianyu Hao, Bingjie Gao, Li Niu, Guangtao Zhai
Computer Vision
OracleAnalyser: Analysing Implicit Semantics of Oracle Bone Scripts through MLLMs with Post-training
Zijia Song, Yelin Wang, Zhengyi Ma et al. (+5 more)
With the advancement of artificial intelligence, research on oracle bone scripts has entered a new era. However, existing methods and
benchmarks remain largely confined to recognition tasks, overlooking the equally crucial aspect of oracle bone analysis. To address this gap, we propose OracleAnalyser, a
reasoning framework for oracle bone analysis based on post-training techniques. Specifically, we fine-tune Qwen2.5-VL-3B-Instruct through multiple post-training stages and introduce a new preference optimization
algorithm, Stable Focal Preference Optimization (SFPO), tailored to the characteristics of oracle bone
datasets. In addition, we release both an oracle bone
reasoning dataset and an oracle bone preference
dataset, and further construct a new
benchmark to evaluate models' analytical capabilities for oracle bone scripts. Extensive experiments validate the superior analytical performance of OracleAnalyser, which achieves remarkable results with only 3B parameters, surpassing models with substantially larger scales.
Read the full abstract ▾
All authors Zijia Song, Yelin Wang, Zhengyi Ma, Zitong Yu, Tianheng Wang, Jiahuan Zhang, Taorui Wang, Kaicheng Yu
Computer Vision
SurgAtlas: A Large-Scale Surgical Video-Language Dataset with 2,391 Hours of Open and Minimally Invasive Surgery
Filippos Bellos, Andre S. Gala-Garza, Miaowei Wang et al. (+8 more)
We introduce SurgAtlas, the largest
surgical video-
language dataset to date, comprising 15,291
videos (2,391 hours) spanning 18
surgical specialties and over 5,000 procedure types, sourced entirely from publicly available YouTube content. SurgAtlas is also the first
surgical video-
language dataset to include open surgery at scale, with 6,182 open procedure
videos alongside over 9,000 minimally invasive recordings, and the first to establish standardized
benchmarks for open-surgery
video understanding. We additionally provide an expert-validated subset with verified visual question-answer pairs across diverse open and minimally invasive procedures, serving as a clinically grounded
benchmark for
surgical reasoning. Compared with existing
surgical video-
language datasets, SurgAtlas provides one of the most diverse annotation schemas, combining segment-level captions, step- and phase-level descriptions,
video-level
surgical descriptions, and
reasoning-oriented question-answer pairs organized within a hierarchical taxonomy. These annotations are constructed through an automated multi-tier pipeline with LLM-based enrichment and a staged VQA generation framework with explicit groundedness verification. The scale and diversity of SurgAtlas enable training
surgical foundation models with broad procedural coverage: we finetune Qwen3-VL-8B through a two-stage captioning-then-instruction pipeline and achieve competitive or state-of-the-art results on multiple established
surgical benchmarks, including phase recognition, triplet detection, and
reasoning question answering. More broadly, SurgAtlas provides a large native public
video corpus that can support future large-scale pretraining of
multimodal surgical AI systems and contribute to the development of next-generation foundation models for surgery.
Dataset publicly available at https://huggingface.co/
datasets/filbel/SurgAtlas
Read the full abstract ▾
All authors Filippos Bellos, Andre S. Gala-Garza, Miaowei Wang, Alyssa M. Hardin, Ahmad M. Hider, Li Yayuan, Jing Bi, Susan Liang, Chenliang Xu, Donald S. Likosky, Jason J. Corso
Computer Vision
Enhancing Brain MRI Anomaly Detection and Reasoning with ROI Rethink and Synthetic Data
Shangkun Li, Jie Xu, Yi Guo et al. (+2 more)
Medical
vision-
language models typically generate diagnoses through single-pass inference without indicating which image regions support their conclusions. This lack of spatial grounding limits clinical utility: outputs cannot be audited, and models may hallucinate findings on normal scans. We present BrReMark (
Brain Rethink via ROI Marking), a framework that introduces explicit region marking into
brain MRI diagnosis. The model first generates hypotheses about potential abnormalities and grounds them through explicit bounding box marking, then verifies conclusions by re-examining the marked evidence. Training combines supervised fine-tuning on structured
reasoning trajectories with reinforcement learning using a composite reward over localization accuracy and diagnostic
reasoning. Furthermore, we integrate a domain randomization-based pathology synthesis augmentation strategy to improve the model's generalizability to out-of-distribution (OOD) data. On internal
benchmark, BrReMark improves mAP50 from 0.74% to 37.54% compared to the base model, while achieving 21.57% Clinical F1 and 45.26% diagnostic accuracy. On NOVA OOD
benchmark, it also achieves competitive overall performance with a 45.7% reduction in false positives compared to the state-of-the-art, indicating reduced
hallucination on rare pathologies. These findings suggest that explicit hypothesis-verification grounding is a practical path toward trustworthy open-ended
brain MRI diagnosis across both in-distribution and OOD settings.
Read the full abstract ▾
All authors Shangkun Li, Jie Xu, Yi Guo, Zeju Li, Yuanyuan Wang
Computer Vision
USS: Unifying Spatial-Semantic Prompting for End to End Embodied Visual Tracking
Yuchen Xie, Xinyu Zhou, Kuangji Zuo et al. (+4 more)
Embodied Visual Tracking (EVT) requires an
agent to continuously follow a designated target while moving through dynamic environments. Existing
embodied tracking methods generally rely on either an implicit target-selection convention or a
language description, leaving the target-specification interface largely fixed. However, different tracking scenarios naturally favor different forms of target specification:
language can specify a target outside the
robot's current view, whereas spatial
prompts provide direct instance designation for visible targets and can be useful for selecting among similar looking people, designating hard-to-describe individuals, or specifying a target under time pressure. We therefore introduce unified spatial-semantic prompting for EVT, in which text, a point, a box, and a mask serve as complementary target specifications that can be selected according to different scenarios, and present USS, an end-to-end architecture that maps any of them to
egocentric waypoints. A modality-specific
prompt encoder feeds a common design comprising hybrid-
attention fusion, temporal
memory, cross-view aggregation, latent prediction, and waypoint decoding, with one policy instance trained for each interface under an identical recipe. Across 320 zero-shot real-
robot trials with policies trained only in simulation, spatial
prompts perform comparably to
language in ordinary tracking scenarios while providing clear benefits when visually similar targets require precise instance designation, supporting our premise that different scenarios favor different target specifications. On simulation EVT-Bench under the standard
language-
prompt protocol, USS obtains the highest success rate among non-MLLM methods at 57 FPS, against the 4.8-10 FPS reported by MLLM trackers that are stronger on several metrics. Project site: https://arescheah.github.io/uss-project-page/.
Read the full abstract ▾
All authors Yuchen Xie, Xinyu Zhou, Kuangji Zuo, Yanshuo Lu, Fengrui Huang, Boyu Ma, Jianfei Yang
Computer Vision · Cross-listed from cs.CR
Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks
Kavindu Herath, Joshua C. Zhao and Saurabh Bagchi
Federated learning is vulnerable to
backdoor attacks in which malicious clients inject poisoned updates while preserving benign-task performance. In this paper, we study a semantics-driven
backdoor mechanism in which attackers use natural visual accessories as triggers and manipulate only the trigger color while keeping the attack pipeline fixed. Our framework considers semantic trigger objects such as masks and sunglasses, instantiated in black and white variants, and evaluates their effect in a controlled
federated learning setting. Malicious clients construct poisoned samples by applying a trigger to source-class images and relabeling them to an attacker-chosen target class, while benign clients train only on clean data. We analyze this mechanism under both a standard poisoning objective and a stronger SABLE-based objective that combines clean classification loss, triggered target loss, feature-separation loss in the penultimate representation space, and regularization to keep malicious updates close to the global model. This design enables the attack to remain effective while reducing excessive update drift. Experiments on a four-class CelebA hair-color task show that trigger color significantly changes attack success rate even when trigger semantics, placement, and poisoning budget are unchanged. White triggers are more effective for attacks targeting the blond class, whereas black triggers perform better for attacks targeting the black class. The same trend persists under robust aggregation, showing that trigger color is a meaningful factor in the operation, persistence, and evaluation of semantic
backdoor mechanisms in
federated learning.
Read the full abstract ▾