Machine Learning · Cross-listed from cs.AI
LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
Jon M Laurent, Albert Bou, Michael Pieler et al. (+9 more)
Optimism for accelerating scientific discovery with AI continues to grow. Current applications of AI in scientific research range from training dedicated foundation models on scientific data to agentic autonomous hypothesis generation systems to AI-driven autonomous labs. The need to measure progress of AI systems in scientific domains correspondingly must not only accelerate, but increasingly shift focus to more real-world capabilities. Beyond rote knowledge and even just
reasoning to actually measuring the ability to perform meaningful work. Prior work introduced the
Language Agent Biology
Benchmark LAB-Bench as an initial attempt at measuring these abilities. Here we introduce an
evolution of that
benchmark, LABBench2, for measuring real-world capabilities of AI systems performing useful scientific tasks. LABBench2 comprises nearly 1,900 tasks and is, for the most part, a continuation of LAB-Bench, measuring similar capabilities but in more realistic contexts. We evaluate performance of current frontier models, and show that while abilities measured by LAB-Bench and LABBench2 have improved substantially, LABBench2 provides a meaningful jump in difficulty (model-specific accuracy differences range from -26% to -46% across subtasks) and underscores continued room for performance improvement. LABBench2 continues the legacy of LAB-Bench as a de facto
benchmark for AI scientific research capabilities and we hope that it continues to help advance development of AI tools for these core research functions. To facilitate community use and development, we provide the task
dataset at https://huggingface.co/
datasets/futurehouse/labbench2 and a public eval harness at https://github.com/EdisonScientific/labbench2.
Read the full abstract ▾
All authors Jon M Laurent, Albert Bou, Michael Pieler, Conor Igoe, Alex Andonian, Siddharth Narayanan, James Braza, Alexandros Sanchez Vassopoulos, Jacob L Steenwyk, Blake Lash, Andrew D White, Samuel G Rodriques
Machine Learning
Scaling DPPs for RAG: Density Meets Diversity
Xun Sun, Baiheng Xie, Li Huang et al. (+1 more)
Retrieval-Augmented Generation (RAG) enhances Large
Language Models (LLMs) by grounding generation in external knowledge, yielding relevance responses that are aligned with factual evidence and evolving corpora. Standard RAG pipelines construct context through relevance ranking, performing point-wise scoring between the user query and each corpora chunk. This formulation, however, ignores interactions among retrieved candidates, leading to redundant contexts that dilute density and fail to surface complementary evidence. We argue that effective
retrieval should optimize jointly for both density and diversity, ensuring the grounding evidence that is dense in information yet diverse in coverage. In this study, we propose ScalDPP, a diversity-aware
retrieval mechanism for RAG that incorporates Determinantal Point Processes (DPPs) through a lightweight P-Adapter, enabling scalable modeling of inter-chunk dependencies and complementary context selection. In addition, we develop a novel set-level objective, Diverse Margin Loss (DML), that enforces ground-truth complementary evidence chains to dominate any equally sized redundant alternatives under DPP geometry. Experimental results demonstrate the superiority of ScalDPP, substantiating our core statement in practice.
Read the full abstract ▾
All authors Xun Sun, Baiheng Xie, Li Huang, Qiang Gao
Machine Learning
Not All Denoising Steps Are Equal: Model Scheduling for Faster Masked Diffusion Language Models
Ivan Sedykh, Nikita Sorokin and Valentin Malykh
Recent advances in masked
diffusion language models (MDLMs) narrow the quality gap to autoregressive LMs, but their sampling remains expensive because generation requires many full-sequence denoising passes with a large
Transformer and, unlike autoregressive decoding, cannot benefit from KV caching. In this work, we exploit the flexibility of the
diffusion framework and study model scheduling, where a smaller MDLM replaces the full model at a subset of denoising steps. Across models trained on OpenWebText and LM1B, we show that early and late denoising steps are substantially more robust to such replacement than middle steps, enabling up to a 17% reduction in FLOPs with only modest degradation in generative perplexity under both unconditional and prefix-conditional generation, while preserving sample diversity. We support these findings with a step-importance analysis based on loss and KL divergence between small and large models across timesteps, as well as an exhaustive search over coarse step segments, both of which identify the middle of the
diffusion trajectory as most sensitive consistently across
datasets. Our results suggest that simple, architecture-agnostic scheduling rules can significantly accelerate MDLM sampling while largely preserving generation quality.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
DF-ACBlurGAN: Structure-Aware Conditional Generation of Internally Repeated Patterns for Biomaterial Microtopography Design
Rongjun Dong, Xin Chen, Morgan R Alexander et al. (+4 more)
Learning to generate images with internally repeated and periodic structures poses a fundamental challenge for machine learning and computer
vision models, which are typically optimised for local texture statistics and semantic realism rather than global structural consistency. This limitation is particularly pronounced in applications requiring strict control over repetition scale, spacing, and boundary coherence, such as microtopographical biomaterial surfaces. In this work, biomaterial design serves as a use case to study conditional generation of repeated patterns under weak supervision and class imbalance. We propose DF-ACBlurGAN, a structure-aware conditional generative
adversarial network that explicitly reasons about long-range repetition during training. The approach integrates frequency-domain repetition scale estimation, scale-adaptive Gaussian blurring, and unit-
cell reconstruction to balance sharp local features with stable global periodicity. Conditioning on experimentally derived biological response labels, the model synthesises designs aligned with target functional outcomes. Evaluation across multiple biomaterial
datasets demonstrates improved repetition consistency and controllable structural variation compared to conventional generative approaches.
Read the full abstract ▾
All authors Rongjun Dong, Xin Chen, Morgan R Alexander, Karthikeyan Sivakumar, Reza Omdivar, David A Winkler, Grazziela Figueredo
Machine Learning · Cross-listed from cs.CL
Hierarchical Embedding Fusion for Retrieval-Augmented Code Generation
Nikita Sorokin, Ivan Sedykh and Valentin Malykh
Retrieval-augmented code generation often conditions the decoder on large retrieved code snippets. This ties online inference cost to repository size and introduces noise from long contexts. We present Hierarchical
Embedding Fusion (HEF), a two-stage approach to repository representation for code completion. First, an offline cache compresses repository chunks into a reusable hierarchy of dense vectors using a small fuser model. Second, an online interface maps a small number of retrieved vectors into learned pseudo-
tokens that are consumed by the code generator. This replaces thousands of retrieved
tokens with a fixed pseudo-
token budget while preserving access to repository-level information. On RepoBench and RepoEval, HEF with a 1.8B-parameter pipeline achieves exact-match accuracy comparable to snippet-based
retrieval baselines, while operating at sub-second median latency on a single A100 GPU. Compared to graph-based and iterative
retrieval systems in our experimental setup, HEF reduces median end-to-end latency by 13 to 26 times. We also introduce a utility-weighted likelihood signal for filtering training contexts and report ablation studies on pseudo-
token budget,
embedding models, and robustness to harmful
retrieval. Overall, these results indicate that hierarchical dense caching is an effective mechanism for low-latency, repository-aware code completion.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Hierarchical Latent Structures in Data Generation Process Unify Mechanistic Phenomena across Scale
Jonas Rohweder, Subhabrata Dutta and Iryna Gurevych
Contemporary studies in mechanistic interpretability have uncovered many puzzling phenomena in the neural information processing of
Transformer-based
language models, such as induction heads, function vectors, and the Hydra effect. Some of these individual phenomena have been independently tied to different data distributional properties, while some have been loosely associated with model architecture and how
Transformers process information. However, a unified understanding of the relationship between data, model architecture, and optimization remains lacking, failing to answer the fundamental question: why do these three phenomena appear universally across different model families and scales, despite their seeming disconnect? In this work, we answer this question by unifying these three phenomena as consequences of hierarchical latent structures in the data generation process, coupled with decorrelated gradients across additive model components and directional concavity in the representation geometry. We validate our theoretical results in a toy model regime and in a large-scale synthetic data regime, comparing them with
language models trained on natural
language data.
Read the full abstract ▾
Machine Learning
What Makes Position Zero Special? A Mechanistic Study of Position Zero Attention Sinks in LLMs
Runyu Peng, Ruixiao Li, Mingshu Chen et al. (+5 more)
Transformers frequently allocate disproportionate
attention to specific
tokens, a phenomenon known as
attention sinks. Causal large
language models reliably form one at position zero, though its role remains debated. We approach this question from a mechanistic perspective, tracing how the position-zero sink arises from the model's internal computation. We identify a two-block subnetwork responsible for this behavior, which we term the P0-Sink
Circuit, and show it arises purely from the structural properties of causal
attention, requiring no semantic content. We further validate through from-scratch pre-training experiments that two proposed parameter-free methods effectively accelerate P0 sink formation, and find that earlier sink formation benefits pre-training and improves downstream performance. Both methods outperform the
Transformer baseline and achieve performance comparable to Gated
Attention across comprehensive settings. Code is available now at https://github.com/Pryest/flash-linear-
attention.
Read the full abstract ▾
All authors Runyu Peng, Ruixiao Li, Mingshu Chen, Yunhua Zhou, Qipeng Guo, Xipeng Qiu, Yucheng Lu, Chen Zhao
Machine Learning · Cross-listed from cs.CL
GRIP: Geometric Refinement and Adaptive Information Potential for Data Efficiency
Changhao Wang, Jiaolong Yang, Xinhao Yao et al. (+7 more)
The performance of Large
Language Models (LLMs) is increasingly governed by data efficiency rather than raw scaling volume. However, existing selection methods often decouple global distribution balancing from local instance selection, compromising the hierarchical integrity of the training set. We introduce GRIP (
Geometric Refinement and Adaptive Information Potential), a framework that unifies these dimensions by modeling the corpus as an information-dense
geometric space. GRIP employs a Rapid Adaptation Probe (RAP) to quantify the information potential of semantic clusters, dynamically re-allocating the sampling budget to regions with the highest representation deficits. Subsequently, we perform Intra-Cluster Selection using a length-rectified
geometric prior to counteract
embedding density artifacts and preserve long-tail logical sequences. Extensive evaluations on Mixture-of-Experts (MoE) models up to 300B
tokens demonstrate that GRIP consistently outperforms state-of-the-art baselines, \textbf{surpassing the performance of models trained on $3\times$ larger uncurated
datasets}. Our work establishes a robust
geometric foundation for adaptive data curation in large-scale pre-training.
Read the full abstract ▾
All authors Changhao Wang, Jiaolong Yang, Xinhao Yao, Yunfei Yu, Peng Jiao, Lu Yu, Junpeng Fang, Riccardo Cantoro, Qing Cui, Jun Zhou
Machine Learning · Cross-listed from math.OC
Bilevel Optimization with Lower-Level Uniform Convexity: Theory and Algorithm
Yuman Wu, Xiaochuan Gong, Jie Hao et al. (+1 more)
Bilevel optimization is a hierarchical framework where an upper-level optimization problem is constrained by a lower-level problem, commonly used in machine learning applications such as hyperparameter optimization. Existing bilevel optimization methods typically assume strong convexity or Polyak-Łojasiewicz (PL) conditions for the lower-level function to establish non-asymptotic convergence to a solution with small hypergradient. However, these assumptions may not hold in practice, and recent work~\citep{chen2024finding} has shown that bilevel optimization is inherently intractable for general convex lower-level functions with the goal of finding small hypergradients. In this paper, we identify a tractable class of bilevel optimization problems that interpolates between lower-level strong convexity and general convexity via lower-level uniform convexity. For uniformly convex lower-level functions with exponent $p\geq 2$, we establish a novel implicit differentiation
theorem characterizing the hyperobjective's smoothness property. Building on this, we design a new
stochastic algorithm, termed UniBiO, with provable convergence guarantees, based on an oracle that provides
stochastic gradient and Hessian-vector product information for the bilevel problems. Our
algorithm achieves $\widetilde{O}(\epsilon^{-5p+6})$ oracle complexity bound for finding $\epsilon$-stationary points. Notably, our complexity bounds match the optimal rates in terms of the $\epsilon$ dependency for strongly convex lower-level functions ($p=2$), up to logarithmic factors. Our theoretical findings are validated through experiments on synthetic tasks and data hyper-cleaning, demonstrating the effectiveness of our proposed
algorithm.
Read the full abstract ▾
All authors Yuman Wu, Xiaochuan Gong, Jie Hao, Mingrui Liu
Machine Learning · Cross-listed from q-fin.GN
Sumin Kim, Jihoon Kwon, Yoon Kim et al. (+9 more)
Mention
markets, a type of prediction
market in which contracts resolve based on whether a specified keyword is mentioned during a future public event, require accurate probabilistic forecasts of keyword-mention outcomes. While recent work shows that large
language models (LLMs) can generate forecasts competitive with human forecasters, it remains unclear how input context should be designed to support accurate prediction. In this paper, we study this question through experiments on earnings-call mention
markets, which require
forecasting whether a company will mention a specified keyword during its upcoming call. We run controlled comparisons varying (i) which contextual information is provided (news and/or prior earnings-call transcripts) and (ii) how
market probability, (i.e., prediction
market contract price) is used. We introduce
Market-Conditioned Prompting (MCP), which explicitly treats the
market-implied probability as a prior and instructs the LLM to update this prior using textual evidence, rather than re-predicting the base rate from scratch. In our experiments, we find three insights: (1) richer context consistently improves
forecasting performance; (2)
market-conditioned prompting (MCP), which treats the
market probability as a prior and updates it using textual evidence, yields better-calibrated forecasts; and (3) a mixture of the
market probability and MCP (MixMCP) outperforms the
market baseline. By dampening the LLM's posterior update with the
market prior, MixMCP yields more robust predictions than either the
market or the LLM alone.
Read the full abstract ▾
All authors Sumin Kim, Jihoon Kwon, Yoon Kim, Nicole Kagan, Raffi Khatchadourian, Wonbin Ahn, Alejandro Lopez-Lira, Jaewon Lee, Yoontae Hwang, Oscar Levy, Yongjae Lee, Chanyeol Choi
Machine Learning · Cross-listed from cs.CV
VISION-ICE: Video-based Interpretation and Spatial Identification of Arrhythmia Origins via Neural Networks in Intracardiac Echocardiography
Dorsa EPMoghaddam, Feng Gao, Drew Bernard et al. (+3 more)
Contemporary high-density mapping techniques and preoperative CT/MRI remain time and resource intensive in localizing arrhythmias. AI has been validated as a clinical decision aid in providing accurate, rapid real-time analysis of echocardiographic images. Building on this, we propose an AI-enabled framework that leverages intracardiac echocardiography (ICE), a routine part of electrophysiology procedures, to guide clinicians toward areas of arrhythmogenesis and potentially reduce procedural time. Arrhythmia source localization is formulated as a three-class classification task, distinguishing normal sinus rhythm, left-sided, and right-sided arrhythmias, based on ICE
video data. We developed a 3D Convolutional Neural Network trained to discriminate among the three aforementioned classes. In ten-fold cross-validation, the model achieved a mean accuracy of 66.2% when evaluated on four previously unseen patients (substantially outperforming the 33.3% random baseline). These results demonstrate the feasibility and clinical promise of using ICE
videos combined with deep learning for automated arrhythmia localization. Leveraging ICE imaging could enable faster, more targeted electrophysiological interventions and reduce the procedural burden of cardiac ablation. Future work will focus on expanding the
dataset to improve model robustness and generalizability across diverse patient populations.
Read the full abstract ▾
All authors Dorsa EPMoghaddam, Feng Gao, Drew Bernard, Kavya Sinha, Mehdi Razavi, Behnaam Aazhang
Machine Learning · Cross-listed from cs.SD
RA-QA: A Benchmarking System for Respiratory Audio Question Answering Under Real-World Heterogeneity
Gaia A. Bertolino, Yuwei Zhang, Tong Xia et al. (+2 more)
As conversational
multimodal AI tools are increasingly adopted to process patient data for health assessment, robust
benchmarks are needed to measure progress and expose failure modes under realistic conditions. Despite the importance of respiratory audio for mobile health screening, respiratory audio question answering remains underexplored, with existing studies evaluated narrowly and lacking real-world heterogeneity across modalities, devices, and question types. We hence introduce the Respiratory-Audio Question-Answering (RA-QA)
benchmark, including a standardized data generation pipeline, a comprehensive
multimodal QA collection, and a unified evaluation protocol. RA-QA harmonizes public RA
datasets into a collection of 9 million format-diverse QA pairs covering diagnostic and contextual attributes. We
benchmark general audio-
language models as well as domain-specific architectures, establishing reproducible reference points and showing how current approaches fail under heterogeneity.
Read the full abstract ▾
All authors Gaia A. Bertolino, Yuwei Zhang, Tong Xia, Domenico Talia, Cecilia Mascolo
Machine Learning
Optimal Multi-Debris Mission Planning in LEO: A Deep Reinforcement Learning Approach with Co-Elliptic Transfers and Refueling
Agni Bandyopadhyay and Gunther Waxenegger-Wilfing
This paper addresses the challenge of multi target active debris removal (ADR) in Low Earth Orbit (LEO) by introducing a unified coelliptic maneuver framework that combines Hohmann transfers, safety ellipse proximity operations, and explicit refueling logic. We
benchmark three distinct planning
algorithms Greedy heuristic, Monte Carlo Tree Search (MCTS), and deep reinforcement learning (RL) using Masked Proximal Policy Optimization (PPO) within a realistic orbital simulation environment featuring randomized debris fields, keep out zones, and delta V constraints. Experimental results over 100 test scenarios demonstrate that Masked PPO achieves superior mission efficiency and computational performance, visiting up to twice as many debris as Greedy and significantly outperforming MCTS in runtime. These findings underscore the promise of modern RL methods for scalable, safe, and resource efficient space mission planning, paving the way for future advancements in ADR autonomy.
Read the full abstract ▾
Machine Learning
CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models
Xiao Zhu, Xinyu Zhou, Boyu Zhu et al. (+5 more)
Reinforcement Learning from Verifiable Rewards (RLVR) has driven recent progress in code large
language models by leveraging execution-based feedback from unit tests, but its scalability is fundamentally constrained by the availability and reliability of high-quality test cases. We propose CodeScaler, a reward model designed to scale both reinforcement learning training and test-time inference for code generation. CodeScaler is trained on carefully curated preference data derived from verified code problems and incorporates syntax-aware code extraction and validity-preserving reward shaping to ensure stable and robust optimization. Across four coding
benchmarks, CodeScaler consistently outperforms execution-based RL by +1.55 points on Qwen3-8B-Base and +4.23 points on Qwen3-14B-Base. By further scaling to 44K problems with additional synthetic data, CodeScaler yields +14.64 points improvement over the base model without requiring any test cases. At inference time, CodeScaler serves as an effective test-time scaling method, achieving performance comparable to unit test approaches while providing a 10-fold reduction in latency. Moreover, CodeScaler surpasses existing reward models on RM-Bench not only in the code domain (+3.3 points), but also in general and
reasoning domains (+2.7 points on average).
Read the full abstract ▾
All authors Xiao Zhu, Xinyu Zhou, Boyu Zhu, Hanxu Hu, Mingzhe Du, Haotian Zhang, Huiming Wang, Zhijiang Guo
Machine Learning
Probabilistic NDVI Forecasting from Sparse Satellite Time Series and Weather Covariates
Irene Iele, Giulia Romoli, Daniele Molino et al. (+4 more)
Short-term
forecasting of vegetation dynamics is a key enabler for data-driven decision support in precision agriculture. Normalized Difference Vegetation Index (NDVI)
forecasting from satellite observations, however, remains challenging due to sparse and irregular sampling caused by cloud masking, as well as the heterogeneous climatic conditions under which crops evolve. In this work, we propose a probabilistic
forecasting framework for field-level NDVI prediction under sparse, irregular clear-sky acquisitions. The architecture separates the encoding of historical NDVI and meteorological observations from future exogenous
covariates, fusing both representations for multi-step
quantile prediction. To address irregular revisit patterns and horizon-dependent uncertainty, we introduce a temporal-distance weighted
quantile loss that aligns the training objective with the effective
forecasting horizon. In addition, we incorporate cumulative and extreme-weather feature engineering to capture delayed meteorological effects relevant to vegetation response. Experiments on European satellite data show that the proposed approach outperforms statistical, deep learning, and time-series baselines on both pointwise and probabilistic evaluation metrics. Ablation studies confirm that target history is the primary driver of performance, with meteorological
covariates providing additional gains in the full
multimodal setting. The code is available at https://github.com/arco-group/ndvi-
forecasting.
Read the full abstract ▾
All authors Irene Iele, Giulia Romoli, Daniele Molino, Elena Mulero Ayllón, Filippo Ruffini, Paolo Soda, Matteo Tortora
Machine Learning
Duality Models: An Embarrassingly Simple One-step Generation Paradigm
Peng Sun, Xinyi Shang, Tao Lin et al. (+1 more)
Consistency-based generative models like Shortcut and MeanFlow achieve impressive results via a target-aware design for solving the Probability Flow ODE (PF-ODE). Typically, such methods introduce a target time $r$ alongside the current time $t$ to modulate outputs between a local multi-step derivative ($r = t$) and a global few-step integral ($r = 0$). However, the conventional "one input, one output" paradigm enforces a partition of the training budget, often allocating a significant portion (e.g., 75% in MeanFlow) solely to the multi-step objective for stability. This separation forces a trade-off: allocating sufficient samples to the multi-step objective leaves the few-step generation undertrained, which harms convergence and limits scalability. To this end, we propose Duality Models (DuMo) via a "one input, dual output" paradigm. Using a shared backbone with dual heads, DuMo simultaneously predicts velocity $v_t$ and flow-map $u_t$ from a single input $x_t$. This applies
geometric constraints from the multi-step objective to every sample, bounding the few-step estimation without separating training objectives, thereby significantly improving stability and efficiency. On ImageNet 256 $\times$ 256, a 679M
Diffusion Transformer with SD-VAE achieves a state-of-the-art (SOTA) FID of 1.79 in just 2 steps. Code is available at: https://github.com/LINs-lab/DuMo
Read the full abstract ▾
All authors Peng Sun, Xinyi Shang, Tao Lin, Zhiqiang Shen
Machine Learning
LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMs
Ofir Gordon, Lior Dikstein, Arnon Netzer et al. (+2 more)
Post-training quantization (PTQ) is a widely used approach for reducing the
memory and compute costs of large
language models (LLMs). Recent studies have shown that applying invertible transformations to activations can significantly improve quantization robustness by reducing activation outliers; however, existing approaches are largely restricted to rotation or Hadamard-based transformations. Moreover, most studies focused primarily on traditional quantization schemes, whereas modern hardware increasingly supports the microscaling (MX) data format. Attempts to combine both showed severe performance degradation, leading prior work to introduce assumptions on the transformations. In this work, we take a complementary perspective. First, we provide a theoretical analysis of transformations under MX quantization by deriving a bound on the quantization error. Our analysis emphasizes the importance of accounting for both the activation distribution and the underlying quantization structure. Building on this analysis, we propose LATMiX, a method that generalizes outlier reduction to learnable invertible affine transformations optimized using standard deep learning tools. Experiments show consistent improvements in average accuracy for MX low-bit quantization over strong baselines on a wide range of zero-shot
benchmarks, across multiple model sizes.
Read the full abstract ▾
All authors Ofir Gordon, Lior Dikstein, Arnon Netzer, Idan Achituve, Hai Victor Habi
Machine Learning
BioBridge: Bridging Proteins and Language for Enhanced Biological Reasoning with LLMs
Yujia Wang, Jihong Guan, Wengen Li et al. (+2 more)
Existing
Protein Language Models (PLMs) often suffer from limited adaptability to multiple tasks and exhibit poor generalization across diverse biological contexts. In contrast, general-purpose Large
Language Models (LLMs) lack the capability to interpret
protein sequences and fall short in domain-specific knowledge, limiting their capacity for effective biosemantic
reasoning. To combine the advantages of both, we propose BioBridge, a domain-adaptive continual pretraining framework for
protein understanding. This framework employs Domain-Incremental Continual Pre-training (DICP) to infuse
protein domain knowledge and general
reasoning corpus into a LLM simultaneously, effectively mitigating catastrophic forgetting. Cross-modal
alignment is achieved via a PLM-Projector-LLM pipeline, which maps
protein sequence
embeddings into the semantic space of the
language model. Ultimately, an end-to-end optimization is adopted to uniformly support various tasks, including
protein property prediction and knowledge question-answering. Our proposed BioBridge demonstrates performance comparable to that of mainstream PLMs on multiple
protein benchmarks, such as EC and BindingDB. It also achieves results on par with LLMs on general understanding tasks like MMLU and RACE. This showcases its innovative advantage of combining domain-specific adaptability with general-purpose
language competency.
Read the full abstract ▾
All authors Yujia Wang, Jihong Guan, Wengen Li, Shuigeng Zhou, Xuhong Wang
Machine Learning
Joint Parameter and State-Space Bayesian Optimization: Using Process Expertise to Accelerate Manufacturing Optimization
Saksham Kiroriwal, Julius Pfrommer and Jürgen Beyerer
Bayesian optimization (BO) is a powerful method for optimizing black-box manufacturing processes, but its performance is often limited when dealing with high-dimensional multi-stage systems, where we can observe intermediate outputs. Standard BO models the process as a black box and ignores the intermediate observations and the underlying process structure. Partially Observable Gaussian Process Networks (POGPN) model the process as a Directed Acyclic Graph (DAG). However, using intermediate observations is challenging when the observations are high-dimensional state-space time series. Process-expert knowledge can be used to extract low-dimensional latent features from the high-dimensional state-space data. We propose POGPN-JPSS, a framework that combines POGPN with Joint Parameter and State-Space (JPSS) modeling to use intermediate extracted information. We demonstrate the effectiveness of POGPN-JPSS on a challenging, high-dimensional simulation of a multi-stage bioethanol production process. Our results show that POGPN-JPSS significantly outperforms state-of-the-art methods by achieving the desired performance threshold twice as fast and with greater reliability. The fast optimization directly translates to substantial savings in time and resources. This highlights the importance of combining expert knowledge with structured probabilistic models for rapid process maturation.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.ao-ph
Symmetric Composition of Anisotropic Operators for Global Subseasonal-to-Seasonal Climate Forecasting
Ziyu Zhou, Yuchen Fang, Weilin Ruan et al. (+4 more)
Accurate global Subseasonal-to-Seasonal (S2S)
climate forecasting is critical for disaster preparedness and resource management, yet it remains challenging due to chaotic atmospheric dynamics. Despite advances in geometry-aware representations, existing methods do not specify how the zonal and meridional interactions that govern real atmospheric dynamics should be explicitly coordinated and composed, leaving an important architectural gap for S2S
forecasting. In this paper, we propose AnisoCast, which explicitly models anisotropic global atmospheric dynamics for accurate S2S
climate forecasting. It couples: (1) an Anisotropic
Embedding strategy that tokenizes the global grid into latitudinal rings, preserving the integrity of zonal periodic structures; and (2) stacked Aniso Blocks that arrange latent Zonal and Meridional Operators in a weight-shared palindromic composition inspired by symmetric operator splitting. Extensive experiments on the ERA5 reanalysis
dataset demonstrate that AnisoCast establishes a new state-of-the-art, significantly outperforming existing methods in both
forecasting accuracy and computational efficiency.
Read the full abstract ▾
All authors Ziyu Zhou, Yuchen Fang, Weilin Ruan, Tian Zhou, Shiyu Wang, James Kwok, Yuxuan Liang
Machine Learning · Cross-listed from cs.AI
Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework
Md Muntasir Jahid Ayan, Md. Shahriar Rashid, Tazzina Afroze Hassan et al. (+6 more)
The increasing complexity and frequency of cyber-threats demand intrusion detection systems (IDS) that are not only accurate but also interpretable. This paper presented a novel IDS framework that integrated Explainable Artificial Intelligence (XAI) to enhance transparency in deep learning models. The framework was evaluated experimentally using the
benchmark dataset NSL-KDD, demonstrating superior performance compared to traditional IDS and black-box deep learning models. The proposed approach combined Convolutional Neural Network (CNN) and Long Short-Term
Memory (LSTM) networks for capturing temporal dependencies in traffic sequences. Our deep learning results showed that both CNN and LSTM reached 0.99 for accuracy, whereas LSTM outperformed CNN at macro average precision, recall, and F-1 score. For weighted average precision, recall, and F-1 score, both models scored almost similarly. To ensure interpretability, the XAI model SHapley Additive exPlanations (SHAP) was incorporated, enabling security analysts to understand and validate model decisions. Some notable influential features were srv_serror_rate, dst_host_srv_serror_rate, and serror_rate for both models, as pointed out by SHAP. We also conducted a trust-focused expert survey based on IPIP6 and Big Five personality traits via an interactive UI to evaluate the system's reliability and usability. This work highlighted the potential of combining performance and transparency in cybersecurity solutions and recommends future enhancements through adaptive learning for real-time threat detection.
Read the full abstract ▾
All authors Md Muntasir Jahid Ayan, Md. Shahriar Rashid, Tazzina Afroze Hassan, Hossain Md. Mubashshir Jamil, Mahbubul Islam, Lisan Al Amin, Rupak Kumar Das, Farzana Akter, Faisal Quader
Machine Learning · Cross-listed from cs.CY
Expected Moral Shortfall for Ethical Competence in Decision-making Models
Aisha Aijaz, Raghava Mutharaju and Manohar Kumar
Moral cognition is a crucial yet underexplored aspect of decision-making in AI models. Regardless of the application domain, it should be a consideration that allows for ethically aligned decision-making. This paper presents a multifaceted contribution to this research space. Firstly, a comparative analysis of techniques to instill ethical competence into AI models has been presented to gauge them on multiple performance metrics. Second, a novel mathematical discretization of morality and a demonstration of its real-life application have been conveyed and tested against other techniques on two
datasets. This value is modeled as the risk of loss incurred by the least moral cases, or an Expected Moral Shortfall (EMS), which we direct the AI model to minimize in order to maximize its performance while retaining ethical competence. Lastly, the paper discusses the tradeoff between preliminary AI decision-making metrics such as model performance, complexity, and scale of ethical competence to recognize the true extent of practical social impact.
Read the full abstract ▾