Machine Learning
From Numbers to Prompts: A Cognitive Symbolic Transition Mechanism for Lightweight Time-Series Forecasting
Namkyung Yoon and Hwangnam Kim
Large
language models have achieved remarkable success in time series prediction tasks, but their substantial computational and
memory requirements limit deployment on lightweight platforms. In this paper, we propose the Symbolic Transition Mechanism (STM) a novel framework that bridges numeric time series data and
language models through symbolic abstraction and
prompt engineering. STM transforms continuous time series values into symbol
tokens with quantization techniques based on human cognitive structures, and captures temporal dynamics through structured transformations of symbols, enabling fast engineering based predictions in which
language models focus on critical parts of time series data. STM is a general purpose mechanisms that ensure the integrity of backbone
language models, but they significantly improve their efficiency by inferring the dynamic and structured patterns inherent in time series data. We evaluated STM on various time series
datasets, paired with four small
language models (SLM) with limited computational environments. For all models, STM achieves error reductions of up to 69% in MAE and 90% in MSE compared to the default backbone SLM without STM. These results demonstrate the potential of STM as an efficient, adaptable layer for symbol-driven time series prediction using foundation models. The accuracy improvements were made at negligible resource costs, with maximum GPU
memory of the base model increasing by approximately 0.06% and latency overhead increasing by only 0.64%.
Read the full abstract ▾
Machine Learning
ECCO: Evidence-Driven Causal Reasoning for Compiler Optimization
Haolin Pan, Lianghong Huang, Jinyuan Dong et al. (+2 more)
Compiler auto-tuning faces a dichotomy between traditional black-box search methods, which lack semantic guidance, and recent Large
Language Model (LLM) approaches, which often suffer from superficial pattern matching and causal opacity. In this paper, we introduce ECCO, a framework that bridges interpretable
reasoning with combinatorial search. We first propose a reverse engineering methodology to construct a Chain-of-Thought
dataset, explicitly mapping static code features to verifiable performance evidence. This enables the model to learn the causal logic governing optimization decisions rather than merely imitating sequences. Leveraging this interpretable prior, we design a collaborative inference mechanism where the LLM functions as a strategist, defining optimization intents that dynamically guide the mutation operations of a genetic
algorithm. Experimental results on seven
datasets demonstrate that ECCO significantly outperforms the LLVM opt -O3 baseline, achieving an average 24.44% reduction in cycles.
Read the full abstract ▾
All authors Haolin Pan, Lianghong Huang, Jinyuan Dong, Mingjie Xing, Yanjun Wu
Machine Learning · Cross-listed from cs.CL
CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval
Akshith Reddy Putta, Jacob Devasier and Chengkai Li
Automated Fact-Checking has largely focused on verifying general knowledge against static corpora, overlooking high-stakes domains like law where truth is evolving and technically complex. We introduce CaseFacts, a
benchmark for verifying colloquial legal claims against U.S. Supreme Court precedents. Unlike existing resources that map formal texts to formal texts, CaseFacts challenges systems to bridge the semantic gap between layperson assertions and technical jurisprudence while accounting for temporal validity. The
dataset consists of 6,294 claims categorized as Supported, Refuted, or Overruled. We construct this
benchmark using a multi-stage pipeline that leverages Large
Language Models (LLMs) to synthesize claims from expert case summaries, employing a novel semantic similarity heuristic to efficiently identify and verify complex legal overrulings. Experiments with state-of-the-art LLMs reveal that the task remains challenging; notably, augmenting models with unrestricted web search degrades performance compared to closed-book baselines due to the
retrieval of noisy, non-authoritative precedents. We release CaseFacts to spur research into legal fact verification systems.
Read the full abstract ▾
Machine Learning
Parameter Inference and Uncertainty Quantification with Diffusion Models: Extending CDI to 2D Spatial Conditioning
Dmitrii Torbunov, Yihui Ren, Lijun Wu et al. (+1 more)
Uncertainty quantification is critical in scientific inverse problems to distinguish identifiable parameters from those that remain ambiguous given available measurements. The Conditional
Diffusion Model-based Inverse Problem Solver (CDI) has previously demonstrated effective probabilistic inference for one-dimensional temporal signals, but its applicability to higher-dimensional spatial data remains unexplored. We extend CDI to two-dimensional spatial conditioning, enabling probabilistic parameter inference directly from spatial observations. We validate this extension on convergent beam electron diffraction (CBED) parameter inference - a challenging multi-parameter inverse problem in materials characterization where sample geometry, electronic structure, and thermal properties must be extracted from 2D diffraction patterns. Using simulated CBED data with ground-truth parameters, we demonstrate that CDI produces well-calibrated posterior distributions that accurately reflect measurement constraints: tight distributions for well-determined quantities and appropriately broad distributions for ambiguous parameters. In contrast, standard regression methods - while appearing accurate on aggregate metrics - mask this underlying uncertainty by predicting training set means for poorly constrained parameters. Our results confirm that CDI successfully extends from temporal to spatial domains, providing the genuine uncertainty information required for robust scientific inference.
Read the full abstract ▾
All authors Dmitrii Torbunov, Yihui Ren, Lijun Wu, Yimei Zhu
Machine Learning · Cross-listed from astro-ph.IM
Improving Generalization and Uncertainty Quantification of Photometric Redshift Models
Jonathan Soriano, Tuan Do, Srinath Saikrishnan et al. (+4 more)
Accurate
redshift estimates are a vital component in understanding
galaxy evolution and precision cosmology. In this paper, we explore approaches to increase the applicability of machine learning models for photometric
redshift estimation on a broader range of
galaxy types. Typical models are trained with ground-truth
redshifts from spectroscopy. We test the utility and effectiveness of two approaches for combining spectroscopic
redshifts and
redshifts derived from multiband ($\sim$35 filters) photometry, which sample different types of
galaxies compared to spectroscopic surveys. The two approaches are (1) training on a composite
dataset and (2) transfer learning from one
dataset to another. We compile photometric
redshifts from the COSMOS2020 catalog (TransferZ) to complement an established spectroscopic
redshift dataset (GalaxiesML). We used two architectures, deterministic neural networks (NN) and
Bayesian neural networks (BNN), to examine and evaluate their performance with respect to the Legacy Survey of Space and Time (LSST) photo-$z$ science requirements. We also use split conformal prediction for calibrating uncertainty estimates and producing prediction intervals for the BNN and NN, respectively. We find that a NN trained on a composite
dataset predicts photo-$z$'s that are 4.5 times less biased within the
redshift range $0.3<z<1.5$, 1.1 times less scattered, and has a 1.4 times lower outlier rate than a model trained on only spectroscopic ground truths. We also find that BNNs produce reliable uncertainty estimates, but are sensitive to the different ground truths. This investigation leverages different sources of ground truths to develop models that can accurately predict photo-$z$'s for a broader population of
galaxies crucial for surveys such as Euclid and LSST.
Read the full abstract ▾
All authors Jonathan Soriano, Tuan Do, Srinath Saikrishnan, Vikram Seenivasan, Bernie Boscoe, Jack Singal, Evan Jones
Machine Learning · Cross-listed from cs.IR
Evaluation on Entity Matching in Recommender Systems
Zihan Huang, Rohan Surana, Zhouhang Xie et al. (+3 more)
Entity matching is a crucial component in various recommender systems, including conversational recommender systems (CRS) and knowledge-based recommender systems. However, the lack of rigorous evaluation frameworks for cross-
dataset entity matching impedes progress in areas such as LLM-driven conversational recommendations and knowledge-grounded
dataset construction. In this paper, we introduce Reddit-Amazon-EM, a novel
dataset comprising naturally occurring items from Reddit and the Amazon '23
dataset. Through careful manual annotation, we identify corresponding movies across Reddit-Movies and Amazon'23, two existing recommender system
datasets with inherently overlapping catalogs. Leveraging Reddit-Amazon-EM, we conduct a comprehensive evaluation of state-of-the-art entity matching methods, including rule-based, graph-based, lexical-based,
embedding-based, and LLM-based approaches. For reproducible research, we release our manually annotated entity matching gold set and provide the mapping between the two
datasets using the best-performing method from our experiments. This serves as a valuable resource for advancing future work on entity matching in recommender systems.Data and Code are accessible at: https://github.com/huang-zihan/Reddit-Amazon-Entity-Matching.
Read the full abstract ▾
All authors Zihan Huang, Rohan Surana, Zhouhang Xie, Junda Wu, Yu Xia, Julian McAuley
Machine Learning · Cross-listed from cs.CV
Spatiotemporal Semantic V2X Framework for Cooperative Collision Prediction
Murat Arda Onsu, Poonam Lohan, Burak Kantarci et al. (+3 more)
Intelligent Transportation Systems (ITS) demand real-time collision prediction to ensure road safety and reduce accident severity. Conventional approaches rely on transmitting raw
video or high-dimensional sensory data from roadside units (RSUs) to vehicles, which is impractical under vehicular communication bandwidth and latency constraints. In this work, we propose a semantic V2X framework in which RSU-mounted cameras generate spatiotemporal semantic
embeddings of future frames using the
Video Joint
Embedding Predictive Architecture (V-JEPA). To evaluate the system, we construct a digital twin of an urban traffic environment enabling the generation of d verse traffic scenarios with both safe and collision events. These
embeddings of the future frame, extracted from V-JEPA, capture task-relevant traffic dynamics and are transmitted via V2X links to vehicles, where a lightweight attentive probe and classifier decode them to predict imminent collisions. By transmitting only semantic
embeddings instead of raw frames, the proposed system significantly reduces communication overhead while maintaining predictive accuracy. Experimental results demonstrate that the framework with an appropriate processing method achieves a 10% F1-score improvement for collision prediction while reducing transmission requirements by four orders of magnitude compared to raw
video. This validates the potential of semantic V2X communication to enable cooperative, real-time collision prediction in ITS.
Read the full abstract ▾
All authors Murat Arda Onsu, Poonam Lohan, Burak Kantarci, Aisha Syed, Matthew Andrews, Sean Kennedy
Machine Learning
JetFormer: A Scalable and Efficient Transformer for Jet Tagging from Offline Analysis to FPGA Triggers
Ruoqing Zheng, Chang Sun, Qibin Liu et al. (+7 more)
We present JetFormer, a versatile and scalable encoder-only
Transformer architecture for particle jet tagging at the Large
Hadron Collider (LHC). Unlike prior approaches that are often tailored to specific deployment regimes, JetFormer is designed to operate effectively across the full spectrum of jet tagging scenarios, from high-accuracy offline analysis to ultra-low-latency online triggering. The model processes variable-length sets of particle features without relying on input of explicit pairwise interactions, yet achieves competitive or superior performance compared to state-of-the-art methods. On the large-scale JetClass
dataset, a large-scale JetFormer matches the accuracy of the interaction-rich ParT model (within 0.7%) while using 37.4% fewer FLOPs, demonstrating its computational efficiency and strong generalization. On
benchmark HLS4ML 150P
datasets, JetFormer consistently outperforms existing models such as MLPs, Deep Sets, and Interaction Networks by 3-4% in accuracy. To bridge the gap to hardware deployment, we further introduce a hardware-aware optimization pipeline based on multi-objective hyperparameter search, yielding compact variants like JetFormer-tiny suitable for FPGA-based trigger systems with sub-microsecond latency requirements. Through structured
pruning and quantization, we show that JetFormer can be aggressively compressed with minimal accuracy loss. By unifying high-performance modeling and deployability within a single architectural framework, JetFormer provides a practical pathway for deploying
Transformer-based jet taggers in both offline and online environments at the LHC. Code is available at https://github.com/walkieq/JetFormer.
Read the full abstract ▾
All authors Ruoqing Zheng, Chang Sun, Qibin Liu, Lauri Laatu, Arianna Cox, Benedikt Maier, Alexander Tapper, Jose G. F. Coutinho, Wayne Luk, Zhiqiang Que
Machine Learning
NewPINNs: Physics-Informing Neural Networks Using Conventional Solvers for Partial Differential Equations
Satish Chandran, Maedeh Makki, Maziar Raissi et al. (+2 more)
We introduce NewPINNs, a physics-informing learning framework that couples neural networks with conventional numerical solvers for solving differential equations. Rather than enforcing governing equations and boundary conditions through residual-based loss terms, NewPINNs integrates the solver directly into the training loop and defines learning objectives through solver-consistency. The neural network produces candidate solution states that are advanced by the numerical solver, and training minimizes the discrepancy between the network prediction and the solver-evolved state. This pull-push interaction enables the network to learn physically admissible solutions through repeated exposure to the solver's action, without requiring problem-specific loss engineering or explicit evaluation of differential equation residuals. By delegating the enforcement of physics, boundary conditions, and numerical stability to established numerical solvers, NewPINNs mitigates several well-known failure modes of standard physics-informed neural networks, including optimization pathologies, sensitivity to loss weighting, and poor performance in stiff or nonlinear regimes. We demonstrate the effectiveness of the proposed approach across multiple forward and inverse problems involving finite volume, finite element, and spectral solvers.
Read the full abstract ▾
All authors Satish Chandran, Maedeh Makki, Maziar Raissi, Adrien Grenier, Behzad Mohebbi
Machine Learning
SpecBridge: Bridging Mass Spectrometry and Molecular Representations via Cross-Modal Alignment
Yinkai Wang, Yan Zhou Chen, Xiaohui Chen et al. (+2 more)
Small-molecule identification from tandem mass spectrometry (MS/MS) remains a bottleneck in untargeted settings where spectral libraries are incomplete. While deep learning offers a solution, current approaches typically fall into two extremes: explicit generative models that construct molecular graphs atom-by-atom, or joint contrastive models that learn cross-modal subspaces from scratch. We introduce SpecBridge, a novel implicit
alignment framework that treats structure identification as a
geometric alignment problem. SpecBridge fine-tunes a self-supervised spectral encoder (DreaMS) to project directly into the latent space of a frozen molecular foundation model (ChemBERTa), and then performs
retrieval by cosine similarity to a fixed bank of precomputed molecular
embeddings. Across MassSpecGym, Spectraverse, and MSnLib
benchmarks, SpecBridge improves top-1
retrieval accuracy by roughly 20-25% relative to strong neural baselines, while keeping the number of trainable parameters small. These results suggest that aligning to frozen foundation models is a practical, stable alternative to designing new architectures from scratch. The code for SpecBridge is released at https://github.com/HassounLab/SpecBridge.
Read the full abstract ▾
All authors Yinkai Wang, Yan Zhou Chen, Xiaohui Chen, Li-Ping Liu, Soha Hassoun
Machine Learning · Cross-listed from cs.CL
Seyyed Saeid Cheshmi, Hahnemann Ortiz, James Mooney et al. (+1 more)
Vision-
language models (VLMs) have demonstrated strong
reasoning abilities in literal
multimodal tasks such as visual mathematics and science question answering. However, figurative
language, such as sarcasm, humor, and metaphor, remains a significant challenge, as it conveys intent and
emotion through subtle incongruities between expressed and intended meanings. In
multimodal settings, accompanying images can amplify or invert textual meaning, demanding models that reason across modalities and account for subjectivity. We propose a three-step framework for developing efficient
multimodal reasoning models that can (i) interpret
multimodal figurative
language, (ii) provide transparent
reasoning traces, and (iii) generalize across multiple figurative styles. Experiments across four styles show that (1) incorporating
reasoning traces substantially improves
multimodal figurative understanding, (2)
reasoning learned in one style can transfer to others, especially between related styles like sarcasm and humor, and (3) training jointly across styles yields a generalized
reasoning VLM that outperforms much larger open- and closed-source models. Our findings show that lightweight VLMs with verifiable
reasoning achieve robust cross-style generalization while providing inspectable
reasoning traces for
multimodal tasks. The code and implementation are available at https://github.com/scheshmi/CrossStyle-MMR.
Read the full abstract ▾
All authors Seyyed Saeid Cheshmi, Hahnemann Ortiz, James Mooney, Dongyeop Kang
Machine Learning
Accelerated Sinkhorn Algorithms for Partial Optimal Transport
Nghia Thu Truong, Qui Phu Pham, Quang Nguyen et al. (+2 more)
Partial Optimal Transport (POT) addresses the problem of transporting only a fraction of the total mass between two distributions, making it suitable when marginals have unequal size or contain outliers. While Sinkhorn-based methods are widely used, their complexity bounds for POT remain suboptimal and can limit scalability. We introduce Accelerated Sinkhorn for POT (ASPOT), which integrates alternating minimization with Nesterov-style acceleration in the POT setting, yielding a complexity of $\mathcal{O}(n^{7/3}\varepsilon^{-5/3})$. We also show that an informed choice of the entropic parameter $\gamma$ improves rates for the classical Sinkhorn method. Experiments on real-world applications validate our theories and demonstrate the favorable performance of our proposed methods.
Read the full abstract ▾
All authors Nghia Thu Truong, Qui Phu Pham, Quang Nguyen, Dung Luong, Mai Tran
Machine Learning
PUNCH: Physics-informed Uncertainty-aware Network for Coronary Hemodynamics
Sukirt Thakur, Marcus Roper, Yang Zhou et al. (+8 more)
More than 10 million coronary angiograms are performed globally each year, providing a gold standard for detecting obstructive coronary artery disease. Yet, no obstructive lesions are identified in 70% of patients evaluated for ischemic heart disease. Up to half of these patients have undiagnosed, life-limiting coronary microvascular dysfunction (CMD), which remains under-detected due to the limited availability of invasive tools required to measure coronary flow reserve (CFR). Here, we introduce PUNCH, a non-invasive, uncertainty-aware framework for estimating CFR directly from standard coronary angiography. PUNCH integrates physics-informed neural networks with variational inference to infer coronary blood flow from first-principles models of contrast transport, without requiring ground-truth flow measurements or population-level training. The pipeline runs in approximately three minutes per patient on a single GPU. Validated on synthetic angiograms with controlled noise and imaging artifacts, as well as on clinical bolus thermodilution data from 20 patients, PUNCH demonstrates accurate and uncertainty-calibrated CFR estimation. This approach establishes a new paradigm for CMD diagnosis and illustrates how physics-informed inference can substantially expand the diagnostic utility of available clinical imaging.
Read the full abstract ▾
All authors Sukirt Thakur, Marcus Roper, Yang Zhou, Dmitry Yu. Isaev, Reza Akbarian Bafghi, Brahmajee K. Nallamothu, C. Alberto Figueroa, Srinivas Paruchuri, Scott Burger, Carlos Collet, Maziar Raissi
Machine Learning
Rethinking Benchmarks for Differentially Private Image Classification
Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra et al. (+2 more)
We revisit
benchmarks for differentially private image classification. We suggest a comprehensive set of
benchmarks, allowing researchers to evaluate techniques for differentially private machine learning in a variety of settings, including with and without additional data, in convex settings, and on a variety of qualitatively different
datasets. We further test established techniques on these
benchmarks in order to see which ideas remain effective in different settings. Finally, we create a publicly available leader board for the community to track progress in differentially private machine learning.
Read the full abstract ▾
All authors Sabrina Mokhtari, Sara Kodeiri, Shubhankar Mohapatra, Florian Tramèr, Gautam Kamath
Machine Learning · Cross-listed from q-bio.GN
FASTR: Reimagining FASTQ via Compact Image-inspired Representation
Adrian Tkachenko, Sepehr Salem, Ayotomiwa Ezekiel Adeniyi et al. (+7 more)
Motivation: High-throughput sequencing (HTS) enables population-scale genomics but generates massive
datasets, creating bottlenecks in storage, transfer, and analysis. FASTQ, the standard format for over two decades, stores one byte per base and one byte per quality score, leading to inefficient I/O, high storage costs, and redundancy. Existing compression tools can mitigate some issues, but often introduce costly decompression or complex dependency issues. Results: We introduce FASTR, a lossless, computation-native successor to FASTQ that encodes each nucleotide together with its base quality score into a single 8-bit value. FASTR reduces file size by at least 2x while remaining fully reversible and directly usable for downstream analyses. Applying general-purpose compression tools on FASTR consistently yields higher compression ratios, 2.47, 3.64, and 4.8x faster compression, and 2.34, 1.96, 1.75x faster decompression than on FASTQ across Illumina, HiFi, and ONT reads. FASTR is machine-learning-ready, allowing reads to be consumed directly as numerical vectors or image-like representations. We provide a highly parallel software ecosystem for FASTQ-FASTR conversion and show that FASTR integrates with existing tools, such as minimap2, with minimal interface changes and no performance overhead. By eliminating decompression costs and reducing data movement, FASTR lays the foundation for scalable genomics analyses and real-time sequencing workflows. Availability and Implementation: https://github.com/ALSER-Lab/FASTR
Read the full abstract ▾
All authors Adrian Tkachenko, Sepehr Salem, Ayotomiwa Ezekiel Adeniyi, Zulal Bingol, Mohammed Nayeem Uddin, Akshat Prasanna, Alexander Zelikovsky, Serghei Mangul, Can Alkan, Mohammed Alser
Machine Learning
Federated Proximal Optimization for Privacy-Preserving Heart Disease Prediction: A Controlled Simulation Study on Non-IID Clinical Data
Farzam Asad, Junaid Saif Khan, Maria Tariq et al. (+2 more)
Healthcare institutions have access to valuable patient data that could be of great help in the development of improved diagnostic models, but
privacy regulations like HIPAA and GDPR prevent hospitals from directly sharing data with one another.
Federated Learning offers a way out to this problem by facilitating collaborative model training without having the raw patient data centralized. However, clinical
datasets intrinsically have non-IID (non-independent and identically distributed) features brought about by demographic disparity and diversity in disease prevalence and institutional practices. This paper presents a comprehensive simulation research of
Federated Proximal Optimization (FedProx) for Heart Disease prediction based on UCI Heart Disease
dataset. We generate realistic non-IID data partitions by simulating four heterogeneous hospital clients from the Cleveland Clinic
dataset (303 patients), by inducing statistical heterogeneity by demographic-based stratification. Our experimental results show that FedProx with proximal parameter mu=0.05 achieves 85.00% accuracy, which is better than both centralized learning (83.33%) and isolated local models (78.45% average) without revealing patient
privacy. Through generous sheer ablation studies with statistical validation on 50 independent runs we demonstrate that proximal regularization is effective in curbing client drift in heterogeneous environments. This proof-of-concept research offers algorithmic insights and practical deployment guidelines for real-world
federated healthcare systems, and thus, our results are directly transferable to hospital IT-administrators, implementing
privacy-preserving collaborative learning.
Read the full abstract ▾
All authors Farzam Asad, Junaid Saif Khan, Maria Tariq, Sundus Munir, Muhammad Adnan Khan
Machine Learning
Conservative & Aggressive NaNs Accelerate U-Nets for Neuroimaging
Inés Gonzalez-Pepe, Vinuyan Sivakolunthu, Jacob Fortin et al. (+2 more)
Deep learning models for neuroimaging increasingly rely on large architectures, making efficiency a persistent concern despite advances in hardware. Through an analysis of numerical uncertainty of convolutional neural networks (CNNs), we observe that many operations are applied to values dominated by numerical noise and have negligible influence on model outputs. In some models, up to two-thirds of convolution operations appear redundant. We introduce Conservative & Aggressive NaNs, two novel variants of max pooling and unpooling that identify numerically unstable voxels and replace them with NaNs, allowing subsequent layers to skip computations on irrelevant data. Both methods are implemented within PyTorch and require no architectural changes. We evaluate these approaches on four CNN models spanning neuroimaging and image classification tasks. For inputs containing at least 50% NaNs, we observe consistent runtime improvements; for data with more than two-thirds NaNs )common in several neuroimaging settings) we achieve an average inference speedup of 1.67x. Conservative NaNs reduces convolution operations by an average of 30% across models and
datasets, with no measurable performance degradation, and can skip up to 64.64% of convolutions in specific layers. Aggressive NaNs can skip up to 69.30% of convolutions but may occasionally affect performance. Overall, these methods demonstrate that numerical uncertainty can be exploited to reduce redundant computation and improve inference efficiency in CNNs.
Read the full abstract ▾
All authors Inés Gonzalez-Pepe, Vinuyan Sivakolunthu, Jacob Fortin, Yohan Chatelain, Tristan Glatard
Machine Learning · Cross-listed from cs.CL
Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text
Tunazzina Islam
Large
language models (LLMs) are increasingly capable of generating personalized, persuasive text at scale, raising new questions about bias and
fairness in automated communication. This paper presents the first systematic analysis of how LLMs behave when tasked with demographic-conditioned targeted messaging. We introduce a controlled evaluation framework using three leading models: GPT-4o, Llama-3.3, and Mistral-Large-2.1, across two generation settings: Standalone Generation, which isolates intrinsic demographic effects, and Context-Rich Generation, which incorporates thematic and regional context to emulate realistic targeting. We evaluate generated messages along three dimensions: lexical content,
language style, and persuasive framing. We instantiate this framework on
climate communication and find consistent age- and gender-based asymmetries across models: male- and youth-targeted messages tend to emphasize more assertive and progressive framing, while female- and senior-targeted messages more often reflect warmth, care, and traditional themes. Contextual
prompts systematically amplify these disparities, with persuasion scores being higher for male-targeted messages, while age-related differences vary across models. Our findings demonstrate how demographic stereotypes can surface and intensify in LLM-generated targeted communication, underscoring the need for bias-aware generation pipelines and transparent
auditing frameworks that explicitly account for demographic conditioning in socially sensitive applications.
Read the full abstract ▾
Machine Learning · Cross-listed from physics.med-ph
Benchmarking Deep Learning-Based Reconstruction Methods for Photoacoustic Computed Tomography with Clinically Relevant Synthetic Datasets
Panpan Chen, Seonyeong Park, Gangwon Jeong et al. (+3 more)
Deep learning (DL)-based image reconstruction methods for photoacoustic computed
tomography (PACT) have developed rapidly in recent years. However, most existing methods have not employed standardized
datasets, and their evaluations rely on traditional image quality (IQ) metrics that may lack clinical relevance. The absence of a standardized framework for clinically meaningful IQ assessment hinders fair comparison and raises concerns about the reproducibility and reliability of reported advancements in PACT. A benchmarking framework is proposed that provides open-source, anatomically plausible synthetic
datasets and evaluation strategies for DL-based acoustic inversion methods in PACT. The
datasets each include over 11,000 two-dimensional (2D)
stochastic breast objects with clinically relevant lesions and paired measurements at varying modeling complexity. The evaluation strategies incorporate both traditional and task-based IQ measures to assess fidelity and clinical utility. A preliminary benchmarking study is conducted to demonstrate the framework's utility by comparing DL-based and physics-based reconstruction methods. The benchmarking study demonstrated that the proposed framework enabled comprehensive, quantitative comparisons of reconstruction performance and revealed important limitations in certain DL-based methods. Although they performed well according to traditional IQ measures, they often failed to accurately recover lesions. This highlights the inadequacy of traditional metrics and motivates the need for task-based assessments. The proposed benchmarking framework enables systematic comparisons of DL-based acoustic inversion methods for 2D PACT. By integrating clinically relevant synthetic
datasets with rigorous evaluation protocols, it enables reproducible, objective assessments and facilitates method development and system optimization in PACT.
Read the full abstract ▾
All authors Panpan Chen, Seonyeong Park, Gangwon Jeong, Refik Mert Cam, Umberto Villa, Mark A. Anastasio
Machine Learning · Cross-listed from stat.ML
Information-Theoretic Causal Bounds under Unmeasured Confounding
Yonghan Jung and Bogyeong Kang
We develop a data-driven information-theoretic framework for sharp partial identification of causal effects under unmeasured confounding. Existing approaches often rely on restrictive assumptions, such as bounded or discrete outcomes; require external inputs (for example, instrumental variables, proxies, or user-specified sensitivity parameters); necessitate full structural causal model specifications; or focus solely on population-level averages while neglecting
covariate-conditional effects. We overcome all four limitations simultaneously by establishing novel information-theoretic, data-driven divergence bounds. Our key theoretical contribution shows that the f-divergence between the observational distribution P(Y | A = a, X = x) and the interventional distribution P(Y | do(A = a), X = x) is upper bounded by a function of the propensity score alone. This result enables sharp partial identification of conditional causal effects directly from observational data, without requiring external sensitivity parameters, auxiliary variables, full structural specifications, or outcome boundedness assumptions. For practical implementation, we develop a semiparametric
estimator satisfying Neyman orthogonality (Chernozhukov et al., 2018), which ensures root-n consistent inference even when nuisance functions are estimated via flexible machine learning methods. Simulation studies and real-world data applications, implemented in the GitHub repository (https://github.com/yonghanjung/Information-Theretic-Bounds), demonstrate that our framework provides tight and valid causal bounds across a wide range of data-generating processes.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Interpretability of the Intent Detection Problem: A New Approach
Eduardo Sanchez-Karhunen, Jose F. Quesada-Moreno and Miguel A. Gutiérrez-Naranjo
Intent detection, a fundamental text classification task, aims to identify and label the semantics of user queries, playing a vital role in numerous business applications. Despite the dominance of deep learning techniques in this field, the internal mechanisms enabling Recurrent Neural Networks (RNNs) to solve intent detection tasks are poorly understood. In this work, we apply dynamical systems theory to analyze how RNN architectures address this problem, using both the balanced SNIPS and the imbalanced ATIS
datasets. By interpreting sentences as trajectories in the hidden state space, we first show that on the balanced SNIPS
dataset, the network learns an ideal solution: the state space, constrained to a low-dimensional
manifold, is partitioned into distinct clusters corresponding to each intent. The application of this framework to the imbalanced ATIS
dataset then reveals how this ideal
geometric solution is distorted by class imbalance, causing the clusters for low-frequency intents to degrade. Our framework decouples
geometric separation from readout
alignment, providing a novel, mechanistic explanation for real world performance disparities. These findings provide new insights into RNN dynamics, offering a
geometric interpretation of how
dataset properties directly shape a network's computational solution.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ME
Falsifying Discriminant Validity of Predictive Algorithms
Amanda Coston
Empirical investigations into unintended model behavior often show that the
algorithm is predicting another outcome than what was intended. These exposés highlight the need to identify when
algorithms predict unintended quantities - ideally before deploying them into consequential settings. We propose a falsification framework that provides a principled statistical test for discriminant validity: the requirement that an
algorithm predict intended outcomes better than impermissible ones. Drawing on falsification practices from causal inference, econometrics, and psychometrics, our framework compares calibrated prediction losses across outcomes to assess whether the
algorithm exhibits discriminant validity with respect to a specified impermissible proxy. In settings where the target outcome is difficult to observe, multiple permissible proxy outcomes may be available; our framework accommodates both this setting and the case with a single permissible proxy. Throughout we use nonparametric hypothesis testing methods that make minimal assumptions on the data-generating process. We illustrate the method in an admissions setting, where the framework establishes discriminant validity with respect to gender but fails to establish discriminant validity with respect to race. This demonstrates how falsification can serve as an early validity check. We also provide analysis in a criminal justice setting, where we highlight the limitations of our framework and emphasize the need for complementary approaches to assess other aspects of construct validity and external validity.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.DC
Communication-Avoiding Linear Algebraic Kernel K-Means on GPUs
Julian Bellavita, Matthew Rubino, Nakul Iyer et al. (+4 more)
Clustering is an important tool in data analysis, with K-means being popular for its simplicity and versatility. However, it cannot handle non-linearly separable clusters. Kernel K-means addresses this limitation but requires a large kernel
matrix, making it computationally and
memory intensive. Prior work has accelerated Kernel K-means by formulating it using sparse linear
algebra primitives and implementing it on a single GPU. However, that approach cannot run on
datasets with more than approximately 80,000 samples due to limited GPU
memory. In this work, we address this issue by presenting a suite of distributed-
memory parallel
algorithms for large-scale Kernel K-means clustering on multi-GPU systems. Our approach maps the most computationally expensive components of Kernel K-means onto communication-efficient distributed linear
algebra primitives uniquely tailored for Kernel K-means, enabling highly scalable implementations that efficiently cluster million-scale
datasets. Central to our work is the design of partitioning schemes that enable communication-efficient composition of the linear
algebra primitives that appear in Kernel K-means. Our 1.5D
algorithm consistently achieves the highest performance, enabling Kernel K-means to scale to data one to two orders of magnitude larger than previously practical. On 256 GPUs, it achieves a
geometric mean weak scaling efficiency of $79.7\%$ and a
geometric mean strong scaling speedup of $4.2\times$. Compared to our 1D
algorithm, the 1.5D approach achieves up to a $3.6\times$ speedup on 256 GPUs and reduces clustering time from over an hour to under two seconds relative to a single-GPU sliding window implementation. Our results show that distributed
algorithms designed with application-specific linear algebraic formulations can achieve substantial performance improvement.
Read the full abstract ▾
All authors Julian Bellavita, Matthew Rubino, Nakul Iyer, Andrew Chang, Aditya Devarakonda, Flavio Vella, Giulia Guidi
Machine Learning
ConceptACT: Episode-Level Concepts for Sample-Efficient Robotic Imitation Learning
Jakob Karalus and Friedhelm Schwenker
Imitation learning enables
robots to acquire complex manipulation skills from human demonstrations, but current methods rely solely on low-level sensorimotor data while ignoring the rich semantic knowledge humans naturally possess about tasks. We present ConceptACT, an extension of Action Chunking with
Transformers that leverages episode-level semantic concept annotations during training to improve learning efficiency. Unlike
language-conditioned approaches that require semantic input at deployment, ConceptACT uses human-provided concepts (object properties, spatial relationships, task constraints) exclusively during demonstration collection, adding minimal annotation burden. We integrate concepts using a modified
transformer architecture in which the final encoder layer implements concept-aware cross-
attention, supervised to align with human annotations. Through experiments on two robotic manipulation tasks with logical constraints, we demonstrate that ConceptACT converges faster and achieves superior sample efficiency compared to standard ACT. Crucially, we show that architectural integration through
attention mechanisms significantly outperforms naive auxiliary prediction losses or
language-conditioned models. These results demonstrate that properly integrated semantic supervision provides powerful inductive biases for more efficient
robot learning.
Read the full abstract ▾
Machine Learning
Learning to Collaborate: An Orchestrated-Decentralized Framework for Peer-to-Peer LLM Federation
Inderjeet Singh, Eleonore Vissol-Gaudin, Andikan Otung et al. (+1 more)
Fine-tuning Large
Language Models (LLMs) for specialized domains is constrained by a fundamental challenge: the need for diverse, cross-organizational data conflicts with the principles of data
privacy and sovereignty. While
Federated Learning (FL) provides a framework for collaboration without raw data exchange, its classic centralized form introduces a single point of failure and remains vulnerable to model inversion attacks. Decentralized FL (DFL) mitigates this risk by removing the central aggregator but typically relies on inefficient, random peer-to-peer (P2P) pairings, forming a collaboration graph that is blind to
agent heterogeneity and risks negative transfer. This paper introduces KNEXA-FL, a novel framework for orchestrated decentralization that resolves this trade-off. KNEXA-FL employs a non-aggregating Central Profiler/Matchmaker (CPM) that formulates P2P collaboration as a contextual bandit problem, using a LinUCB
algorithm on abstract
agent profiles to learn an optimal matchmaking policy. It orchestrates direct knowledge exchange between heterogeneous, PEFT-based LLM
agents via secure
distillation, without ever accessing the models themselves. Our comprehensive experiments on a challenging code generation task show that KNEXA-FL yields substantial gains, improving Pass@1 by approx. 50% relative to random P2P collaboration. Critically, our orchestrated approach demonstrates stable convergence, in stark contrast to a powerful centralized
distillation baseline which suffers from catastrophic performance collapse. Our work establishes adaptive, learning-based orchestration as a foundational principle for building robust and effective decentralized AI ecosystems.
Read the full abstract ▾
All authors Inderjeet Singh, Eleonore Vissol-Gaudin, Andikan Otung, Motoyoshi Sekiya
Machine Learning
Impact of Graph Structure on Membership-Inference Risk for Graph Neural Networks
Megha Khosla
Graph neural networks (GNNs) are widely used for tasks such as node classification and link prediction, but their use in sensitive settings raises concerns about training-data leakage. Prior work on
privacy leakage in GNNs largely borrows assumptions from non-graph domains, overlooking the role of graph structure. We argue for a graph-specific analysis of
privacy risk and study how graph structure affects node-level membership inference. We formalize membership inference (MI) over node-neighborhood tuples and investigate two important dimensions: (i) training-graph construction and (ii) inference-time edge access. We compare snowball sampling, a structure-aware procedure, with uniform random node sampling for constructing training graphs. Our experiments show that snowball sampling often hurts generalization relative to random sampling due to its coverage bias. In contrast, allowing access to inter-train-test edges at inference improves test accuracy, reduces the train-test gap, while also having a strong and setting-dependent effect on membership advantage. These results show that graph structure directly shapes
privacy risk. We further show that the generalization gap, measured as the performance difference between training and test nodes, is an incomplete proxy for membership inference risk: membership advantage can rise or fall independently of changes in this gap, with inference-time edge access often playing a crucial role. Theoretically, we show that for node-level tasks, standard
privacy-
auditing results based on membership inference do not directly carry over to inductive graph settings, because training and test nodes are structurally dependent rather than interchangeable. We release the code and data at https://github.com/PriXAI/GraphStructurePrivacyAnalysis-public.
Read the full abstract ▾
Machine Learning · Cross-listed from hep-ex
Foundation model for particle collision data analysis
Ting-Hsiang Hsu, Bai-Hong Zhou, Qibin Liu et al. (+8 more)
Data-intensive
collider experiments increasingly require models that transfer across high-dimensional, partially observed measurements, yet most analyses still optimize separate models for each task. Here, to address this bottleneck, we introduce EVENET, an event-level foundation model pretrained on 500 million simulated collision events using self-supervised learning and physics-informed supervision. The model constructs a shared particle-cloud representation from the variable-length set of physics objects in each event. Across searches for heavy resonances and exotic
Higgs decays, EVENET outperforms both established and state-of-the-art baselines, particularly in low-statistics regimes. We validate transfer to experimental data by rediscovering the $\Upsilon$
meson in CMS Open Data. The pretrained model also supports precise extraction of
quantum-correlation observables that remain stable under the tested systematic variations. Together, these results establish a reusable backbone for diverse
collider analyses, improving downstream adaptation after a substantial one-time pretraining investment.
Read the full abstract ▾
All authors Ting-Hsiang Hsu, Bai-Hong Zhou, Qibin Liu, Yue Xu, Shu Li, George Wei-Shu Hou, Benjamin Nachman, Shih-Chieh Hsu, Vinicius Mikuni, Yuan-Tang Chou, Yulei Zhang
Machine Learning
Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization
A. El Ichi and K. Jbilou
Large
Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural
language tasks but suffer from extremely large
memory footprints and computational costs. In this paper, we introduce a tensor compression framework based on the cproduct for computing low rank approximation In the first part of our approach, we leverage the algebraic structure of the cproduct to represent weight tensors such as those in
embedding layers,
attention projections, and feed forward networks in a transform domain where frontal slices can be jointly approximated by low rank tensor factors. This enables computationally efficient compression that exploits multidimensional correlations beyond traditional SVD methods.
Read the full abstract ▾
Machine Learning
Least-Loaded Expert Parallelism: Load Balancing An Imbalanced Mixture-of-Experts
Xuan-Phi Nguyen, Shrey Pandit, Austin Xu et al. (+2 more)
Mixture-of-Experts (MoE) models are typically pre-trained with explicit load-balancing constraints to ensure statistically balanced expert routing. Despite this, we observe that even well-trained MoE models exhibit significantly imbalanced routing. This behavior is arguably natural-and even desirable - as imbalanced routing allows models to concentrate domain-specific knowledge within a subset of experts. Expert parallelism (EP) is designed to scale MoE models by distributing experts across multiple devices, but with a less-discussed assumption of balanced routing. Under extreme imbalance, EP can funnel a disproportionate number of
tokens to a small number of experts, leading to compute- and
memory-bound failures on overloaded devices during post-training or inference, where explicit load balancing is often inapplicable. We propose Least-Loaded Expert Parallelism (LLEP), a novel EP
algorithm that dynamically reroutes excess
tokens and associated expert parameters from overloaded devices to underutilized ones. This ensures that all devices complete their workloads within the minimum collective latency while respecting
memory constraints. Across different model scales, LLEP achieves up to 5x speedup and 4x reduction in peak
memory usage compared to standard EP. This enables faster and higher-throughput post-training and inference, with ~1.9x faster for gpt-oss-120b. We support our method with extensive theoretical analysis and comprehensive empirical evaluations, including ablation studies. These results illuminate key trade-offs and enable a principled framework for hardware-specific hyper-parameter tuning to achieve optimal performance.
Read the full abstract ▾
All authors Xuan-Phi Nguyen, Shrey Pandit, Austin Xu, Caiming Xiong, Shafiq Joty
Machine Learning · Cross-listed from cs.CY
Forecasting Energy Consumption using Recurrent Neural Networks: A Comparative Analysis
Abhishek Maity and Viraj Tukarul
Accurate short-term energy consumption
forecasting is essential for efficient power grid management, resource allocation, and
market stability. Traditional time-series models often fail to capture the complex, non-linear dependencies and external factors affecting energy demand. In this study, we propose a
forecasting approach based on Recurrent Neural Networks (RNNs) and their advanced variant, Long Short-Term
Memory (LSTM) networks. Our methodology integrates historical energy consumption data with external variables, including temperature, humidity, and time-based features. The LSTM model is trained and evaluated on a publicly available
dataset, and its performance is compared against a conventional feed-forward neural network baseline. Experimental results show that the LSTM model substantially outperforms the baseline, achieving lower Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). These findings demonstrate the effectiveness of deep learning models in providing reliable and precise short-term energy forecasts for real-world applications.
Read the full abstract ▾