Machine Learning
Architecture–Optimization Co-Design for Physics-Informed Neural Networks via Layer-wise Coordinate Adaptation and Gradient Conflict Resolution
Pancheng Niu, Jun Guo, Qiaolin He et al. (+2 more)
Physics-informed neural networks (PINNs) can be limited by coordinate representations and conflicting gradients from heterogeneous physical constraints. We propose Architecture–Conflict-Resolved PINN (ACR-PINN), combining Layer-wise Dynamic Adaptation (LDA) and Gradient-Conflict-Resolved PINN (GCR-PINN). LDA constructs layer-specific coordinate features and fuses two encoding branches through input-conditioned, feature-wise gates. GCR-PINN treats PDE, initial-condition, and boundary-condition losses as separate tasks and conditionally projects negatively aligned gradients before aggregation, without changing the physical loss formulation. Across seven
benchmark problems, ACR-PINN achieves the lowest observed mean across all metrics among four core architecture–optimizer combinations, reducing mean relative $L_2$ error by approximately 56–97% relative to Std-PINN and improving on the better single-component variant. Capacity-controlled comparisons and gate interventions indicate that these gains reflect the learned coordinate-fusion pathway rather than parameter count alone. Gradient diagnostics suggest that LDA can improve task-gradient compatibility and reduce the correction required by GCR-PINN, with effects depending on the problem and update rule. Comparisons with representative PINN baselines and tests over sampling and temporal-horizon changes support the observed accuracy gains within the studied settings, while equal-time experiments reveal an accuracy–cost trade-off from the richer LDA representation.
Read the full abstract ▾
All authors Pancheng Niu, Jun Guo, Qiaolin He, Yongming Chen, Yanchao Shi
Machine Learning
Deterministic Dynamics of Sampling Processes in Score-Based Diffusion Models with Multiplicative Noise Conditioning
Doheon Kim
Score-based
diffusion models generate new samples by learning the score function associated with a
diffusion process. While the effectiveness of these models can be theoretically explained using differential equations related to the sampling process, previous work by Song and Ermon (2020) demonstrated that neural networks using multiplicative noise conditioning can still generate satisfactory samples. In this setup, the model is expressed as the product of two functions: one depending on the spatial variable and the other on the noise magnitude. This structure limits the model's ability to represent a more general relationship between the spatial variable and the noise, indicating that it cannot fully learn the correct score. Despite this limitation, the models perform well in practice. In this work, we provide a theoretical explanation for this phenomenon by studying the deterministic dynamics of the associated differential equations, offering insight into how the model operates.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CY
AI-generated data contamination erodes pathological variability and diagnostic reliability
Hongyu He, Shaowen Xiang, Ye Zhang et al. (+15 more)
Generative artificial intelligence (AI) is rapidly populating medical records with synthetic content, creating a feedback loop where future models are increasingly at risk of training on uncurated AI-generated data. However, the clinical consequences of this AI-generated data contamination remain unexplored. Here, we show that in the absence of mandatory human verification, this self-referential cycle drives a rapid erosion of pathological variability and diagnostic reliability. By analysing more than 800,000 synthetic data points across clinical text generation,
vision-
language reporting, and medical image synthesis, we find that models progressively converge toward generic phenotypes regardless of the model architecture. Specifically, rare but critical findings, including pneumothorax and effusions, vanish from the synthetic content generated by AI models, while demographic representations skew heavily toward middle-aged male phenotypes. Crucially, this degradation is masked by false diagnostic confidence; models continue to issue reassuring reports while failing to detect life-threatening pathology, with false reassurance rates tripling to 40%. Blinded physician evaluation confirms that this decoupling of confidence and accuracy renders AI-generated documentation clinically useless after just two generations. We systematically evaluate three mitigation strategies, finding that while synthetic volume scaling fails to prevent collapse, mixing real data with quality-aware filtering effectively preserves diversity. Ultimately, our results suggest that without policy-mandated human oversight, the deployment of generative AI threatens to degrade the very healthcare data ecosystems it relies upon.
Read the full abstract ▾
All authors Hongyu He, Shaowen Xiang, Ye Zhang, Yingtao Zhu, Jin Zhang, Hao Deng, Emily Alsentzer, Yun Liu, Qingyu Chen, Kun-Hsing Yu, Andrew Marshall, Tingting Chen, Srinivas Anumasa, Daniel Ebner, Dean Ho, Kee Yuan Ngiam, Ching-Yu Cheng, Dianbo Liu
Machine Learning · Cross-listed from cs.CL
A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Bandits
Siguang Chen, Chunli Lv and Miao Xie
Large
language models (LLMs) have become powerful and widely used systems for
language understanding and generation, while multi-armed bandit (MAB)
algorithms provide a principled framework for adaptive decision-making under uncertainty. This survey explores the potential at the intersection of these two fields. As we know, it is the first survey to systematically review the bidirectional interaction between large
language models and multi-armed bandits at the component level. We highlight the bidirectional benefits: MAB
algorithms address critical LLM challenges, spanning from pre-training to
retrieval-augmented generation (RAG) and personalization. Conversely, LLMs enhance MAB systems by redefining core components such as arm definition and environment modeling, thereby improving decision-making in sequential tasks. We analyze existing LLM-enhanced bandit systems and bandit-enhanced LLM systems, providing insights into their design, methodologies, and performance. Key challenges and representative findings are identified to help guide future research. An accompanying GitHub repository that indexes relevant literature is available at https://github.com/bucky1119/Awesome-LLM-Bandit-Interaction.
Read the full abstract ▾
Machine Learning
Online Continual Learning for Time Series: a Natural Score-driven Approach
Edoardo Urettini, Daniele Atzeni, Ioanna-Yvonni Tsaknaki et al. (+1 more)
Online continual learning (OCL) methods adapt to changing environments without forgetting past knowledge. Similarly, online time series
forecasting (OTSF) is a real-world problem where data evolve in time and success depends on both rapid adaptation and long-term
memory. Indeed, time-varying and regime-switching
forecasting models have been extensively studied, offering a strong justification for the use of OCL in these settings. Building on recent work that applies OCL to OTSF, this paper aims to strengthen the theoretical and practical connections between time series methods and OCL. First, we reframe neural network optimization as a parameter filtering problem, showing that natural gradient descent is a score-driven method and proving its information-theoretic optimality. Then, we show that using a Student's t likelihood in addition to natural gradient induces a bounded update, which improves robustness to outliers. Finally, we introduce Natural Score-driven Replay (NatSR), which combines our robust optimizer with a replay buffer and a dynamic scale heuristic that improves fast adaptation at regime drifts. Empirical results demonstrate that NatSR achieves stronger
forecasting performance than more complex state-of-the-art methods.
Read the full abstract ▾
All authors Edoardo Urettini, Daniele Atzeni, Ioanna-Yvonni Tsaknaki, Antonio Carta
Machine Learning
An efficient heuristic for geometric analysis of cell deformations
Yaima Paz Soto, Silena Herold Garcia, Ximo Gual-Arnau et al. (+2 more)
Sickle
cell disease causes erythrocytes to become sickle-shaped, affecting their movement in the bloodstream and reducing oxygen delivery. It has a high global prevalence and places a significant burden on healthcare systems, especially in resource-limited regions. Automated classification of sickle
cells in blood images is crucial, allowing the specialist to reduce the effort required and avoid errors when quantifying the deformed
cells and assessing the severity of a crisis. Recent studies have proposed various erythrocyte representation and classification methods. Since classification depends solely on
cell shape, a suitable approach models erythrocytes as closed planar curves in shape space. This approach employs elastic distances between shapes, which are invariant under rotations, translations, scaling, and reparameterizations, ensuring consistent distance measurements regardless of the curves' position, starting point, or traversal speed. While previous methods exploiting shape space distances had achieved high accuracy, we refined the model by considering the
geometric characteristics of healthy and sickled erythrocytes. Our method proposes (1) to employ a fixed parameterization based on the major axis of each
cell to compute distances and (2) to align each
cell with two templates using this parameterization before computing distances. Aligning shapes to templates before distance computation, a concept successfully applied in areas such as molecular dynamics, and using a fixed parameterization, instead of minimizing distances across all possible parameterizations, simplifies calculations. This strategy achieves 96.03% accuracy rate in both supervised classification and unsupervised clustering. Our method ensures efficient erythrocyte classification, maintaining or improving accuracy over shape space models while significantly reducing computational costs.
Read the full abstract ▾
All authors Yaima Paz Soto, Silena Herold Garcia, Ximo Gual-Arnau, Antoni Jaume-i-Capó, Manuel González-Hidalgo
Machine Learning · Cross-listed from cs.CR
Your Privacy Depends on Others: Collusion Vulnerabilities in Individual Differential Privacy
Johannes Kaiser, Alexander Ziller, Eleni Triantafillou et al. (+2 more)
Individual Differential
Privacy (iDP) promises users control over their
privacy, but this promise can be broken in practice. We reveal a previously overlooked vulnerability in sampling-based iDP mechanisms: while conforming to the iDP guarantees, an individual's
privacy risk is not solely governed by their own
privacy budget, but critically depends on the
privacy choices of all other data contributors. This creates a mismatch between the promise of individual
privacy control and the reality of a system where risk is collectively determined. We demonstrate empirically that certain distributions of
privacy preferences can unintentionally inflate the
privacy risk of individuals, even when their formal guarantees are met. Moreover, this excess risk provides an exploitable attack vector. A central adversary or a set of colluding adversaries can deliberately choose
privacy budgets to amplify vulnerabilities of targeted individuals. Most importantly, this attack operates entirely within the guarantees of DP, hiding this excess vulnerability. Our empirical evaluation demonstrates successful attacks against 62% of targeted individuals, substantially increasing their membership inference susceptibility. To mitigate this, we propose $(\varepsilon_i,\delta_i,\overline{\Delta})$-iDP a
privacy contract that uses $\Delta$-divergences to provide users with a hard upper bound on their excess vulnerability, while offering flexibility to mechanism design. Our findings expose a fundamental challenge to the current paradigm, demanding a re-evaluation of how iDP systems are designed, audited, communicated, and deployed to make excess risks transparent and controllable.
Read the full abstract ▾
All authors Johannes Kaiser, Alexander Ziller, Eleni Triantafillou, Daniel Rückert, Georgios Kaissis
Machine Learning · Cross-listed from cs.RO
Dynamic Hand Gesture Recognition for Robot Manipulator Tasks
Dharmendra Sharma, Peeyush Thakur, Sandeep Gupta et al. (+2 more)
This paper proposes a novel approach to recognizing dynamic hand gestures facilitating seamless interaction between humans and
robots. Here, each
robot manipulator task is assigned a specific gesture. There may be several such tasks, hence, several gestures. These gestures may be prone to several dynamic variations. All such variations for different gestures shown to the
robot are accurately recognized in real-time using the proposed unsupervised model based on the Gaussian Mixture model. The accuracy during training and real-time testing prove the efficacy of this methodology.
Read the full abstract ▾
All authors Dharmendra Sharma, Peeyush Thakur, Sandeep Gupta, Narendra Kumar Dhar, Laxmidhar Behera
Machine Learning
CooperLLM: Cloud-Edge-End Cooperative Federated Fine-tuning for LLMs via ZOO-based Gradient Correction
He Sun, Jinrui Zhou, Li Li et al. (+1 more)
Large
Language Models (LLMs) perform well on many NLP tasks, but fine-tuning them on resource-constrained mobile devices is challenging due to high
memory and computation costs, despite growing demands for
privacy-preserving personalization.
Federated Learning (FL) enables local-data training, yet existing methods either rely on
memory-intensive backpropagation or use zeroth-order optimization (ZOO), which avoids backward passes but suffers from slow convergence and degraded accuracy. We propose CooperLLM, a cloud-assisted edge-end cooperative
federated fine-tuning framework that combines ZOO on mobile devices with cloud-guided gradient rectification. Mobile clients perform lightweight ZOO updates on private data, while the cloud fine-tunes on auxiliary public data using backpropagation and injects guided perturbations to rectify local updates, improving convergence and accuracy without violating
privacy. To address system bottlenecks, CooperLLM introduces pipeline scheduling and adaptive compression to overlap computation and communication and reduce
memory usage. Experiments on multiple
Transformer models and
datasets show that CooperLLM reduces on-device
memory by up to $86.4\%$, accelerates convergence by $8.8 \times$, and improves accuracy by up to 10 percentage points over state-of-the-art ZOO-based baselines.
Read the full abstract ▾
All authors He Sun, Jinrui Zhou, Li Li, Mingjun Xiao
Machine Learning · Cross-listed from cs.AI
Actionable Interpretability Must Be Defined in Terms of Symmetries
Pietro Barbiero, Mateo Espinosa Zarlenga, Francesco Giannini et al. (+4 more)
This paper argues that interpretability research in Artificial Intelligence (AI) is fundamentally ill-posed as existing definitions of interpretability fail to describe how interpretability can be formally tested or designed for. We posit that actionable definitions of interpretability must be formulated in terms of *
symmetries* that inform model design and lead to testable conditions. Under a probabilistic view, we hypothesise that four
symmetries (inference equivariance, information invariance, concept-closure invariance, and structural invariance) suffice to (i) formalise interpretable models as a subclass of probabilistic models, (ii) yield a unified formulation of interpretable inference (e.g.,
alignment, interventions, and counterfactuals) as a form of
Bayesian inversion, and (iii) provide a formal framework to verify compliance with safety standards and regulations.
Read the full abstract ▾
All authors Pietro Barbiero, Mateo Espinosa Zarlenga, Francesco Giannini, Alberto Termine, Filippo Bonchi, Mateja Jamnik, Giuseppe Marra
Machine Learning
Deep Temporal Graph Clustering: A Comprehensive Benchmark and Datasets
Meng Liu, Ke Liang, Siwei Wang et al. (+3 more)
Temporal Graph Clustering (TGC) is a new task with little
attention, focusing on node clustering in temporal graphs. Compared with existing static graph clustering, it can find the balance between time requirement and space requirement (Time-Space Balance) through the interaction sequence-based batch-processing pattern. However, there are two major challenges that hinder the development of TGC, i.e., inapplicable clustering techniques and inapplicable
datasets. To address these challenges, we propose a comprehensive
benchmark, called BenchTGC. Specially, we design a BenchTGC Framework to illustrate the paradigm of temporal graph clustering and improve existing clustering techniques to fit temporal graphs. In addition, we also discuss problems with public temporal graph
datasets and develop multiple
datasets suitable for TGC task, called BenchTGC
Datasets. According to extensive experiments, we not only verify the advantages of BenchTGC, but also demonstrate the necessity and importance of TGC task. We wish to point out that the dynamically changing and complex scenarios in real world are the foundation of temporal graph clustering. The code and data is available at: https://github.com/MGitHubL/BenchTGC.
Read the full abstract ▾
All authors Meng Liu, Ke Liang, Siwei Wang, Xingchen Hu, Sihang Zhou, Xinwang Liu
Machine Learning
Supervised Learning for the (s,S) Inventory Model with General Interarrival Demands and General Lead Times
Eliran Sherzer and Yonit Barron
The continuous-review (s,S) inventory model is a cornerstone of
stochastic inventory theory, yet its analysis becomes analytically intractable when dealing with non-Markovian systems. In such systems, evaluating long-run performance measures typically relies on costly simulation. This paper proposes a supervised learning framework via a neural network model for approximating stationary performance measures of (s,S) inventory systems with general distributions for the interarrival time between demands and lead times under lost sales. Simulations are first used to generate training labels, after which the neural network is trained. After training, the neural network provides almost instantaneous predictions of various metrics of the system, such as the stationary distribution of inventory levels, the expected cycle time, and the probability of lost sales. We find that using a small number of low-order moments of the distributions as input is sufficient to train the neural networks and to accurately capture the steady-state distribution. Extensive numerical experiments demonstrate high accuracy over a wide range of system parameters. As such, it effectively replaces repeated and costly simulation runs. Our framework is easily extendable to other inventory models, offering an efficient and fast alternative for analyzing complex
stochastic systems.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
TwoHead-SwinFPN: A Unified DL Architecture for Synthetic Manipulation, Detection and Localization in Identity Documents
Chan Naseeb, Adeel Ashraf Cheema, Hassan Sami et al. (+3 more)
The proliferation of sophisticated generative AI models has significantly escalated the threat of synthetic manipulations in identity documents, particularly through face swapping and text inpainting attacks. This paper presents TwoHead-SwinFPN, a unified deep learning architecture that simultaneously performs binary classification and precise localization of manipulated regions in ID documents. Our approach integrates a Swin
Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block
Attention Module (CBAM) for improved feature representation. The model employs a dual-head architecture for joint optimization of detection and
segmentation tasks, utilizing uncertainty-weighted multi-task learning. Extensive experiments on the FantasyIDiap
dataset demonstrate superior performance with 84.31% accuracy, 90.78% AUC for classification, and 57.24% mean Dice score for localization. The proposed method achieves an F1-score of 88.61% for binary classification while maintaining computational efficiency suitable for real-world deployment through FastAPI implementation. Our comprehensive evaluation includes ablation studies, cross-device generalization analysis, and detailed performance assessment across 10
languages and 3 acquisition devices.
Read the full abstract ▾
All authors Chan Naseeb, Adeel Ashraf Cheema, Hassan Sami, Tayyab Afzal, Muhammad Omair, Usman Habib
Machine Learning
AdaNODEs: Test Time Adaptation for Time Series Forecasting Using Neural ODEs
Ting Dang, Soumyajit Chatterjee, Hong Jia et al. (+3 more)
Test time adaptation (TTA) has emerged as a promising solution to adapt pre-trained models to new, unseen data distributions using unlabeled target domain data. However, most TTA methods are designed for independent data, often overlooking the time series data and rarely addressing
forecasting tasks. This paper presents AdaNODEs, an innovative source-free TTA method tailored explicitly for time series
forecasting. By leveraging Neural Ordinary Differential Equations (NODEs), we propose a novel adaptation framework that accommodates the unique characteristics of distribution shifts in time series data. Moreover, we innovatively propose a new loss function to tackle TTA for
forecasting tasks. AdaNODEs only requires updating limited model parameters, showing effectiveness in capturing temporal dependencies while avoiding significant
memory usage. Extensive experiments with one- and high-dimensional data demonstrate that AdaNODEs offer relative improvements of 5.88% and 28.4% over the SOTA baselines, especially demonstrating robustness across higher severity distribution shifts.
Read the full abstract ▾
All authors Ting Dang, Soumyajit Chatterjee, Hong Jia, Yu Wu, Flora Salim, Fahim Kawsar
Machine Learning · Cross-listed from cs.MA
Communication Methods in Multi-Agent Reinforcement Learning
Christoph Wittner
Multi-
agent reinforcement learning is a promising research area that extends established reinforcement learning approaches to problems formulated as multi-
agent systems. Recently, a multitude of communication methods have been introduced to this field to address problems such as partially observable environments, non-stationarity, and exponentially growing action spaces. Communication further enables efficient cooperation among all
agents interacting in an environment. This work aims at providing an overview of communication techniques in multi-
agent reinforcement learning. By an in-depth analysis of 29 publications on this topic, the strengths and weaknesses of explicit, implicit,
attention-based, graph-based, and hierarchical/role-based communication are evaluated. The results of this comparison show that there is no general, optimal communication framework for every problem. On the contrary, the choice of communication depends heavily on the problem at hand. The comparison also highlights the importance of communication methods with low computational overhead to enable scalability to environments where many
agents interact. Finally, the paper discusses current research gaps, emphasizing the need for standardized benchmarking of system-level metrics and improved robustness under realistic communication conditions to enhance the real-world applicability of these approaches.
Read the full abstract ▾
Machine Learning
Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decomposition
Mohammed Mudassir Uddin, Shahnawaz Alam and Mohammed Kaif Pasha
Extracting sparse
circuits from billion-parameter
transformers is constrained by $O(2^n)$ search cost and pervasive feature reuse across co-active pathways. Hierarchical Attribution Graph Decomposition (HAGD) addresses this through four stages: cross-layer transcoder training, spectral coarsening of attribution graphs, graph-neural-network (GNN)-guided hierarchical traversal, and causal intervention verification, reducing worst-case complexity to $O(n^2 \log n)$. Per-layer transcoders trained on the RedPajama corpus yield monosemantic dictionaries; gradient-activation products form weighted attribution graphs; normalized-
Laplacian spectral clustering builds multi-resolution hierarchies; an
attention-based GNN assigns
circuit-membership scores at successive coarsening stages. Evaluation spans GPT-2 (117M-774M), Pythia (1.4B-6.9B), and Llama (7B-70B) across modular arithmetic, parity computation, integer sorting, coreference resolution (WinoGrande), commonsense
reasoning (HellaSwag), and factual recall. Behavioral preservation reaches 91% ($\pm$2.3%) on modular arithmetic with 49-347-node
circuits, while ACDC exhausts
memory beyond 1.4B parameters. Cross-architecture transfer coefficients span 0.38-0.82, with within-family pairs (Llama-7B $\to$ Llama-70B) attaining 0.82. Limitations include omitted
attention-head
circuits, 15-20% unexplained reconstruction variance, ablation-based validation circularity, and uncertain interpretability of
circuits exceeding several hundred nodes.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Tracing the Latent Threads: A Mechanistic Study of How LLMs Represent and Operationalize Race and Ethnicity Cues
Shiyue Hu, Ruizhe Li and Yanjun Gao
Large
language models (LLMs) increasingly operate in high-stakes settings where demographic attributes such as race and ethnicity may be explicitly stated or implicitly suggested through textual cues. However, existing studies primarily document outcome-level disparities, offering limited insight into internal mechanisms underlying these effects. We present a mechanistic study of how race and ethnicity cues are represented and operationalized within LLMs. Using two publicly available
datasets spanning toxicity-related generation and clinical narrative understanding tasks, we analyze three open-source models with a reproducible interpretability pipeline combining probing,
neuron-level attribution, and targeted intervention. We find that sensitivity to demographic cues is distributed across internal units and varies substantially across models. These units often align with entangled semantic facets, including explicit group labels, geography,
language, culture, and associations related to stereotypes. Interventions on selected units can change some biased prediction patterns, but substantial residual effects remain, suggesting that effective mitigation requires understanding distributed, task-specific mechanisms rather than manipulating a small set of identified
neurons alone. Code: https://github.com/LARK-NLP-Lab/LLM-Bias-Interpretability.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
PDFInspect: A Unified Feature Extraction Framework for Malicious Document Detection
Sharmila S P
The increasing prevalence of malicious Portable Document Format (PDF) files necessitates robust and comprehensive feature extraction techniques for effective detection and analysis. This work presents a unified framework that integrates graph-based, structural, and metadata-driven analysis to generate a rich feature representation for each PDF document. The system extracts text from PDF pages and constructs undirected graphs based on pairwise word relationships, enabling the computation of graph-theoretic features such as node count, edge density, and clustering coefficient. Simultaneously, the framework parses embedded metadata to quantify character distributions,
entropy patterns, and inconsistencies across fields such as author, title, and producer. Temporal features are derived from creation and modification timestamps to capture behavioral signatures, while structural elements including, object streams, fonts, and embedded images, are quantified to reflect document complexity. Boolean flags for potentially malicious PDF constructs (e.g., JavaScript, launch actions) are also extracted. Together, these features form a high-dimensional vector representation (170 dimensions) that is well-suited for downstream tasks such as malware classification, anomaly detection, and forensic analysis. The proposed approach is scalable, extensible, and designed to support real-world PDF threat intelligence workflows.6
Read the full abstract ▾
Machine Learning
Generating Cyclic Conformers with Flow Matching in Cremer-Pople Coordinates
Luca Schaufelberger, Aline Hartgers and Kjell Jorner
Cyclic molecules are ubiquitous across applications in chemistry and biology. Their restricted conformational flexibility provides structural pre-organization that is key to their function in drug discovery and catalysis. However, reliably sampling the conformer ensembles of ring systems remains challenging. Here, we introduce PuckerFlow, a generative machine learning model that performs flow matching on the Cremer-Pople space, a low-dimensional internal coordinate system capturing the relevant degrees of freedom of rings. Our approach enables generation of valid closed rings by design and demonstrates strong performance in generating conformers that are both diverse and precise. We show that PuckerFlow outperforms other conformer generation methods on nearly all quantitative metrics and illustrate the potential of PuckerFlow for ring systems relevant to chemical applications, particularly in catalysis and drug discovery. This work enables efficient and reliable conformer generation of cyclic structures, paving the way towards modeling structure-property relationships and the property-guided generation of rings across a wide range of applications in chemistry and biology.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
Mining Citywide Dengue Spread Patterns in Singapore Through Hotspot Dynamics from Open Web Data
Liping Huang, Gaoxi Xiao, Stefan Ma et al. (+3 more)
Dengue, a mosquito-borne disease, continues to pose a persistent public health challenge in urban areas, particularly in tropical regions such as Singapore. Effective and affordable control requires anticipating where transmission risks are likely to emerge so that interventions can be deployed proactively rather than reactively. This study introduces a novel framework that uncovers and exploits latent transmission links between urban regions, mined directly from publicly available dengue case data. Instead of treating cases as isolated reports, we model how hotspot formation in one area is influenced by epidemic dynamics in neighboring regions. While mosquito movement is highly localized, long-distance transmission is often driven by human mobility, and in our case study, the learned network aligns closely with commuting flows, providing an interpretable explanation for citywide spread. These hidden links are optimized through gradient descent and used not only to forecast hotspot status but also to verify the consistency of spreading patterns, by examining the stability of the inferred network across consecutive weeks. Case studies on Singapore during 2013-2018 and 2020 show that four weeks of hotspot history are sufficient to achieve an average F-score of 0.79. Importantly, the learned transmission links align with commuting flows, highlighting the interpretable interplay between hidden epidemic spread and human mobility. By shifting from simply reporting dengue cases to mining and validating hidden spreading dynamics, this work transforms open web-based case data into a predictive and explanatory resource. The proposed framework advances epidemic modeling while providing a scalable, low-cost tool for public health planning, early intervention, and urban resilience.
Read the full abstract ▾
All authors Liping Huang, Gaoxi Xiao, Stefan Ma, Hechang Chen, Shisong Tang, Flora Salim
Machine Learning · Cross-listed from cs.AI
SCULPT: Constraint-Guided Pruned MCTS that Carves Efficient Paths for Mathematical Reasoning
Qitong Fang, Haotian Li and Xu Wang
Automated
agent workflows can enhance the problem-solving ability of large
language models (LLMs), but common search strategies rely on
stochastic exploration and often traverse implausible branches. This occurs because current pipelines sample candidate steps from generic
prompts or learned policies with weak domain priors, yielding near-random walks over operators, units, and formats. To promote ordered exploration, this paper introduces SCULPT, a constraint-guided approach for Monte Carlo Tree Search (MCTS) that integrates domain-aware scoring into selection, expansion, simulation, and backpropagation. SCULPT scores and prunes actions using a combination of symbolic checks (dimensional consistency, type compatibility, magnitude sanity, depth control, and diversity) and structural pattern guidance, thereby steering the search toward plausible
reasoning paths. Under matched LLM configurations, SCULPT yields stable improvements on multiple
datasets; additional results with GPT-5.2 assess executor transferability and performance on frontier
reasoning models. Overall, domain-aware constraints can improve accuracy while maintaining efficiency and
reasoning stability.
Read the full abstract ▾
Machine Learning
Knowledge-Integrated Representation Learning for Crypto Anomaly Detection under Extreme Label Scarcity; Relational Domain-Logic Integration with Retrieval-Grounded Context and Path-Level Explanations
Gyuyeon Na, Minjung Park, Soyoun Kim et al. (+2 more)
Detecting anomalous trajectories in decentralized crypto networks is fundamentally challenged by extreme label scarcity and the adaptive evasion strategies of illicit actors. While Graph Neural Networks (GNNs) effectively capture local structural patterns, they struggle to internalize multi hop, logic driven motifs such as fund dispersal and layering that characterize sophisticated money laundering, limiting their forensic accountability under regulations like the FATF Travel Rule. To address this limitation, we propose Relational Domain Logic Integration (RDLI), a framework that embeds expert derived heuristics as differentiable, logic aware latent signals within representation learning. Unlike static rule based approaches, RDLI enables the detection of complex transactional flows that evade standard message passing. To further account for
market volatility, we incorporate a
Retrieval Grounded Context (RGC) module that conditions anomaly scoring on regulatory and macroeconomic context, mitigating false positives caused by benign regime shifts. Under extreme label scarcity (0.01%), RDLI outperforms state of the art GNN baselines by 28.9% in F1 score. A micro expert user study further confirms that RDLI path level explanations significantly improve trustworthiness, perceived usefulness, and clarity compared to existing methods, highlighting the importance of integrating domain logic with contextual grounding for both accuracy and explainability.
Read the full abstract ▾
All authors Gyuyeon Na, Minjung Park, Soyoun Kim, Jungbin Shin, Sangmi Chai
Machine Learning
Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning
Ishir Garg, Neel Kolhe, Andy Peng et al. (+1 more)
Continual learning aims to enable neural networks to acquire new knowledge on sequential tasks. However, the key challenge in such settings is to learn new tasks without catastrophically forgetting previously learned tasks. We propose the Fisher-Orthogonal Projected Natural Gradient Descent (FOPNG) optimizer, which enforces Fisher-orthogonal constraints on parameter updates to preserve old task performance while learning new tasks. Unlike existing methods that operate in Euclidean parameter space, FOPNG projects gradients onto the Fisher-orthogonal complement of previous task gradients. This approach unifies natural gradient descent with orthogonal gradient methods within an information-
geometric framework. We provide theoretical analysis deriving the projected update, describe efficient and practical implementations using the diagonal Fisher, and demonstrate strong results on standard continual learning
benchmarks such as Permuted-MNIST, Split-MNIST, Rotated-MNIST, Split-CIFAR10, and Split-CIFAR100. Our code is available at https://github.com/ishirgarg/FOPNG.
Read the full abstract ▾
All authors Ishir Garg, Neel Kolhe, Andy Peng, Rohan Gopalam
Machine Learning · Cross-listed from cs.CV
Left-Right Symmetry Breaking in CLIP-style Vision-Language Models Trained on Synthetic Spatial-Relation Data
Takaki Yamamoto, Chihiro Noguchi and Toshihiro Tanizawa
Spatial understanding remains a key challenge in
vision-
language models. Yet it is still unclear whether such understanding is truly acquired, and if so, through what mechanisms. We present a controllable 1D image-text testbed to probe how left-right relational understanding emerges in
Transformer-based
vision and text encoders trained with a CLIP-style contrastive objective. We train lightweight
Transformer-based
vision and text encoders end-to-end on paired descriptions of one- and two-object scenes and evaluate generalization to unseen object pairs while systematically varying label and layout diversity. We find that contrastive training learns left-right relations and that label diversity, more than layout diversity, is the primary driver of generalization in this setting. To gain the mechanistic understanding, we perform an
attention decomposition and show that interactions between positional and
token embeddings induce a horizontal
attention gradient that breaks left-right
symmetry in the encoders; ablating this contribution substantially reduces left-right discrimination. Our results provide a mechanistic insight of when and how CLIP-style models acquire relational competence.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.IT
Joint Source-Channel-Generation Coding: From Distortion-oriented Reconstruction to Semantic-consistent Generation
Tong Wu, Zhiyong Chen, Guo Lu et al. (+4 more)
Conventional communication systems, including both separation-based coding and AI-driven joint source-channel coding (JSCC), are largely guided by Shannon's rate-distortion theory. However, relying on generic distortion metrics fails to capture complex human visual perception, often resulting in blurred or unrealistic reconstructions. In this paper, we propose Joint Source-Channel-Generation Coding (JSCGC), a novel paradigm that shifts the focus from deterministic reconstruction to probabilistic generation. JSCGC leverages a generative model at the receiver as a generator rather than a conventional decoder to parameterize the data distribution, enabling direct maximization of mutual information under channel constraints while controlling
stochastic sampling to produce outputs residing on the authentic data
manifold with high fidelity. We further derive a theoretical lower bound on the maximum semantic inconsistency with given transmitted mutual information, elucidating the fundamental limits of communication in controlling the generative process. Extensive experiments on image transmission demonstrate that JSCGC substantially improves perceptual quality and semantic fidelity, significantly outperforming conventional distortion-oriented JSCC methods.
Read the full abstract ▾
All authors Tong Wu, Zhiyong Chen, Guo Lu, Li Song, Feng Yang, Meixia Tao, Wenjun Zhang
Machine Learning
Semi-supervised Instruction Tuning for Large Language Models on Text-Attributed Graphs
Zixing Song and Irwin King
The emergent
reasoning capabilities of Large
Language Models (LLMs) offer a transformative paradigm for analyzing text-attributed graphs. While instruction tuning is the prevailing method for adapting pre-trained LLMs to graph learning tasks like node classification, it requires a substantial volume of annotated (INSTRUCTION, OUTPUT) pairs deriving from labeled nodes. This requirement is particularly prohibitive in the social domain, where obtaining expert labels for sensitive or evolving content is costly and slow. Furthermore, standard graph instruction tuning fails to exploit the vast amount of unlabeled nodes, which contain latent correlations due to edge connections that are beneficial for downstream predictions. To bridge this gap, we propose a novel Semi-supervised Instruction Tuning pipeline for Graph Learning, named SIT-Graph. Notably, SIT-Graph is model-agnostic and can be seamlessly integrated into any graph instruction tuning method that utilizes LLMs as the predictor. SIT-Graph operates via an iterative self-training process. Initially, the model is fine-tuned using instruction pairs constructed solely from the labeled nodes. Then it generates confidence-filtered pseudo-responses for unlabeled nodes to strategically augment the
dataset for the next round of fine-tuning. Finally, this iterative refinement progressively aligns the LLM with the underlying node correlations. Extensive experiments demonstrate that when incorporated into state-of-the-art graph instruction tuning methods, SIT-Graph significantly enhances their performance on text-attributed graph
benchmarks, achieving over 20% improvement under the low label ratio settings.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.AI
SL-CBM: Enhancing Concept Bottleneck Models with Semantic Locality for Better Interpretability
Hanwei Zhang, Luo Cheng, Rui Wen et al. (+3 more)
Explainable AI (XAI) is crucial for building transparent and trustworthy machine learning systems, especially in high-stakes domains. Concept Bottleneck Models (CBMs) have emerged as a promising ante-hoc approach that provides interpretable, concept-level explanations by explicitly modeling human-understandable concepts. However, existing CBMs often suffer from poor locality faithfulness, failing to spatially align concepts with meaningful image regions, which limits their interpretability and reliability. In this work, we propose SL-CBM (CBM with Semantic Locality), a novel extension that enforces locality faithfulness by generating spatially coherent saliency maps at both concept and class levels. SL-CBM integrates a 1x1 convolutional layer with a cross-
attention mechanism to enhance
alignment between concepts, image regions, and final predictions. Unlike prior methods, SL-CBM produces faithful saliency maps inherently tied to the model's internal
reasoning, facilitating more effective debugging and intervention. Extensive experiments on image
datasets demonstrate that SL-CBM substantially improves locality faithfulness, explanation quality, and intervention efficacy while maintaining competitive classification accuracy. Our ablation studies highlight the importance of contrastive and
entropy-based regularization for balancing accuracy, sparsity, and faithfulness. Overall, SL-CBM bridges the gap between concept-based
reasoning and spatial explainability, setting a new standard for interpretable and trustworthy concept-based models.
Read the full abstract ▾
All authors Hanwei Zhang, Luo Cheng, Rui Wen, Yang Zhang, Lijun Zhang, Holger Hermanns
Machine Learning
Distilling Time Series Foundation Models for Efficient Forecasting
Yuqi Li, Kuiye Ding, Chuanguang Yang et al. (+2 more)
Time Series foundation models (TSFMs) deliver strong
forecasting performance through large-scale pretraining, but their large parameter sizes make deployment costly. While knowledge
distillation offers a natural and effective approach for model compression, techniques developed for general machine learning tasks are not directly applicable to time series
forecasting due to the unique characteristics. To address this, we present DistilTS, the first
distillation framework specifically designed for TSFMs. DistilTS addresses two key challenges: (1) task difficulty discrepancy, specific to
forecasting, where uniform weighting makes optimization dominated by easier short-term horizons, while long-term horizons receive weaker supervision; and (2) architecture discrepancy, a general challenge in
distillation, for which we design an
alignment mechanism in the time series
forecasting. To overcome these issues, DistilTS introduces horizon-weighted objectives to balance learning across horizons, and a temporal
alignment strategy that reduces architectural mismatch, enabling compact models. Experiments on multiple
benchmarks demonstrate that DistilTS achieves
forecasting performance comparable to full-sized TSFMs, while reducing parameters by up to 1/150 and accelerating inference by up to 6000x. Code is available at: https://github.com/itsnotacie/DistilTS-ICASSP2026.
Read the full abstract ▾
All authors Yuqi Li, Kuiye Ding, Chuanguang Yang, Szu-Yu Chen, Yingli Tian
Machine Learning
Eddy-Resolving Global Ocean Forecasting with Multi-Scale Graph Neural Networks
Yuta Hirabayashi, Daisuke Matusoka and Konobu Kimura
Research on data-driven ocean models has progressed rapidly in recent years; however, the application of these models to global eddy-resolving ocean
forecasting remains limited. The accurate representation of ocean dynamics across a wide range of spatial scales remains a major challenge in such applications. This study proposes a multi-scale graph neural network-based ocean model for 10-day global
forecasting that improves short-term prediction skill and enhances the representation of multi-scale ocean variability. The model employs an encoder-processor-decoder architecture and uses two spherical meshes with different resolutions to better capture the multi-scale nature of ocean dynamics. In addition, the model incorporates surface atmospheric variables along with ocean state variables as node inputs to improve short-term prediction accuracy by representing atmospheric forcing. Evaluation using surface kinetic energy spectra and case studies shows that the model accurately represents a broad range of spatial scales, while root mean square error comparisons demonstrate improved skill in short-term predictions. These results indicate that the proposed model delivers more accurate short-term forecasts and improved representation of multi-scale ocean dynamics, thereby highlighting its potential to advance data-driven, eddy-resolving global ocean
forecasting.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
VISPA: Pluralistic Alignment via Automatic Value Selection and Activation
Shenyan Zheng, Jiayou Zhong, Anudeex Shetty et al. (+3 more)
As large
language models are increasingly used in high-stakes domains, it is essential that their outputs reflect not average} human preference, rather range of varying perspectives. Achieving such pluralism, however, remains challenging. Existing approaches consider limited values or rely on
prompt-level interventions, lacking value control and representation. To address this, we introduce VISPA, a training-free pluralistic
alignment framework, that enables direct control over value expression by dynamic selection and internal model activation steering. Across extensive empirical studies spanning multiple models and evaluation settings, we show VISPA is performant across all pluralistic
alignment modes in healthcare and beyond. Further analysis reveals VISPA is adaptable with different steering initiations, model, and/or values. These results suggest that pluralistic
alignment can be achieved through internal activation mechanisms, offering a scalable path toward
language models that serves all.
Read the full abstract ▾
All authors Shenyan Zheng, Jiayou Zhong, Anudeex Shetty, Heng Ji, Preslav Nakov, Usman Naseem