Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Physics Physics Data Analysis and Statistics

physics.data-an Watch Data Analysis and Statistics

Monday 12 October

Data Analysis and Statistics · Cross-listed from astro-ph.CO

Beyond the Kullback–Leibler divergence: the likelihood-ratio distribution as a detectability measure

Amirhossein Samandar, Mikel Martin Barandiaran, Linn Htat Lu et al. (+13 more)

The Kullback–Leibler (KL) divergence is the expectation value of the log-likelihood ratio between two specified models, evaluated with respect to one of the two models. It provides an average measure of the distinguishability of the models. We examine its connection to discovery potential and the accuracy of parameter estimates when one model represents a null hypothesis, and the alternative is a family of models characterized by one or more continuous parameters. For data drawn from a multivariate Gaussian distribution, we derive the log-likelihood-ratio distribution and its cumulants under two such hypotheses. We then search over the alternative model's parameter space and use peak prominence, the height of a likelihood-profile peak above its surrounding baseline, as an operational detection statistic. Repeating this search on simulated data drawn from the null model allows us to calibrate the prominence threshold for a specified false-positive rate. We study a toy Gaussian model as well as three cosmological examples involving cosmic topology, the tensor-to-scalar ratio, and dark energy. In all these cases, detectability of a particular model depends not only on KL divergence, but also on the size of the model parameter space, the likelihood-ratio distributions, and the distance of the model from any parameter space boundaries. The relationship between KL divergence, discovery potential, and parameter estimation must therefore be calibrated for each family of models and detection statistic under consideration.
Read the full abstract

All authors Amirhossein Samandar, Mikel Martin Barandiaran, Linn Htat Lu, Andrew H. Jaffe, Deyan P. Mihaylov, Glenn D. Starkman, Cynthia Trendafilova, Yashar Akrami, Stefano Anselmi, Javier Carrón Duque, Craig J. Copi, Fernando Cornet-Gomez, Arthur Kosowsky, Anna Negro, Ricardo G. Rodrigues, Andrius Tamosiunas

Data Analysis and Statistics · Cross-listed from stat.ME

Implicit and explicit reparameterizations of the Folded Normal and Rice distributions

Luis Aldama, Kevin Dalton, Minhuan Li et al. (+1 more)

Variational inference is a flexible tool for modeling experimental data and has proven useful in crystallography. Pathwise gradient estimators provide low-variance gradients for reparameterized variational inference, but are not readily available for some distributions central to crystallography. This work derives and implements implicit and explicit reparameterized gradient estimators for the folded Normal and Rice distributions in \prog{PyTorch}, and compares their variance analytically and empirically. In each case the implicit estimator achieves lower variance for the location derivative, particularly at small location-to-scale ratios, while the scale-derivative variances are nearly equal. For both distributions, implicit estimator is proved to be the conditional expectation of the explicit estimator given the sampled magnitude, so its variance is never larger. As a practical application, French-Wilson scaling is formulated as a variational inference problem, using the folded Normal and Rice distributions as variational families for centric and acentric reflections, respectively. Both estimators recover the exact posterior to better than $1\%$ on the amplitude, and converge at the same rate. The implementations are available in the open-source package \prog{rs-distributions}.
Read the full abstract

All authors Luis Aldama, Kevin Dalton, Minhuan Li, Doeke Hekstra

Data Analysis and Statistics · Cross-listed from physics.geo-ph

Bayesian Inversion Beyond the Generator's Range: Reducing Non-physical Constraints from Generative Neural Networks

Pratyay Roy, Xuebin Zhao and Andrew Curtis

To address non-uniqueness in geophysical inversions, Generative Neural Networks (GNNs) are often used to parameterise geological or other prior information in a lower-dimensional space (the so-called latent space), thereby reducing the number of inversion parameters. However, the training samples that represent this information are then always incorrect, because Earth structure is not intrinsically low-dimensional, and all GNNs are subject to training errors. Consequently, even when solutions are mapped back to the original higher-dimensional space, they inherit this imperfection and remain restricted to a low-dimensional manifold. From a mathematical point of view, latent space inversion thus fails fundamentally: generally, it can not even reproduce the best, higher-dimensional approximation to the true Earth. To overcome this limitation, we introduce an `escaping prior' distribution, which provides the necessary freedom to find solutions beyond the manifold of GNN-generated solutions, while still using the GNN as a guide. We use a hierarchical Bayesian approach to adapt the strength of prior information to the observed data and synthetic forward function $\textit{a posteriori}$. We illustrate our method on low-dimensional problems where the escaping prior is defined over a 1D manifold in 2D space. We then compare our approach with standard latent space and model space inversions using a uniform prior distribution in synthetic tomography problems, ranging from simple to more geologically realistic cases. Furthermore, we tested scenarios where the true solution is either far from or close to the manifold. In each case, our escaping prior shows significantly better results than the alternatives. It results in solutions beyond each GNN's generative capabilities, solutions which include the best possible model, and clearly demonstrates the limitations of latent space inversion.
Read the full abstract

Saturday 10 October

Data Analysis and Statistics · Cross-listed from cs.NI

Stackelberg Game with Multi-Agent Transformer for ISRS-Aware Resource Re-optimization in Multi-Band Optical Networks

Jiaxin Liu, Rentao Gu, Miao Zhu et al. (+3 more)

The rapid expansion of generative AI is driving computing infrastructures toward interconnected AI data centers. Multi-band optical networks have become the backbone of such connectivity. To accommodate increasingly dynamic traffic demands, these networks require continual resource re-optimization to improve resource utilization and reduce service blocking. However, multi-band transmission amplifies inter-channel stimulated Raman scattering(ISRS), inducing nonlinear coupling among services. This coupling complicates re-optimization, as local adjustments perturb the global power profile, creating interdependent optimization decisions across services. Moreover, the nonlinear interactions vary with the subset of services selected for re-optimization, and inappropriate selection may reduce re-optimization effectiveness or diminish achievable benefits. To address these challenges, we propose a Stackelberg Game with Multi-Agent Transformer (SG-MAT) approach for ISRS-aware resource re-optimization. SG-MAT employs a leader-follower structure, in which the MAT-based follower models nonlinear coupling and generates coordinated channel reallocations, feeding the utilization improvements to the leader. Using this feedback, the leader identifies critical service subsets, approximating a Stackelberg equilibrium to guide global decisions and enabling consistent and efficient re-optimization. Simulation results show that SG-MAT achieves the highest utilization improvement and reduces the blocking ratio by approximately 30% under moderate traffic loads, with computational complexity comparable to benchmarks.
Read the full abstract

All authors Jiaxin Liu, Rentao Gu, Miao Zhu, Zeguang Zou, Hui Li, Yuefeng Ji

Friday 9 October

Data Analysis and Statistics · Cross-listed from astro-ph.HE

A Pipeline for the AI-Assisted Analysis of Imaging X-ray Polarimetry Explorer Observations of Black Hole X-ray Binaries and Limits on Their Polarization Variability

B. Domine, B. Liu, H. Krawczynski et al. (+3 more)

The Imaging X-ray Polarimetry Explorer (IXPE) has opened a new window into the high-energy Universe, with the capability to measure the arrival time, arrival direction, energy, and, for the first time with the required sensitivity, the polarization of X-rays from black holes, neutron stars, and other cosmic sources of X-rays. In this paper, we discuss the use of Artificial Intelligence (AI) agents for the analysis of IXPE data. The pipeline combines an immutable code base with evolving knowledge databases. The generation of analysis projects makes use of a literature data base to judge the novelty of the proposed analysis projects. We emphasize that the analysis requires two types of expert knowledge: detailed instructions about how to reduce the X-ray data, and expert knowledge about how to interpret the results. We employ several levels of auditors that scrutinize the analysis plan, the analysis scripts, and the analysis results for data and method adequacy. The pipeline includes a staggered approach to interpret the analysis results. We present exemplary results from the pipeline applied to IXPE observations of Black Hole X-ray Binaries (BHXRBs) including comparisons of results from the analysis pipeline with published results and a systematic search for polarization degree and polarization angle variability in long IXPE observations. We critically discuss the performance of the pipeline and future avenues for improving it.
Read the full abstract

All authors B. Domine, B. Liu, H. Krawczynski, M. Kundu, Shravan Vengalil Menon, Fang Zhou

Data Analysis and Statistics · Cross-listed from hep-ph

Precision Tokenization for Multi-Modal Data in High Energy Physics

Jeffrey Krupa, Samuel Klein, Nilotpal Kakati et al. (+5 more)

Foundation models have recently gained significant attention in fields such as high energy physics (HEP). The tokenizers that convert detector data into discrete inputs for such models, however, are comparatively unexplored. This paper addresses this gap by learning discrete and compressed tokenized representations of HEP datasets. We demonstrate the approach on four modalities covering two different simulated detectors that vary in complexity. Treating tokenization as a rate-distortion problem, a systematic hyperparameter scan yields four design rules for tokenizing HEP data that hold across all set-based modalities studied. The cost of tokenization is evaluated on tasks including jet $p_T$ regression, particle flow reconstruction, and flavor tagging. We find that tokenization preserves much of the information needed for these tasks at compression factors of approximately 4-11$\times$ while keeping the jet $p_T$ resolution below 1%. We also present heptokens, an open-source library for tokenizing HEP data.
Read the full abstract

All authors Jeffrey Krupa, Samuel Klein, Nilotpal Kakati, Daniel Murnane, Baran Hashemi, Eilam Gross, Lukas Heinrich, Michael Kagan

Data Analysis and Statistics · Cross-listed from nucl-th

Final-state centrality in relativistic heavy-ion simulations from neural networks

Anibal Bezerra and Fernando G. Gardim

Experiments in relativistic heavy-ion collisions determine centrality from final-state quantities, usually the charged-particle multiplicity. Performing the same in hydrodynamic simulations requires millions of events, which is impractical for state-of-the-art hybrid models, and centrality is therefore commonly approximated by the initial entropy or energy. We show that neural networks trained on a few hundred hybrid events predict the event-by-event multiplicity and mean transverse momentum $[p_T]$ with relative errors of 1–2%, which makes it possible to generate millions of collisions at a small fraction of the cost of the full simulation. For Pb–Pb collisions at $2.76$~TeV and Au–Au collisions at $200$~GeV, the initial-state and final-state definitions select nearly the same events in 10%-wide classes, but up to 20% of the events change class when the classes are 1% wide. We then compute the fluctuations of $[p_T]$ and its Pearson correlations with a proxy of the elliptic flow, estimated from the initial eccentricity, and with multiplicity. The first correlation is insensitive to how centrality is defined, while the second changes substantially. The approach can be applied at every design point of Bayesian analyses at modest additional cost.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.geo-ph

Differences among preparatory processes preceding major earthquakes in Kamchatka, 2014-2025, including the MW 8.8 event

Stanislaw Lasocki, Beata Orlecka-Sikora and Mateusz Olszewski

Earthquake preparation involves progressive damage accumulation, strain localization, and changes in the organization of seismicity. We investigate preparatory processes preceding seven episodes of major earthquakes in the Kuril - Kamchatka subduction zone during 2014 - 2025, including the 29 July 2025 MW 8.8 Kamchatka megathrust earthquake. We track damage evolution using the seismic regularity coefficient dc and seismic strain dynamics coefficient SSD, defined for seismic series. Before 2022, each episode of strong seismicity was preceded by a pronounced dc minimum, commonly accompanied by an SSD minimum, followed by an increase in dc toward a maximum near the strong event or event group. After 2022, dc minima became shallower and were superimposed on a sustained upward trend. We interpret this transition as evidence that the metrics recorded two superimposed processes. The cycles before 2022 reflect complete strain-localization histories on southern or more distant structures that did not contribute to the segment ruptured in 2025. In contrast, the upward dc trend from 2022, with only shallow minima preceding Episodes 6 and 7, marks coupled behavior within the eventual main rupture segment, also influenced by stress transfer from southern source regions.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.soc-ph

A data-driven heuristic-ensemble method for behavioural inference in collective-risk dilemma experiments

Marc Sadurní, Martín F. Díaz, Josep Perelló et al. (+1 more)

The collective-risk dilemma has become a paradigm for studying cooperation in a wide variety of social experiments, initiatives that generate increasingly rich datasets describing individual actions over successive decision-making processes. Beyond the purely descriptive analysis of such empirical datasets, a methodological framework is required to interpret and reproduce the observed dynamics, a scheme that may be skewed by exogenous biases. Here, we introduce a data-driven, non-parametric method for simulating individual decisions together with a validation protocol for assessing the fidelity level of the outcomes they generate. Our methodological contribution is based on an empirical heuristic ensemble: agents are represented by a finite set of behavioural heuristics, probabilities conditioned on different observable features of the game state, endowed with a hard-voting arbitration that determines their per-round final decisions. This reproducible generative framework allows for behavioural inference in different collective-risk dilemma setups: Behavioural heuristics are seeded directly from empirical data assessing conditional probabilities, without a priori assumptions; a plurality vote with uniform random tie-breaking resolves between competing heuristics each round, adding no free parameters; validation follows a pattern-oriented approach, comparing synthetic and empirical macro-indicators against single-heuristic and uniform-random baselines; synthetic data generated allow exploration of scenarios or experimental setups that have not been empirically realised; as a worked example, we instantiate our methodological approach on the xAire citizen science collective-risk dilemma experiment run in Barcelona.
Read the full abstract

All authors Marc Sadurní, Martín F. Díaz, Josep Perelló, Miquel Montero

Data Analysis and Statistics · Cross-listed from stat.AP

Warning of infrastructure deformation anomaly through probabilistic analysis of InSAR observations

Hao Kuai, Na Hao, Pantelis Karamitopoulos et al. (+2 more)

Interferometric Synthetic Aperture Radar (InSAR) offers a low-cost, wide-area solution for near continuous infrastructure monitoring. However, translating InSAR observations into automated anomaly warnings remains challenging because displacements are relative, affected by uncertainty, and acquired along line of sight (LOS). We introduced a probabilistic warning framework that accounts for these factors. LOS displacements were modelled through a Gaussian distribution. A limit state function was defined to describe the displacement exceedance, and the time-dependent probability of exceedance was evaluated. Instead of a fixed threshold, warnings were identified through abrupt increases in exceedance probability. The framework was validated using three case studies of quay walls in Amsterdam, providing timely anomaly warnings and issued no warnings for the non-collapse case. Results demonstrate the potential of a probabilistic interpretation of InSAR time series for deformation anomaly warning.
Read the full abstract

All authors Hao Kuai, Na Hao, Pantelis Karamitopoulos, Alice Cicirello, Giorgia Giardina

Data Analysis and Statistics · Cross-listed from cs.AI

Local Diagnostics of Learned Event Representations for Anomaly Detection at the HL-LHC

Donatella Genovese, Philip Ploner, Simone Scardapane et al. (+2 more)

Anomaly detection, the identification of collision events that deviate from Standard Model expectations, is one of the most promising model-agnostic strategies for new-physics searches at the Large Hadron Collider. Searching for rare events in learned low-dimensional representations removes the need to commit to hand-crafted observables a priori, but sacrifices the immediate physical interpretation those observables provide. For anomaly detection to enter a discovery pipeline, flagged events must therefore be diagnosed locally, both to distinguish potential new physics from mismodeling or detector effects and to identify observables for targeted follow-up searches. We evaluate supervised and self-supervised neural embedding strategies on COLLIDE-2V, a 750-million-event dataset simulated under realistic High-Luminosity LHC conditions, using the reconstruction error of an autoencoder trained in the learned latent space as the anomaly score. Across the configurations and benchmark signals considered, supervised objectives, which exploit simulation process labels during training, generally outperform their self-supervised counterparts and, in their best configurations, improve on an autoencoder trained directly on kinematic features, raising the signal efficiency at 10% background acceptance on all held-out processes. We further introduce a local diagnostic pipeline that fits a Gaussian Mixture Model to the SM latent space, profiles each component through high-level observables without process labels, localizes the flagged events, and ranks observables by their local discriminating power. The ranking recovers the defining kinematics of the held-out processes, while correlating the anomaly score with the same observables quantifies how closely the score tracks them, revealing cases where the most discriminating observable is not the one the score follows most closely.
Read the full abstract

All authors Donatella Genovese, Philip Ploner, Simone Scardapane, Stefano Giagu, Thea Klaeboe Aarrestad

Data Analysis and Statistics · Cross-listed from physics.geo-ph

Is Earthquake Replenishment Sufficient for Unbiased b-Value Estimation?

Herbian Hokky Prabowo and Giuseppe Petrillo

Short-term aftershock incompleteness biases Gutenberg-Richter b-value estimates because many small earthquakes remain undetected immediately after large events. Two strategies address this problem. Earthquake replenishment reconstructs missing events to restore a more complete catalog, whereas robust positive-difference estimators infer the b-value directly from incomplete observations. Whether these approaches provide equivalent information on the magnitude distribution remains open. We address this issue using synthetic aftershock sequences generated with the Epidemic-Type Aftershock Sequence (ETAS) model and degraded through a time- and magnitude-dependent incompleteness model. Incomplete catalogs are replenished using AutoBETA, and b is estimated using the standard maximum-likelihood, b-more-positive, and b-more-incomplete estimators. Replenishment reconstructs the overall structure of the missing-event population and substantially improves conventional maximum-likelihood estimates, although small residual biases remain. By contrast, both positive-difference estimators recover the imposed b-value directly from incomplete synthetic catalogs. Applications to the Ridgecrest, Landers, and Hector Mine sequences show that b-more-incomplete is less sensitive to replenishment than b-more-positive. Ten independent replenishment seeds indicate that seedto-seed variability is small relative to differences among estimators. Replenishment and robust b-value estimation therefore address different aspects of catalog incompleteness: replenishment provides a more complete aftershock representation, whereas robust estimators are preferable when accurate b-value estimation is the primary objective.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.geo-ph

Is the Number of Foreshocks Underestimated? The Role of Declustering Symmetry

Herbian Hokky Prabowo, Eugenio Lippiello, Sebastian Hainzl et al. (+1 more)

The physical origin of foreshocks remains one of the major unresolved problems in statistical seismology. In bottom-up (BU) triggering models, foreshocks are interpreted as ordinary earthquakes that stochastically trigger a larger event through cascading interactions, as in the Epidemic-Type Aftershock Sequence (ETAS) model. In contrast, top-down (TD) models interpret foreshocks as manifestations of a preparatory process preceding the mainshock. This ambiguity has important implications not only for earthquake forecasting, but also for declustering itself, since the inferred number of foreshocks may depend on the assumed physical mechanism governing seismicity. Here we investigate the detectability of foreshocks using synthetic catalogues representing two limiting physical scenarios: purely bottom-up triggering and mixed preparatory-triggering dynamics. These cases are implemented through the ETAS and ETAFS models, respectively. We compare the classical nearest-neighbor (NN) declustering metric with a reverse nearest-neighbor metric (rNN) introduced here. While the standard NN metric scales the correlation metric to the magnitude of the earlier event, rNN scales it with the magnitude of the later event. We show that rNN is mathematically equivalent to applying the standard nearest-neighbor metric to the time-reversed catalog, establishing a direct symmetry between the two approaches. Our simulations reveal [...]
Read the full abstract

All authors Herbian Hokky Prabowo, Eugenio Lippiello, Sebastian Hainzl, Giuseppe Petrillo

Thursday 8 October

Data Analysis and Statistics · Cross-listed from q-bio.NC

Energy, Information, and the Limits of Neuropsychiatric Classification

Farbod Khodadadi, Mina Kheirkhah, Bita Shariatpanahi et al. (+15 more)

Importance: The search for reliable neuroimaging biomarkers of mental disorders has remained elusive despite decades of research. Why clinically consequential disorders yield weak neuroimaging separability remains unresolved. Drawing on statistical thermodynamics and the metabolic costs of neural coding, we propose that psychiatric disorders produce constrained deformations of a largely shared repertoire of brain activity patterns, limiting the class-specific information available for classification. Observations: We develop a framework linking physical resource constraints, landscape deformation, distributional separation, and attainable classification, and examine its components using human neuroimaging, synthetic data, and an exactly specified physical model. In resting-state fMRI from 1952 FOR2107 participants (823 with major depressive disorder; 1129 healthy controls), held-out energy-landscape estimates and nonlinear classifiers agreed across 13 comparisons (r = 0.996). Similar results emerged at both FOR2107 centers and in 900 SRPBS participants spanning multiple psychiatric diagnoses. In synthetic data, classification improved with distributional separation and approached the optimum. In an equilibrium three-well model, statistical energy equaled physical potential energy scaled by inverse temperature. Modest deformation redistributed state occupancy while optimal snapshot classification remained weak. A literature map found insufficiently comparable evidence to test the human metabolic constraint. Conclusions: Clinically consequential differences may arise within a largely shared neural repertoire. The thermodynamic framework connects physical deformation to classification under specified assumptions; its metabolic implementation in humans remains untested. Testing this mechanism requires joint measurements of brain activity, bodily physiology, and metabolism.
Read the full abstract

All authors Farbod Khodadadi, Mina Kheirkhah, Bita Shariatpanahi, Sarah Alizadeh, Madleen Stenger, Amaia Zurinaga Gutierrez, Shakiba Bakhshian, José C. García Alanis, Nils R. Winter, FOR2107 Consortium, Frank Bremmer, Stefan G. Hofmann, Udo Dannlowski, Tilo Kircher, Sharmili Edwin Thanarajah, Igor Nenadić, Tim Hahn, Hamidreza Jamalabadi

Data Analysis and Statistics · Cross-listed from cond-mat.mtrl-sci

Generation, Accumulation, and Utilization of Data in the Automatically Controlled End-Station of a Soft X-ray Beamline

Yasunobu Ando, Masashige Miyamamoto, Keita Hiromori et al. (+4 more)

AI is rapidly transforming materials exploration, yet reliable AI-assisted research depends on experimental data that retains their context, provenance, and machine-readable structure. High-throughput synchrotron measurements therefore require more than automated data acquisition: data generation, accumulation, and utilization must be connected within a continuous workflow. Here, we present a data-centric research framework that integrates the PIONEER system, the OMNES, and ML analysis. The PIONEER system automates sample handling, vacuum transfer, and scanning XAS at the BL08U soft X-ray beamline of NanoTerasu, systematically generating large volumes of spatially resolved spectral data. Measurement and analysis files are stored in the cloud-based ARIM-mdx data system, while OMNES manages metadata describing samples, preparation procedures, measurement conditions, instrument settings, data locations, and analysis histories. Tabular and JavaScript Object Notation formats are used according to the metadata structure, with numerical values stored separately from units to facilitate machine processing. OMNES organizes these records based on causal relationships among experimental and analytical processes, linking samples, measurements, raw data, processed data, and analysis results through a relational data model and visualizing their relationships as a graph. This causality-based organization preserves experimental and analytical provenance and clarifies how each result was derived from the preceding processes and data. The accumulated spectra can subsequently be processed using ML methods, including dimensionality reduction and materials classification, and the resulting data can be registered back into OMNES. Previously reported ML analysis of BN XAS spectra is presented as an example of the analytical workflow that can be applied to data acquired at the beamline.
Read the full abstract

All authors Yasunobu Ando, Masashige Miyamamoto, Keita Hiromori, Takeru Nakashima, Masafumi Horio, Masato Kotsugi, Iwao Matsuda

Data Analysis and Statistics · Cross-listed from cond-mat.mtrl-sci

Data-Driven Variable-Exponent Analysis for Photoemission Yield Spectroscopy: An Autonomous Self-Diagnosing Framework Based on Integrated Residual Metrics

Shinjiro Yagyu, Takahiro Nagata and Yoshiyuki Nakajima

Photoemission yield spectroscopy (PYS) is widely used for evaluating the electronic states of materials. As automated materials discovery advances, unsupervised extraction of physical information from ambient-air PYS data becomes important. Conventional fixed-exponent analyses and logarithmic transformations suffer from heteroscedasticity, which destabilizes estimation in low-signal regions. To address this, we propose a data-driven analysis framework based on the 1/n-Scan method, which operates directly in the original signal space, removing the geometric bias inherent in log-transform approaches. We further integrate a self-diagnostic quality-evaluation system that quantifies estimation uncertainty with Akaike weights, together with independent residual metrics–the normalized mean absolute error (NMAE), the RMSE-to-MAE ratio (RMR), the Durbin-Watson (DW) statistic, and a macroscopic metric ($\Delta R^2$)–that distinguish hardware-related data degradation from a physical-model mismatch. Applying the framework to differently doped Si and a polycrystalline Au reference in air, we demonstrate autonomous detection, without assumptions on the emission mechanism, of the breakdown of the single-component approximation in heavily doped p-type Si, arising from the overlap of two emission components with different thresholds, as a statistical anomaly–a decrease in DW below its critical value with an auxiliary increase in $\Delta R^2$, despite sound NMAE and RMR–independently confirmed by a two-component fit ($\Delta$AIC $\approx$ 57, DW recovering from 0.8 to 2.0). For heavily doped n-type Si, the gradual surface evolution was classified as a change within the single-component description. This framework provides a robust, self-diagnosing analysis engine for closed-loop autonomous materials exploration.
Read the full abstract

Wednesday 7 October

Data Analysis and Statistics · Cross-listed from cs.SI

Some Twitch chats exhibit multifractal characteristics resembling stream-of-consciousness narrative

Marcin Ból, Stanisław Drożdż, Robert Kluszczyński et al. (+2 more)

Live-streaming platforms generate high-frequency records of collective human activity in which many loosely coordinated users react in real time to shared external stimuli and to one another. We investigate whether the temporal organization of Twitch chat messages exhibits multifractal signatures characteristic of complex dynamical systems. Selected high-activity chats were converted into time series by measuring the length of each consecutive message in words, with platform emotes treated as individual tokens. The resulting signals display heterogeneous fluctuations, bursty organization, and heavy-tailed distributions that, in several cases, are better captured by q-Weibull forms than by conventional light-tailed models. Autocorrelation analysis reveals persistent temporal dependence over broad ranges of message lags, indicating that the variability is not produced by independent message generation alone. MFDFA shows that some chats possess well-developed multifractal spectra and generalized Hurst exponents strongly dependent on moment order, whereas others exhibit weak scaling, approximately monofractal behavior, or no well-defined scaling regime. Surrogate tests indicate that temporal correlations are essential for the observed multifractality, while heavy tails enhance its apparent strength. Under particular dynamical conditions, Twitch chat may therefore form an emergent collective signal whose fragmented, reactive, associative, and multiscale organization is structurally reminiscent of stream-of-consciousness narrative. This expression is used strictly as a structural and dynamical analogy, not as a claim of a unified collective mind or of semantic, cognitive, or phenomenological equivalence with literary stream-of-consciousness writing. The results position live-stream chat dynamics within the broader physics of complex, intermittent, human-generated signals.
Read the full abstract

All authors Marcin Ból, Stanisław Drożdż, Robert Kluszczyński, Jarosław Kwapień, Marcin Wątorek

Data Analysis and Statistics · Cross-listed from econ.EM

From a Hierarchy of Stochastic Differential Equations to a Hierarchy of Generalized Beta Distributions

Siqi Shao and R. A. Serota

We introduce a mean-reverting stochastic differential equation with a three-component stochastic term and show that it generates a hierarchy of steady-state (stationary) distributions. At the top level, the hierarchy is described by a modified-Beta distribution, while one- and two-parameter reductions produce compact-support, power-law-tailed, and exponential-type limiting families within a single stochastic framework. We then construct two generalized extensions of this hierarchy. In the first, the power transformation is applied directly at the level of the stochastic differential equation; in the second, the same transformation is applied only after the stationary modified-Beta hierarchy has been obtained. While these two procedures agree on important lower branches and limiting cases they generally differ at the top level. The generalized hierarchy is therefore not unique: nonlinear transformation and stationary-state reduction do not commute. For both routes, we derive the probability density and cumulative distribution functions, express their parameters in terms of the underlying stochastic dynamics, clarify the relations among their limiting cases, and compare the resulting families with the traditional Generalized Beta framework.
Read the full abstract

Data Analysis and Statistics · Cross-listed from cond-mat.other

Progress and Prospect of AI in ARPES Workflow

Sandy Adhitia Ekahana, Aalok Tiwari, Pratik Saud et al. (+4 more)

Artificial intelligence (AI) is becoming an increasingly useful tool across the experimental sciences, including angle-resolved photoemission spectroscopy (ARPES), which routinely produces large, multidimensional datasets of electronic structure. Recent advances in AI and machine learning (ML) have opened new opportunities across the entire ARPES workflow, from automated sample preparation and real-time data acquisition to post-experiment data analysis and comparison with theoretical calculations. Despite this progress, a comprehensive review of ML applications, their capabilities, and reliability across the different stages of ARPES workflow is still lacking. In this review, we first introduce ML methods that are most relevant to experimentalists working in condensed matter physics and materials science. We then follow the ARPES workflow, reviewing existing ML applications at each step and discussing their advantages, limitations and potential for future development. We also examine the current ARPES data landscape, where several open databases are available but remain relatively small and fragmented compared with large, shared datasets such as ImageNet. Given these limitations, we suggest that the community focus on sharing pretrained models that can be further trained, adapted to specific tasks, and redistributed, while working toward a larger and standardized open ARPES dataset repository. Finally, we discuss our perspectives on the future of AI within the ARPES workflow using a six-level framework of laboratory automation, highlighting the opportunities and challenges in moving toward a fully autonomous, self-driving ARPES laboratory.
Read the full abstract

All authors Sandy Adhitia Ekahana, Aalok Tiwari, Pratik Saud, Aaron Bostwick, Chris Jozwiak, Eli Rotenberg, Jyoti Katoch

Data Analysis and Statistics · Cross-listed from cs.LG

Matching of signal, noise and hardware timescales for filtering and forecasting of correlated noise signals

Joshua Donald, Alex Gabbitas, Arthur G. T. Coveney et al. (+2 more)

Physical reservoir computing exploits the nonlinear dynamics of physical systems to process time-dependent data with greater energy efficiency than conventional machine learning approaches. However, physical reservoirs have fixed intrinsic response timescales, whereas real-world signals combine deterministic and stochastic components across multiple timescales. Here we show, using a nanoporous niobium oxide reservoir, synthetic noisy signals and cryptocurrency-price volatility, that the relationship among noise correlation time, reservoir memory and forecast horizon determines whether correlated noise is filtered or predicted. Noise varying faster than the relevant reservoir memory and forecast horizon is averaged by the reservoir, whereas the temporal structure of slower-varying noise is sufficient for algorithmic forecasting. We introduce the reservoir memory horizon and forecasting regime index to distinguish these operating regimes. These contributions demonstrate that timescale matching can guide the encoding of input time series and development of physical reservoir architectures that filter, analyse and predict stochastic signal components across distinct temporal scales.
Read the full abstract

All authors Joshua Donald, Alex Gabbitas, Arthur G. T. Coveney, Sergey Savel'ev, Pavel Borisov

Data Analysis and Statistics · Cross-listed from stat.AP

Statistical Oceanography of Profiling Floats and Surface Drifters

Mikael Kuusela, Sofia C. Olhede and Adam M. Sykulski

The Global Ocean Observing System is key to understanding oceanic variability and climate change. The ocean is vast and often sparsely and irregularly sampled in time and space by various instruments and systems. Statistical models, especially spatio-temporal ones, are useful for enabling inferences, forecasts and decisions from sparse oceanic observations. This article focuses on the statistical treatment of two types of in situ observations in the Global Ocean Observing System: from profiling floats and surface drifters, focusing on the Argo and Global Drifter Programs. We describe the spatio-temporal models that have been developed in recent years for these data, to give a picture of the statistical challenges faced in modern physical oceanography. We will discuss in detail two types of reference frames when modeling such data: Eulerian and Lagrangian, the former of which is more appropriate for profiling floats and the latter for surface drifters.
Read the full abstract

Data Analysis and Statistics · Cross-listed from cond-mat.soft

Resolving the Effects of the Mixing Process on Hierarchical Structures in Polymer Nanocomposites by Bayesian Ultra-Small-Angle X-ray Scattering Analysis

Yui Hayashi, Kazuki Mita, Shigeo Kuwamoto et al. (+1 more)

Bayesian inference was combined with Unified Fit to resolve the effects of the mixing history on polymer nanocomposites whose ultra-small-angle X-ray scattering profiles are nearly indistinguishable by conventional analysis. Bayesian Unified Fit was applied to comparing the aggregate size, internal mass-fractal structure, and surface roughness for three carbon black–filled ethylene propylene diene copolymer rubber samples prepared by different mixing processes. Whereas conventional Unified Fit reported essentially identical carbon-black dispersions, the radius of gyration, mass fractal dimension, and surface fractal dimension for the three samples all followed the same ordering, which coincides with the reported ordering of the tensile strength and elongation at break and suggests that the inferred aggregate hierarchy is physically meaningful for the macroscopic mechanical response. These results identify the aggregate size as a candidate structural descriptor of rubber reinforcement and provide an uncertainty-aware basis for examining processing–structure–property relationships in polymer nanocomposites.
Read the full abstract

All authors Yui Hayashi, Kazuki Mita, Shigeo Kuwamoto, Masato Okada

Tuesday 6 October

Data Analysis and Statistics · Cross-listed from physics.soc-ph

Causal asymmetry suggests productivity underlies scientific collaboration

Diego A. Frota, Cesar I. N. Sampaio Filho, Vitor H. Ribeiro et al. (+5 more)

Scientific productivity and collaboration are closely related, yet the causal direction between these two dimensions of academic careers remains unclear. Here we analyze the careers of 26,876 Brazilian researchers holding a nationally prestigious fellowship across multiple disciplines to test whether productivity shapes collaboration or the reverse. Focusing on total publications and distinct coauthors, we assess their directional relationship using a causal discovery framework based on asymmetries in conditional and distributional structure to determine the more plausible generative direction. Collaboration conditioned on productivity exhibits a highly regular mean-variance relation consistent with a geometric model, whereas the reverse conditioning yields a much less coherent pattern. Consistently, publication counts follow a lognormal distribution, whereas the coauthor distribution is reproduced by a geometric-lognormal mixture where productivity determines the expected number of collaborators. Surrogate tests indicate that the data are considerably more compatible with productivity shaping collaboration than with the reverse. The two quantities also accumulate asymmetrically: publication inequality rises steadily with career age, whereas coauthor inequality does not, indicating that advantage compounds more in output than in network breadth. Area-resolved analyses reveal a more nuanced picture, and although the direction where collaboration shapes productivity is never favored, the two directions are most clearly distinguished in areas more organized around laboratory and research-group models, where productivity inequality also grows most clearly. Among this cohort of prominent scientists, larger collaboration networks are thus more likely to reflect cumulative advantage in productivity than to independently generate greater output.
Read the full abstract

All authors Diego A. Frota, Cesar I. N. Sampaio Filho, Vitor H. Ribeiro, Germano F. C. Luz, Humberto A. Carmona, Matjaz Perc, Haroldo V. Ribeiro, Jose S. Andrade Jr

Data Analysis and Statistics

Data geometry preserves prediction but reshapes explanation

Nicola Amoroso, Mario Caruso, Marianna La Rocca et al. (+8 more)

Across scientific domains, empirical data often give rise to positive semidefinite matrices that encode similarities, couplings or interactions and induce natural geometries. We investigate whether changing the geometry used to compare the same empirical representations preserves subject-level performance and the local structures from which explanations are derived, a question we term the prediction explanation invariance problem. We address this problem across morphometric MRI, fMRI and EEG data by comparing Frobenius and trace geometries applied to the same empirical matrices. Predictive performance was broadly preserved across geometries, but comparable performance did not imply agreement in subject-level decision scores or predicted labels. Predictive similarity also masked geometry-dependent differences in local neighbourhoods, perturbation sensitivities and explanatory rankings. Dimensionality reduction made the two subject-space geometries progressively more concordant, while classification performance and decision-level agreement declined. These results identify data geometry as a hidden degree of freedom in explainability: similar performance does not guarantee invariant decisions or explanations.
Read the full abstract

All authors Nicola Amoroso, Mario Caruso, Marianna La Rocca, Loredana Bellantuono, Tommaso Maggipinto, Michele Morelli, Sabina Tangaro, Marco Tatullo, Roberto Bellotti, Ester Pantaleo, Alfonso Monaco

Monday 5 October

Data Analysis and Statistics · Cross-listed from physics.flu-dyn

(Why) Jets quantise trapped acoustic modes

Brandon Yeung, Michael N. Stavropoulos, Vincent Jaunet et al. (+2 more)

High-subsonic turbulent jets trap acoustic modes in their potential core. The modes lie in narrow frequency bands that are $O(0.1)$-wide in Strouhal number, $St$. By studying a Mach 0.9, Reynolds number $10^6$ jet, this work confirms an earlier hypothesis: each band is made up of $O(10)$ discrete frequencies, $\Delta St \sim O(0.01)$ apart. To resolve them, we collect $O(100\,000)$ snapshots from dedicated schlieren experiments and large-eddy simulations (LES), and apply spectral proper orthogonal decomposition (SPOD). Our SPOD algorithm uses sine tapers and enforces the jet's rotational and reflectional symmetries. For both the schlieren and the LES, the trapped SPOD modes are bounded in the $St$-$x$ and $St$-$k$ planes, where $x$ and $k$ are respectively the axial coordinate and wavenumber. The bounds quantise the modes in $St$, $x$, and $k$, such that the $n_x$th mode in $St$ has $n_x$ antinodes along $x$ and $k$. We then predict trapped modes with local, global, modal, and non-modal stability theories. We find evidence of a competition between modal and non-modal stability. From only the theoretical bounds, we build a simple in-phase standing-wave resonance model that predicts the quantised global-mode frequencies.
Read the full abstract

All authors Brandon Yeung, Michael N. Stavropoulos, Vincent Jaunet, Peter Jordan, Oliver T. Schmidt

Data Analysis and Statistics · Cross-listed from hep-ex

How to scale your HEP ML models: A recipe for robust architecture comparisons at scale

Matthias Vigl, Nikita Pond, Jackson Barr et al. (+5 more)

Much of the recent progress in machine learning domains such as language models has come from scaling laws that predict performance as a function of training effort. In high-energy physics (HEP) similar behavior has now been observed. To aid further study, we present a systematic procedure to derive robust scaling laws and compare design choices on the relevant budget axes for HEP tasks. We first validate the full scaling trajectory on toy problems and then apply the procedure to multi-task transformers on the ~11 billion-jet ATLAS JetSet2 dataset, in both the compute- and data-constrained regimes. For the latter, we predict, to the best of our knowledge for the first time, the jointly optimal model size, training horizon, learning rate and batch size under early stopping. At compute-optimal scaling, we recover a near-equal $\sqrt{C}$ dependence of model and dataset size, and find that auxiliary objectives lower the primary jet-classification loss at equal compute budget. Expanding the inputs toward lower-level data systematically lowers the loss while leaving the scaling exponent nearly unchanged. The onset of the power-law regime is itself set by scale: below a threshold in dataset size the loss carries little information about high-compute scaling, underscoring the value of large, high-quality full-simulation datasets as a foundation for scaling studies and the development of foundation models in HEP.
Read the full abstract

All authors Matthias Vigl, Nikita Pond, Jackson Barr, Alexander Froch, Dan Guest, Nicole Hartman, Michael Kagan, Lukas Heinrich

Saturday 3 October

Data Analysis and Statistics · Cross-listed from gr-qc

Covariance scaling and a dual-mode torque diagnostic in measurements of the gravitational constant

Vineet Kumar

Laboratory determinations of the Newtonian gravitational constant $G$ disagree by more than their quoted uncertainties allow. We examine the sixteen determinations comprising the CODATA-2018 archive using a proportional covariance-scale model that retains the full reported covariance matrix. Because the 2026 NIST torsion-balance redetermination shares apparatus lineage with the BIPM Mark II balance, our primary historical fit excludes the BIPM-14 determination, giving $\hat{\lambda} = 2.967$ for the remaining fifteen measurements. Evaluated against this historical model under the declared uniform covariance-scaling convention, the correlated four-configuration NIST result is highly unlikely under the archive's reported, unscaled covariance but is statistically unremarkable once the historically inferred extra dispersion is applied ($Q_\lambda = 4.840$; finite-sample $F$ tail $p = 0.382$); a mean-only scaling sensitivity leaves the vector discrepant. Retaining the complete archive also gives a comparable result. We further examine four matched free-minus-servo torque differences reported by NIST. They are consistent with a single common absolute torque offset across the four tested configurations, $\widehat{\Delta N} = 1.918 \pm 0.215$ pN m, and poorly described by an equally simple fractional-scale alternative under the stated independent-Type-A treatment, with a chi-square contrast of $\Delta\chi^{2} = 11.041$ between the two torque models. The paper provides a covariance audit of the historical archive, a lineage-qualified consistency check, and an experimentally testable diagnostic for dual-mode torsion balances, without proposing a revised value of $G$.
Read the full abstract

Data Analysis and Statistics · Cross-listed from cond-mat.stat-mech

MUTACO: Simulator-Efficient Response Matching in Stochastic Nonlinear Systems via Compatibility-Aware Sequential Design

Ege Karadeniz, İlter Onat Korkmaz, Umut Can Turhan et al. (+2 more)

Many physical inverse problems are mediated by expensive stochastic simulators for which response gradients, adjoint sensitivities, or a differentiable simulator interface are unavailable. We introduce Multiple Target-based Confident Acquisition (MUTACO), an uncertainty-aware sequential-design method for response matching that requires only noisy forward evaluations and learns the multicomponent response together with its predictive uncertainty. We develop and evaluate MUTACO for a disordered network of stochastic phase oscillators. The inverse task is to infer statistical properties of the local potential and interactions from macroscopic rotation and synchronization curves under multiple driving conditions. Rather than requiring unique recovery of these statistical properties, MUTACO searches for parameter configurations whose predicted rotation and synchronization responses are compatible with the target response while accounting for predictive uncertainty. This response-centered formulation is appropriate when experimentally accessible observables characterize ensemble statistics rather than a particular microscopic realization. Under strictly limited simulation budgets, MUTACO achieves the lowest held-out response error among the tested adaptive-search, space-filling, and simulation-based inference methods, while also yielding lower parameter-recovery error than the compared methods. These results demonstrate that response-targeted sequential design is a simulator-efficient approach to inverse modeling from finite noisy forward evaluations and suggest broader applicability to stochastic physical systems with correlated multicomponent responses and weakly constrained parameter directions.
Read the full abstract

All authors Ege Karadeniz, İlter Onat Korkmaz, Umut Can Turhan, Cem Tekin, Ayd{\i}n Cem Keser

Friday 2 October

Data Analysis and Statistics · Cross-listed from stat.ML

When Is Accuracy Evidence? A Unified Theory of Generalisation, Validation, and Information Fusion

JM Gorriz

K-fold cross-validation (CV) is widely used as evidence of out-of-sample performance, although folds are neither independent experiments nor equally informative under heterogeneous data. Cross Upper-Bound Validation (CUBV) replaces point-wise CV accuracy by conservative upper bounds on true risk. Here we generalise CUBV through a single exponential framework in which the moment-generating function of the generalisation gap is controlled by a cumulant envelope gamma(lambda). This yields a family of risk bounds covering Hoeffding-, Bernstein-, dependency-aware, PAC-Bayesian, and heterogeneous source-fusion settings. For K-fold CV, dependence between fold-wise gaps is modelled through a joint sub-Gaussian proxy matrix. Under equicorrelation, this gives an effective number of folds, Keff = K/[1+(K-1)rho], showing that increasing K does not necessarily increase statistical evidence when folds are strongly dependent. The framework is also extended to posterior distributions over predictors and weighted multi-source fusion, where weights are selected by minimising an upper bound on future risk rather than empirical error alone. Experiments with trained linear classifiers on heterogeneous multimodal Gaussian mixtures compare K-fold CV with full-sample resubstitution plus risk correction. Bounds are evaluated by coverage and tightness. In low-dimensional small-sample settings, K-fold partitioning can increase uncertainty because individual folds under-represent minority modes, while corrected resubstitution can remain valid and tighter; this effect disappears as sample size increases. Overall, gamma-CUBV separates observed performance, uncertainty, dependence, model complexity, and confidence into explicit terms, providing a unified route from CV scores to risk statements and a principled validation criterion for heterogeneous small-sample applications such as neuroimaging.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.chem-ph

Explainable Molecular Structure Inference from GC–MS with Diffusion Models and LLM Reranking

Changlin Liu, Tianyu Yi, Chengchun Liu et al. (+2 more)

GC–EI–MS is an important technique for analyzing volatile and semivolatile compounds in complex samples. However, conventional methods rely heavily on reference spectral library matching, limiting their ability to identify compounds absent from these libraries and to infer complete molecular structures directly from fragmentation information. Here, we present DiffGCMS, a spectrum-conditioned discrete graph diffusion model for de novo structure elucidation from GC–EI–MS, and further develop a framework that integrates DiffGCMS with second-stage reasoning by a large language model (LLM). In the first stage, DiffGCMS generates candidate molecular structures from input spectra; in the second stage, the LLM uses mass spectral information to validate, repair, and rerank the candidates and provides interpretable analysis of fragment-ion peaks. This framework can generate plausible molecular structures for compounds absent from reference spectral libraries and provide traceable evidence supporting its decisions. On a test set comprising 13,696 spectra from NIST 20, the generative model achieved Acc@1 and Acc@10 of 6.01% and 15.76%, respectively. On the test subset containing molecules with no more than 10 heavy atoms, LLM-assisted molecular graph repair and reranking increased Acc@1 from 21.28% to 21.95%, Acc@10 from 46.91% to 47.99%, and candidate validity from 91.04% to 100%. These results demonstrate that spectrum-aware postprocessing can correct errors produced by the generative model while providing auditable and traceable explanations for the final ranking.
Read the full abstract

All authors Changlin Liu, Tianyu Yi, Chengchun Liu, Boxuan Zhao, Fanyang Mo