Peer Revenue Publish or profit

Papers

Fresh from arXiv, typeset properly. Highlighted words trade on the market.

Physics Physics Data Analysis and Statistics

physics.data-an Watch Data Analysis and Statistics (sign in to use)

Saturday 12 September

Data Analysis and Statistics · Cross-listed from physics.flu-dyn

Task-preserving neural segmentation of overlapping shocks and vortex cores in compressible flows

Shock fronts and vortex cores often coexist and overlap in compressible flows, where walls, wakes and shear layers also produce strong gradients. Refining a network to detect one structure can therefore degrade its prediction of the other without revealing the loss in the optimized score. We present a task-preserving formulation for simultaneous shock and vortex-core segmentation. The experiments use two solvers and include supersonic diamond-airfoil flows at several incidences and Reynolds numbers, together with circular- and elliptical-cylinder flows. A shared primitive encoder feeds separate shock and vortex decoders with independent sigmoid outputs, allowing the two classes to overlap. Adaptation is confined to the relevant branch: zero-initialized adapters supply rotational diagnostics only to the vortex decoder, while corrected shock supervision updates only the shock decoder. All dependencies of the protected output remain fixed, and bitwise equality is verified on every evaluation field. Compression and conservation-jump (Rankine–Hugoniot) signatures provide weak shock supervision; rotation and topology provide vortex candidates. Analytical oblique-shock rays, Billig's bow-shock correlation and an isentropic vortex supply references independent of these labels. Under the same corrected-target budget, the restricted model and a capacity-matched shared-decoder U-Net obtain comparable shock agreement. Their airfoil vortex-core Dice overlap scores, however, are 0.83 and 0.08, respectively. A soft retention penalty recovers most of the U-Net's lost core agreement. The frozen shock-adapted models locate the tested analytical oblique-shock rays within 0.003 chord. An additional branch identifies expanding-flow regions, which are distinguished from centred Prandtl–Meyer expansion fans through comparison with ideal shock–expansion theory.
Read the full abstract

Data Analysis and Statistics

Finite Asimov Sample Construction in Unbinned Neural Simulation-Based Inference

Expected sensitivity calculations in frequentist analysis using neural simulation-based inference often require large simulated samples to approximate an unbinned Asimov dataset. We construct a finite weighted reference sample, for analyses based on learned density ratios, for which the generating parameters globally maximize the weighted likelihood. A toy example with five-dimensional observable space based loosely on high-energy physics models shows that an Asimov dataset consisting of as few as 256 weighted reference events closely reproduce the expected test statistic scans obtained with two million simulated reference events. The resulting test statistic distributions are also shown to agree with independent simulator-based pseudo-experiments. We note that this agreement with the simulator depends on the accuracy of the learned ratios and must be validated explicitly.
Read the full abstract

Data Analysis and Statistics · Cross-listed from cs.IT

Finite-Mask Gaussian-Process Reconstruction on a Periodic Sensor Ring: Mask-Geometry Dependence, Fourier-Mode Coupling, and Posterior Trace

We analyze how an arbitrary observation mask on a finite periodic ring of equally spaced sensors affects the posterior covariance, Fourier-mode coupling, and normalized posterior trace in Gaussian-process reconstruction. Under a rotationally stationary prior and homogeneous independent measurement noise, complete observation gives independent scalar posterior formulas for the Fourier modes. For an arbitrary mask, however, the matrix $Q_M=F D_MF^\ast$ is generally non-diagonal; its off-diagonal entries are finite Fourier components of the realized mask and couple modes in the posterior precision. Consequently, masks with the same unavailable-channel fraction can have different normalized posterior traces because their geometries differ. A dimensionless 64-channel synthetic benchmark illustrates this finite-matrix effect. A circumferential array of equally spaced wall-mounted microphones at a fixed axial station of a circular fan or compressor duct provides one concrete mechanical-engineering interpretation: failed, saturated, corrupted, or dropped-out channels form the observation mask. The analysis is a finite-dimensional reference calculation under the stated rotational-stationarity and common-noise assumptions, not a performance claim for a nonuniform or unequally instrumented operating duct.
Read the full abstract

Friday 11 September

Data Analysis and Statistics · Cross-listed from eess.IV

PiMiX 2.02: Toward AI-Driven Data Fusion in Radiographic Imaging and Tomography

PiMiX (Physics-informed Meta-instrument for eXperiments) was introduced for multi-instrument, multi-experiment, and simulation-experiment data fusion in radiographic imaging and tomography (RadIT). Here we present PiMiX 2.02 as an evolving AI-enhanced cyber-physical meta-instrument integrating imaging sensors, near-sensor computing, data fusion, physics-informed inference, and human-supervised AI workflows across X-ray, neutron, and other modalities. Demonstrated capabilities include multimodal CMOS radiation imaging, simulation-assisted sub-pixel neutron localization, and edge-deployed optical-neural-network (ONN) inference; GPU and ONN implementations achieved greater than 96% precision for neutron-event detection with sub-micron localization. A further advance is human-in-the-loop agentic-AI co-analysis of X-ray and neutron images from inertial-confinement-fusion experiments. Beyond conventional preprocessing, the workflow generates competing feature hypotheses, ranks contours using physics-informed evidence, estimates confidence, and presents alternatives for human review. The same architecture adapts to different physics: X-ray analysis emphasizes dark, nonuniform ring structures using deformable closed paths and multi-scale evidence, whereas neutron analysis targets bright emission envelopes using fractional-emission levels, intensity gradients, and cross-filter persistence. We also highlight automated comparison of an as-designed stereolithography model with an X-ray CT reconstruction of an additively manufactured metal lattice. Together, these examples show progression from AI assistance in specific processing tasks to multi-task, multi-domain scientific co-analysis. PiMiX 2.0 further provides a pathway toward PRISM, a RadIT scientific foundation model, and tighter integration of diagnostics, digital representations, inference, and experimental control.
Read the full abstract

All authors Zhehui Wang, Shanny Lin, Nicholas Amano, Ramya Gurunathan, Katie Liu, Nathan E. Peterson, Michelle A. Espy, Adam Thompson, Amy J. Clarke, Ray T. Chen

Data Analysis and Statistics · Cross-listed from cs.AI

Interpreting the predictions of neural network classification based on a Taylor Coefficient Analysis (TCA)

We introduce a rigid and comprehensive taxonomy and paradigm for characterizing the influence of the input feature space $X$ on the predictions $\hat{y}$ of a neural network (NN) used for event classification, based on a Taylor expansion of $\hat{y}$ in $X$. The complete process of introspection we refer to as Taylor Coefficient Analysis (TCA). Based on two simplistic example tasks, which can be easily understood and bencmarked, we illustrate the power of the TCA when it comes to revealing, what properties of $X$ have led to what value of $\hat{y}$, of a given NN model, building up intuition for the method. A more complex application is meant to represent $X$ of a typical classification task at a CERN LHC experiment. Based on this application, we play through the different levels of introspection that the TCA offers and discuss a number of practical aspects for a TCA application typical for the analysis of CERN LHC data. We conclude with a study to support the assumption that those properties of $X$ most relevant for tasks of the complexity typical for a CERN LHC experiment, are usually caught by a TCA up to the second order.
Read the full abstract

All authors Markus Klute, Artur Monsch, Lars Sowa, Roger Wolf

Thursday 10 September

Data Analysis and Statistics · Cross-listed from physics.acc-ph

Physics-Informed Drift Diagnosis for Laser-Plasma Accelerator Operations

Laser-plasma accelerators (LPAs) sustain accelerating gradients of order $100\,\mathrm{GV/m}$, but routine operation remains difficult: electron beam metrics drift over an operating shift, and the root physical cause is often invisible to the available diagnostics. We formulate LPA operation as a latent state-space model in which three effective interaction-point variables, the normalized laser amplitude $a_0$, the normalized plasma electron density $\tilde n_e$ and the residual pulse chirp $\mathcal{C}$, are inferred from routine electron beam observations by an extended Kalman filter. The emission model, which maps the latent state to the diagnostics, is kept structurally separate from the {transition} model, which describes how the latent state evolves between shots. The separation supports diagnosis in two stages, one asking which latent variable moved and one asking what moved it. The implemented emission model is a toy model, yielding an expected performance in line with current facilities and using 3D blow-out regime dependencies where relevant. We conduct synthetic sessions to test the effectiveness of the detection and attribution protocols, finding that attribution is limited by excitation rather than by shot count or diagnostic resolution. Because the construction needs only a set of physical latent variables, an emission model and a family of hardware-derived transition models, it transfers to other drift-prone subsystems. We argue that the accuracy of the whole procedure is limited by the emission model rather than by the inference method.
Read the full abstract

All authors Ou Labun, Calin Hojbota, Mara Klebonas, Mike Downer, Rafal Zgadzaj, Phil Franke, Lance Labun

Data Analysis and Statistics · Cross-listed from astro-ph.IM

Bayesian Superiority in On/Off analysis

We present a detailed comparison of Bayesian criteria with three non-informative priors - flat, Jeffreys, and scale-invariant - for testing a signal against an unknown background and compare them with the classical frequentist Li-Ma approach in the On/Off problem. We perform Monte Carlo simulations for various background levels and evaluate the Li-Ma and Bayesian criteria by their Type I error rates. We then simulate a nonzero signal and compare the criteria in terms of Type II error rates. We find that both the Li-Ma criterion and the Bayesian criterion with the Jeffreys prior control Type I error well, excluding the case of small background $b < 0.5$; the Bayesian criterion with the Jeffreys prior is statistically more powerful than the Li-Ma criterion for $b \lesssim 40$.
Read the full abstract

Data Analysis and Statistics

Combinations of measurements that are simultaneous fits of parameters of interest and systematic uncertainties using the BLUE method

Combining estimates of the same physics parameter obtained from different measurements improves the precision and robustness of the parameter determination. Modern particle physics measurements are often performed using likelihood fits that include the physics parameter(s) of interest together with nuisance parameters representing systematic uncertainties. In high-statistics analyses, typical of many analyses at the Large Hadron Collider, these nuisance parameters can be constrained in the likelihood fits. We describe how the Best Linear Unbiased Estimator method for combinations can be applied to both the estimates of the parameters of interest and the estimates of the nuisance parameters. We show with concrete example combinations that including the nuisance parameters can improve the precision on the parameters of interest. We show with pseudo-experiments that the uncertainty reported by the combination is reliable and that the approximate likelihood combination proposed in a previous publication and implemented in the Convino software reports an underestimated uncertainty. The method is implemented in an open-source software tool, Combiner.
Read the full abstract

Wednesday 9 September

Data Analysis and Statistics · Cross-listed from physics.app-ph

Surrogate-accelerated parameterisation of physics-based Li-ion battery models

Physics-based lithium-ion battery models provide access to physically meaningful internal electrochemical states and processes, but cell-specific parameter inference from terminal current-voltage data is computationally expensive and limited by identifiability. We present a surrogate-accelerated inverse framework based on a single-particle model with electrolyte dynamics (SPMe). Its forward map uses our Artiphy surrogate framework for rapid, differentiable evaluation of voltage and selected internal states. After rescaling to remove exact structural redundancies, we infer non-redundant transport, kinetic and capacity parameter groups, including concentration-dependent solid and electrolyte diffusivities. Synthetic voltage data from a Doyle-Fuller-Newman (DFN) model under a WLTP-like current protocol provide a benchmark with known reference parameters and controlled model discrepancy. The inferred SPMe reproduces the benchmark voltage with an error of order 1 mV and recovers electrode capacities well. Positive-electrode diffusivity is recovered accurately over much of the probed stoichiometric range. Local sensitivity and Fisher-information analysis identifies correlated kinetic-Ohmic and electrolyte-transport directions, and shows how localised information and the global diffusivity parameterisation can yield narrow Fisher-curvature envelopes despite weak voltage sensitivity to negative-electrode diffusion over much of the drive cycle. These results represent a step towards rapid physics-based in-silico parameterisation and reduced reliance on destructive cell characterisation.
Read the full abstract

All authors A. Emir Gumrukcuoglu, Josh Pearson, Jamie M. Foster, James Burridge

Data Analysis and Statistics · Cross-listed from quant-ph

Information capacity of quantum statistics: Fock-state tests of a discrete binary-sequence model on cloud photonic quantum processors

Our central premise is that quantum mechanics may be the statistical limit of a more fundamental discrete theory: any such theory equips a physical system with a finite information capacity, and its departure from quantum statistics is controlled by how much of that capacity the system uses. We show that commercial cloud photonic quantum processors have reached the precision required to bound this capacity from below, using the binary-sequence model of Powers et al. as the concrete test theory: outcome probabilities arise from counting discrete sequences of length $n$, quantum mechanics is recovered as $n \to \infty$, and $n$ measures the information capacity of the register behind a prepared state. Photon Fock states $|1\rangle$, $|1,1\rangle$, heralded $|2\rangle$, and cascaded beam-splitter pairs are measured on programmable interferometers with dominant systematics determined in situ. The model's composition-consistent parametrization, singled out by requiring that rotations compose, recovers quantum mechanics with deviations $1.24/n$; a random-effects likelihood analysis calibrated by parametric bootstrap excludes all $n \le 100$: the information capacity of the register carrying the two-photon state, if finite, exceeds $10^2$. Cascaded beam splitters test the composition law directly: the data are split-invariant, excluding naive count composition at $8\sigma$ and confirming the interference-sign rule. Model-independently, curve-averaged deviations from the quantum partition law larger than $2.3\times10^{-2}$ are excluded at 95% CL, and the originally published linear parametrization is excluded outright. Because the compilation offset is frozen per circuit it is calibratable, opening the $10^{-3}$ floor ($n \sim 10^3$) to current hardware: cloud photonic processors are quantitative instruments for quantum foundations, and information capacity an experimentally boundable quantity.
Read the full abstract

All authors Chiran Wijesundara, Octavia T. Volpe, Dejan Stojkovic, Herbert Fotso, Tim Thomay

Tuesday 8 September

Data Analysis and Statistics

A State-Transition Information Space for Time-Series Dynamics: Theory and Application

Characterizing dynamical organization in time series requires distinguishing the diversity of accessible states from uncertainty in their temporal transitions. Here we introduce a state-transition information space based on two normalized entropy measures derived from ordinal patterns: K_q, quantifying ordinal-state diversity, and K_t, quantifying transition uncertainty. Their joint K_t-K_q representation provides a two-dimensional framework in which dynamical regimes and their temporal evolution can be examined. Theoretical properties establish the bounded relation 0 <= K_t <= K_q <= 1, while canonical time series identify distinct empirical domains ranging from ordered dynamics to near-maximal randomness. Sliding-window analysis further shows that systems can exhibit temporal trajectories through the K_t-K_q plane rather than remaining at a fixed dynamical state. Application to 1,879 monthly naturalized streamflow records from the U.S. rivers shows that river dynamics occupy a distinct intermediate region between ordered and highly disordered regimes. Moreover, river positions shift systematically toward higher K_t and K_q with increasing drainage area, revealing scale-dependent organization of streamflow dynamics. The framework therefore provides a compact means of comparing state diversity, transition uncertainty, and their temporal and spatial organization across time-series systems.
Read the full abstract

Data Analysis and Statistics

Quantity, quality, and timing: Guiding glacier data assimilation strategies in the high Arctic

Accurate simulation of glacier surface mass balance is essential for predicting sea level rise and freshwater resources, but it is constrained by uncertainties in meteorological forcing and model parameters. Here, we deploy glacier data assimilation strategies to assess the value of observations for improving surface mass balance simulation, focusing on observation quantity, quality, and timing. We perform synthetic twin experiments on Kongsvegen glacier, Svalbard, using a Particle Batch Smoother with 1000 ensemble members. Synthetic observations of albedo, snow depth, and surface temperature are assimilated at two quality levels, under two climatic scenarios, and over 12 years. Assimilation benefit is measured as the percentage improvement in the continuous ranked probability score of the posterior glacier surface mass balance relative to the prior. A single optimally timed high quality observation yields mean improvements of up to 80%. Larger numbers of low quality observations partially compensate for lower improvement. In the accumulation zone, however, additional snow depth observations degrade performance through particle degeneracy. Optimal timing is governed by the seasonal transitions of the truth trajectory rather than by prior ensemble spread alone. The optimal windows shift by up to six weeks between early and late melting years. Joint assimilation adds value through temporal diversity rather than observational diversity, while independently timed observations outperform same day combinations. The asynchronously optimally timed combined assimilation of three variables sustains improvements of 85 to 97% across all years in the ablation zone. These findings provide guidelines for adaptive observation scheduling in glacier monitoring and reanalysis.
Read the full abstract

All authors Wenxue Cao, Kristoffer Aalstad, Louise S. Schmidt, Thomas V. Schuler

Data Analysis and Statistics · Cross-listed from cond-mat.stat-mech

Nonequilibrium stochastic thermodynamics of boundary functionals: From Zubarev's ensemble to stochastic particle separation

This paper develops a generalization of Zubarev's nonequilibrium statistical operator method for the case of simultaneous inclusion of additive and nonlocal boundary functionals of trajectories. Using a unified thermodynamic approach, a three-parameter model of nonequilibrium systems is constructed, including the first-passage time, the dwell time above a given level, and the absolute extremum of the process. A correspondence is demonstrated between the maximum information entropy method for trajectories and the Donsker-Varadan large deviation formalism. Using the Doob`s h-transform, it is demonstrated that fixing extremal functionals of history induces efficient non-Markovian transport in the system with dynamic adaptation to historical records. Criteria for the applicability of the developed apparatus in the "large time" domain and the possibilities of its use for optimizing stochastic particle separation in periodic potentials are discussed.
Read the full abstract

Data Analysis and Statistics · Cross-listed from astro-ph.CO

Finding the distribution of matter using lenses - II: deconvolution-based reconstruction with 3x2pt measurements

We present a deconvolution-based framework for testing scale-dependent departures of the late-time matter power spectrum from a fiducial cosmological model using $3\times2$pt measurements. We introduce a free-form scale-dependent modulation $A(k)$ of the fiducial nonlinear matter power spectrum and construct the linear response of binned galaxy-clustering, galaxy-galaxy-lensing, and cosmic-shear spectra to the discretized modulation $A(k)$. The response is evaluated with full-sky, beyond-Limber kernels including density, redshift-space-distortion, gravitational-shear, and intrinsic-alignment contributions. We reconstruct $A(k)$ using a regularized modified Richardson-Lucy algorithm, with weak diffusion in $\ln k$ and selection of the minimum-$\chi^2$ solution along the iteration history. Using Rubin/LSST Year 10-like synthetic data, we find that oscillatory modulations with amplitudes $\gtrsim1\%$ can be recovered over $0.1\lesssim k\lesssim0.5\,{\rm Mpc}^{-1}$, provided the oscillation frequency $f\lesssim10$ on $\log_{10}[k/(0.2\,{\rm Mpc}^{-1})]$. We further introduce a posterior-weighted consistency statistic calibrated with posterior-predictive null mocks, thereby accounting for cosmological and nuisance-parameter uncertainties without relying on Wilks' theorem. The null case is consistent with $A(k)=1$, while a $1\%$ oscillatory modulation is detected at $\sim 2.6\sigma$. These results demonstrate the potential of regularized deconvolution as a model-independent consistency test of the matter power spectrum in future $3\times2$pt surveys.
Read the full abstract

All authors Jun-Qian Jiang, Ajoy Dawn, Dhiraj Kumar Hazra, Benjamin L'Huillier, Arman Shafieloo

Data Analysis and Statistics · Cross-listed from astro-ph.CO

Finding the distribution of matter using lenses - I: deconvolution-based reconstruction with CMB lensing

The matter power spectrum is one of the primary statistical descriptors of the large-scale distribution of matter in the Universe and provides a powerful probe of cosmic structure formation. Measurements of cosmic microwave background (CMB) lensing offer an integrated view of the matter distribution over a wide range of redshifts, enabling the reconstruction of the underlying matter power spectrum. In this work, we reconstruct the reference linear matter power spectrum $ P_\text{lin}(k,0)$ from the baseline joint CMB lensing measurements of Planck PR4, ACT DR6, and SPT-3G using a covariance-weighted modified Richardson-Lucy(MRL) deconvolution algorithm. The reconstructed spectrum is found to be consistent with the fiducial linear prediction on large scales, while exhibiting a systematic enhancement for $k \gtrsim 0.1\,{\rm Mpc}^{-1}$, where nonlinear gravitational evolution becomes important. To investigate this behavior, we introduce a scale-dependent correction factor, $A(k)$, defined through $P(k)=A(k)\,P_{\rm nl}(k),$ where $P_{\rm nl}(k)$ is the fiducial nonlinear matter power spectrum obtained from 2LPT simulations. The reconstructed correction factor remains consistent with unity within $2\sigma$ confidence over the reconstructed range, indicating that the observed enhancement is well explained by the standard nonlinear evolution of the matter power spectrum. In addition, the reconstruction shows agreement with the fiducial BAO template around the BAO feature at $k\sim(0.04-0.06)\ {\rm Mpc}^{-1}$, indicating that some BAO-scale information survives the lensing projection.
Read the full abstract

All authors Ajoy Dawn, Jun-Qian Jiang, Dhiraj Kumar Hazra, Benjamin L'Huillier, Arman Shafieloo

Monday 7 September

Data Analysis and Statistics · Cross-listed from q-bio.NC

Exploring the robustness of permutation entropy analysis to differentiate between closed-eyes and open-eyes resting states

Electroencephalography (EEG) is a noninvasive technology that is widely used to monitor brain states, and many efforts are focused on developing reliable and efficient data analysis methods for EEG recordings. Here, we apply ordinal analysis to the EEG recording of the resting state of 109 healthy subjects measured in two different conditions: with eyes closed (EC state) or eyes open (EO state). We study the robustness of the temporal permutation entropy ($PE$) and the spatial permutation entropy ($SPE$) with respect to the presence of blinking artifacts in the EO recordings, the duration of the recordings, and the number of EEG channels analyzed. We perform a paired statistical test to assess whether these quantities differ significantly in the EO and EC recordings of the same subject. We find that $PE$ and $SPE$ perform surprisingly well, as they detect significant differences when calculated from the raw signals, even within a short time interval ($PE$) or from a reduced number of electrodes ($SPE$).
Read the full abstract

All authors Juan Gancio, Natalia López López, Antonio J. Pons, Giulio Tirabassi, Cristina Masoller

2609.22265 · 7 Sep · This article may be downloaded for personal use only. Any other use requires prior permission of the author and AIP Publishing. This article appeared in Gancio, Juan, et al. "Exploring the robustness of permutation entropy analysis to differentiate between closed-eyes and open-eyes resting states." Chaos (2026), and may be found at https://doi.org/10.1063/5.0326211

Data Analysis and Statistics · Cross-listed from physics.ins-det

Real-time blind indexing by cross-frame consensus

Classical crystallography determines orientation and unit-cell geometry from a rotation series collected from a single specimen, whereas serial crystallography acquires one still exposure from each of many randomly oriented microcrystals and must reconstruct a complete dataset by pooling partial measurements across the ensemble. In serial experiments, each frame typically contains only a subset of reflections, and frames with few reliable peaks (the small-N regime) are especially difficult to index blindly, because multiple incorrect lattices or orientations can explain sparse observations. The approach implemented in GLINT addresses this small-N regime by treating the dataset, rather than the individual frame, as the unit of inference. Candidate solutions are generated for many frames, weak but recurrent lattice hypotheses are pooled across the ensemble to identify a shared unit cell, and each frame is then registered against that consensus cell through a known-cell orientation search with lower dimensionality than blind indexing. In benchmark tests, GLINT matched the strongest blind indexer in the comparison on indexing yield at roughly 40x lower per-frame blind-indexing latency against XGANDALF's fastest configuration, and batched known-cell registration ran entirely on one GPU, allowing cell discovery on the fly during serial data collection. On experimental serial data, frames indexed blind by GLINT merged to crystallographic quality (CC* = 0.90), while the method rejected non-crystal images rather than forcing unsupported indexing assignments. These results demonstrate that indexing performance can be evaluated not only by indexing yield and throughput, but also by the quality of the resulting merged crystallographic data.
Read the full abstract

Saturday 5 September

Data Analysis and Statistics · Cross-listed from q-fin.TR

Explainable Deep Learning for Price-Trade Dynamics: From Black-Box Forecasts to Effective Parametric Models

Understanding the joint dynamics of prices and trades is central to market microstructure, where returns and order flow interact through nonlinear and state-dependent mechanisms. Linear models are interpretable but may miss these effects, while deep neural networks improve forecasting at the cost of transparency. We use neural networks as tools for structural discovery rather than only for prediction. A deep feed-forward network is trained on high-frequency returns and signed volumes for large- and small-tick stocks and compared with a linear VAR benchmark. The neural network improves predictive performance, especially for returns, revealing nonlinear dependencies beyond the linear specification. Using Shapley-based explainability, we show that the dominant contributions are concentrated at the most recent lags. Model-implied responses are consistent with conditional averages reconstructed from the data. Unlike empirical averages, however, the neural-network decomposition isolates individual regressor contributions to the aggregate dependence. Lagged signed volume generates sign-preserving and saturating effects, consistent with nonlinear price impact and order-flow persistence. Lagged returns act as state variables: when the previous trade does not move the price, the model predicts continuation in the direction of past order flow, whereas non-zero returns generate attenuation or reversal. Building on these findings, we introduce a parsimonious SHAP-inspired nonlinear parametric model. It reproduces the main return-volume dependencies, outperforms the linear VAR benchmark, and achieves performance comparable to the neural network. A multi-lag extension captures residual longer-memory effects while preserving interpretability. Overall, explainability offers a route from black-box prediction to economically meaningful parametric models of price and trade dynamics.
Read the full abstract

Friday 4 September

Data Analysis and Statistics

Continuous Event Weighting for Resonance Searches

We develop a continuous event weighting framework for resonance searches that uses auxiliary event information without dividing the data into increasingly fine sensitivity categories. A hard selection is a binary weight, finite categories correspond to piecewise constant weights, and in the background dominated limit their fine partition approaches a continuous signal to background density ratio weight. The construction retains auxiliary discrimination in a single physically interpretable resonance spectrum, with calculable statistical variance and direct recovery of the selected signal yield. One million fully explicit event level pseudoexperiments using Gaussian, Gamma, and Beta auxiliary observables verify yield recovery, uncertainty propagation, and frequentist coverage while substantially improving discovery sensitivity. The method is further tested on publicly available CMS Run2016G and Run2016H diphoton Open Data as an independent real data benchmark, using detector level gluon fusion Higgs simulation only as the signal reference. This is a methodological test of the framework, not a CMS Higgs analysis. The results demonstrate a practical route to exploiting continuous auxiliary information without proliferating sensitivity driven event categories.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.ins-det

Kinematic Fitting of electromagnetic calorimeter data - an improved method

Kinematic fitting is widely used in particle physics experiments as a powerful tool to improve experimental resolutions, to suppress background and to provide selection criteria for the identification of specific reactions. Kinematic fitting methods, however, typically assume that the fitted quantities follow Gaussian distributions; an assumption which does not always hold. This is particularly true for energy measurements from electromagnetic calorimeters. This issue can largely be overcome by performing a transformation of the non-Gaussian variables into variables which follow a Gaussian distribution. The new adapted kinematic fitting procedure allows to better account for the non-Gaussian nature of the measured calorimeter energies, at the same time improving the accuracy and robustness of the kinematic fit. This leads to improved pull and confidence-level distributions of the kinematic fit as well as improved invariant mass distributions and signal-to-background ratios in the final event sample.
Read the full abstract

Data Analysis and Statistics · Cross-listed from stat.ML

FluxDisco: Symbolic Regression for Stoichiometric Dynamical Systems via Monte Carlo Graph Search

Dynamical symbolic regression methods identify governing differential equations from noisy data, balancing interpretability and predictive accuracy. However, standard methods often produce expressions that violate known physical laws. To address this, we propose FluxDisco, a physics-informed framework tailored for flux-based, stoichiometric ODE systems. By leveraging a known stoichiometry, we reduce the expression search space and ensure physical adherence. Our framework adapts the Monte Carlo Graph Search algorithm for the unique challenges associated with joint flux discovery of stoichiometric systems. We evaluate our method across a range of physical and biological systems, demonstrating its ability to accurately recover governing dynamics through interpretable equations.
Read the full abstract

All authors Cassandra Durr, Alvaro Köhn-Luque, Chris Jewell, Lloyd A. C. Chapman

Data Analysis and Statistics · Cross-listed from physics.chem-ph

Derivation of the Sample-Size Scaling of TWO-NN Intrinsic-Dimension Estimates from Molecular Dynamics Trajectories

The intrinsic dimension of a dataset is the number of independent directions needed to describe the space occupied by its data. Estimators based on nearest neighbors infer this number from how the probability to find a neighbor point grows around each sampled point. Because the distances $r$ between neighbor points decrease as the sample grows, the estimated dimension can depend strongly on the number of available points. Here, we derive the large-sample behavior of the TWO-NN estimator for data drawn from a smooth $d$-dimensional space. The typical nearest-neighbor distance scales as $N^{-1/d}$, and smooth deviations from a locally uniform distribution produce successive corrections proportional to $N^{-2/d}$. We test this result using the trajectories coming from ten independent $100~\mu$s simulations of alanine dipeptide. Configurations are represented by all pairwise distances among the ten heavy atoms. This representation has a known geometric dimension of $3n_{\mathrm{at}}-6=24$. Over the investigated range, the TWO-NN estimate shows no systematic dependence on the temporal spacing between configurations, but increases from approximately $7.5$ to $15.6$ as the sample size grows from $10^2$ to $2\times10^5$. Extrapolations that retain corrections through $r^2$, $r^4$, and $r^6$ give limiting dimensions of $25.23$, $22.89$, and $27.00$, respectively. All three estimates lie close to the known dimension and collectively bracket it, supporting the proposed scaling. Their spread provides a direct estimate of the systematic uncertainty associated with the truncation. The derived scaling therefore explains the strong sample-size dependence of TWO-NN and provides a practical route from finite sample estimates to the underlying geometric dimension.
Read the full abstract

Thursday 3 September

Data Analysis and Statistics

Theoretical–operational modelling of complex experiments: parameter robustness and degeneracy in muon–electron conversion

Complex experiments infer theory parameters through a coupled chain of physical models and data reduction. We formulate the theoretical–operational model (TOM) as a typed factorization of this forward prediction and study how changes of preparation, phenomenon modelling, readout, backgrounds, and analysis project onto the local manifold generated by the physics parameters. For a smooth prediction and a locally identifiable weighted least-squares estimate, the resulting response map separates each model deformation into a parameter-equivalent component and a residual component that cannot be absorbed by a change of the fitted physics parameters. This gives local criteria for robustness, exact degeneracy, and partial degeneracy of parameter inference. The construction is applied to charged-lepton-flavour-violating muon–electron conversion in aluminium. A one-bin conversion-rate model exhibits an exact normalization degeneracy. In a two-template model of the elastic–inelastic spectrum, a common signal normalization is absorbed by the fitted conversion rate without changing an operator-sensitive nuclear-response ratio, whereas a relative elastic–inelastic efficiency change is exactly parameter-equivalent to a change of that ratio at first order. A numerical Run-I example based on published Mu2e spectra shows that a \(100\,\mathrm{keV}/c\) momentum-scale mismatch is only partially parameter-equivalent: its projection ratio onto the local tangent space generated by the elastic normalization \(R_0\) and the logarithmic inelastic-to-elastic response ratio \(\rho\) is \(\eta=0.331\), while most of the weighted spectral deformation remains as a residual shape. TOM thereby provides a local theoretical description of parameter robustness and degeneracy at the interface of particle/nuclear phenomenology and experimental realization.
Read the full abstract

Data Analysis and Statistics · Cross-listed from physics.soc-ph

Crowding controls the scaling of bus frequency with demand

Cities must allocate limited resources to maintain mobility, with uncertainties about the resulting state of the system. Analyzing roughly 3,000 bus routes with more than 4 billion yearly riders across 19 metropolitan areas worldwide, we uncover a robust scaling law of the form $f \sim (d/t)^\alpha$ with exponent $\alpha \in [1/2,\,2/3]$, linking the service frequency $f$ to passenger demand $d$ and route duration $t$. We show that this scaling emerges from a simple optimization principle: cities implicitly minimize total passenger waiting time under a fixed operational budget when both schedule frequency and crowding are taken into account. This mechanism produces two universal regimes: a frequency-dominated regime with $\alpha = 1/2$ when crowding is negligible, and a capacity-dominated regime with $\alpha = 2/3$ when most routes are overloaded. Intermediate exponents arise when only part of the network operates near capacity. Furthermore, we find that the benefits of additional investment are highly uneven across systems. For instance, our model suggests that a $20\%$ budget increase yields nearly a 5-minute reduction in daily waiting time per passenger in Boston, compared to only about 1 minute in Paris. These findings place urban transit within a broader class of constrained capacity-allocation problems, while highlighting a distinct regime in which prescribed route demands shape the allocation of limited service resources. The resulting scaling laws show how simple optimization principles can generate systematic exponents in complex transport systems, beyond the dissipation-based frameworks usually considered in physical and biological flow networks.
Read the full abstract

All authors Siddharth Patwardhan, Şirag Erkol, Filippo Radicchi, Marc Barthelemy

Data Analysis and Statistics · Cross-listed from physics.chem-ph

Learning multistate kinetics with a variational multistate committor network

The long-time dynamics of complex molecular systems often involves rare transitions across networks of metastable states. Building on transition-path theory, which provides a rigorous framework for describing rare transitions between two metastable states, we introduce the variational multistate committor network (VMCN), a neural framework that learns the probabilities of reaching each metastable state directly from molecular simulation data. From this representation, VMCN identifies state-specific commitment, candidate transition regions and committor-consistent pathways between state pairs, and an effective kinetic network characterized by transition rates. The model is trained using finite time-lag trajectory data together with boundary conditions defined on conservative state cores. Applications to a triple-well potential, trialanine isomerization, and the $c$–ring rotation in the V$_{\rm o}$ domain of a vacuolar ATPase show that VMCN recovers metastable organization, provides committor-consistent descriptions of transition mechanisms, and estimates state-to-state kinetics. VMCN further provides diagnostics for incomplete state decompositions and enables adaptive exploration of candidate metastable states and their connecting regions. By integrating VMCN with generative committor-guided path sampling (Gen-COMPAS) for chignolin, we start from two end point structures, identify a misfolded state and a candidate folding intermediate, and we direct subsequent sampling toward the resulting multistate transition network.
Read the full abstract

All authors Chenyu Tang, Cheng Giuseppe Chen, Benoît Roux, Christophe Chipot

Data Analysis and Statistics · Cross-listed from cs.LG

Neural-Network Maxent: a general extension with learned nonlinearity, applied to time-series for Desert Locust distribution modelling

Species Distribution Modelling (SDM) is essential for understanding how environmental conditions shape biodiversity, particularly for destructive pests such as the Desert Locust (Schistocerca gregaria), whose breeding dynamics are tightly coupled to rapidly evolving environmental conditions. Maxent has become the dominant method for presence-only data, but its reliance on a linear combination of hand chosen feature transforms limits its ability to capture the nonlinear, temporal relationships common in ecological monitoring, where covariates such as precipitation, soil moisture, and vegetation indices evolve meaningfully over time. Standard implementations flatten time-series covariates into independent features, discarding sequential structure that carries critical signal. We introduce RNN Maxent, an extension of the Maxent framework that replaces the fixed feature dictionary with a neural network, specifically a Gated Recurrent Unit (GRU), trained end to end via backpropagation. The approach preserves Maxent's presence only statistical foundations, background normalization, and probability calibration, differing only in that the nonlinearity is learned from data rather than fixed in advance. We apply RNN Maxent to map suitable habitat for the Desert Locust using 50 day environmental time series derived from ERA5 Land, MODIS, and Sentinel 3, maintaining a 7 day gap between covariates and presence records to yield forecasting behavior. Compared against standard Maxent, RNN Maxent improves performance across metrics (ROC AUC 0.862 std 0.036 vs. 0.792; F1 0.671 std 0.056 vs. 0.590).
Read the full abstract

All authors Alessandro Grassi, Edoardo Kimani Bellotto, Wassim El Azami, Sabrina Outmani, Maximilien Houel

Data Analysis and Statistics

Reduced latent leakage does not reliably predict lower likelihood bias in collider inference

Reusable collider representations can be evaluated through downstream discrimination and probes of retained information, but neither quantity directly tests the behaviour of score templates in a profiled likelihood. We test a specific prediction in a controlled two-channel routing protocol: if reduced physics-label readability in a nuisance branch indicates a more inference-robust representation, it should accompany a smaller profiled signal-strength bias under fixed unmodelled shifts. In a public Compact Muon Solenoid $H\rightarrow ZZ\rightarrow4\ell$ workflow, a downstream split of fixed EveNet embeddings preserves signal/background area under the receiver operating characteristic curve ($0.9894\pm0.0004$) while reducing nuisance-branch physics readability from $0.961\pm0.013$ to $0.593\pm0.030$. Probe-sensitivity and effective-rank controls exclude a failed readout and branch collapse. In a separate top quark jet-tagging workflow, the leakage reduction recurs with preserved task performance. Across two development event shards, however, its Spearman association with maximum absolute profiled bias is $0.036$, and three of six material leakage-improving transitions do not reduce that bias. A one-shot preregistered confirmation on an independently accessed shard produces material leakage reductions in all three paired seeds, while the maximum absolute bias increases in two. Thus, within the tested protocol, latent readability is a useful routing diagnostic but not a likelihood-robustness certificate. The result supports a practical validation rule: claims about inference robustness require a prespecified likelihood-facing stress test and held-out confirmation.
Read the full abstract

Data Analysis and Statistics

The Greedy Bump Bias: Local Profiling Geometry and the Look-Elsewhere Effect

When fitting a localized signal whose position or shape is not known in advance, one typically allows these parameters to vary together with the signal amplitude and chooses the values that maximize the likelihood. This freedom has two related statistical consequences. If a genuine signal is present, its fitted amplitude will be affected by a positive bias; otherwise, the same freedom increases the chance of finding an unusually signal-like background fluctuation, giving rise to the look-elsewhere effect. We show that these two effects can be understood as consequences of the same local geometry of the family of signal templates. We study this connection in a Gaussian matched-filter model, where a smooth D-dimensional family of normalized templates describes the unknown signal location or shape. In the normalized matched-filter problem, the curvature of a genuine signal peak and the fluctuations that determine the curvature of a high background peak are governed by the same template metric. This allows us to derive an explicit asymptotic relation. We then follow the problem away from the strong-signal and high-threshold limits. Separating the signal-associated maximum from the best competing maximum gives an exact decomposition of the global bias into a local profiling contribution and a contribution from remote-peak competition. In a one-dimensional Gaussian example, the second factorial cumulant accounts for most of this correction, while the third brings the prediction into close agreement with simulation. A two-point Kac–Rice calculation reproduces the second cumulant and reveals a quartic short-distance suppression of nearby maxima. The resulting picture separates the roles of local dimension, model-dependent curvature, and global extremal competition within a common framework.
Read the full abstract

Wednesday 2 September

Data Analysis and Statistics · Cross-listed from physics.ao-ph

Adjoint-based Inversion for the Diffusion Coefficient in Marine Lake Models

Marine lakes offer a unique opportunity to study how physical processes, such as turbulent mixing, biomixing, and tidal exchanges, regulate the vertical transport of heat, oxygen, and nutrients in relatively isolated ecosystems. The effective diffusion coefficient characterizes the overall mixing and is critical in models of marine lakes, but it is difficult to measure directly. This paper addresses the problem of inferring the depth-dependent diffusion coefficient from synthetic and real measurements of vertical temperature and salinity profiles. To do so, an inverse problem governed by a marine lake model described by the screened-Poisson equation (similar to Helmholtz) is formulated. The inverse problem is formulated as a nonlinear least-squares optimization problem, where the cost functional quantifies the misfit between the observed and recovered profiles. A Tikhonov regularization term is added to improve the conditioning of the discretized problem, with a regularization parameter chosen using the L-curve method. We solve this problem using an adjoint-based inexact Gauss-Newton method. To leverage the complementary information in these profiles, the formulation optimizes the fit across both datasets simultaneously. The accuracy of the reconstructed diffusion coefficient and the method's robustness to noise are investigated through comprehensive synthetic studies. Additionally, we contrast this framework with physics-informed neural networks, highlighting the advantages and limitations of each. When applied to real marine lake data, the adjoint method reconstructed a smaller diffusion coefficient in regions of strong density stratification, in line with physical expectations. The results indicate that the proposed approach provides a foundation for accurate modeling of marine lake dynamics.
Read the full abstract

All authors Alex Ho, François Blanchette, Noémi Petra, Michael N Dawson

Tuesday 1 September

Data Analysis and Statistics

Multi-fidelity Monte Carlo estimation of floor response spectra under combined seismic and structural parameter uncertainties

Floor response spectra (FRS) are essential tools for the design of non-structural elements (such as equipment or components). Given the various physical phenomena influencing FRS, high-fidelity (HF) mechanical models of the primary structure may be required to estimate them. Since numerical simulations based on such models are generally computationally expensive, this paper proposes using a multi-fidelity Monte Carlo (MFMC) approach for the efficient estimation of FRS. The method relies on using observations from a fast low-fidelity (LF) model as control variables. If the absolute value of the correlation between LF and HF samples is close to 1, this approach reduces both variance and estimation error compared to a standard Monte Carlo estimate based solely on HF data samples. Through a case study involving the reactor building of the Kashiwazaki-Kariwa nuclear power plant, we demonstrate the suitability of this method for FRS estimation. It effectively reduces variance and estimation error, even when using a LF model as simple as a single-degree-of-freedom system. We also show that the method accounts for modeling uncertainties while maintaining comparable performance. Its ease of use makes it a valuable tool for practitioners.
Read the full abstract

All authors Nils Baillie, Baptiste Kerleguer, Cyril Feau, Josselin Garnier, Fan Wang