Papers

Deep neural networks are capable of both learning generalizing solutions on natural tasks and memorizing arbitrary mappings. We argue that understanding the former requires understanding the latter, which is much less studied. In what ways do networks that generalize or memorize differ in their behavior or their training dynamics? Do memorizing networks simply function as a lookup table where input-output mappings are stored independently? We investigate these questions in the context of CNNs trained on image classification tasks with randomized labels or pixels. We first study the behavior of such memorizing networks on unseen images. We find that independently initialized networks trained on the same random labels learn different predictive functions: they make different predictions on test examples, in contrast with generalizing networks. We then examine the effect of dataset size on memorization through the required network capacity and training time to interpolate the training set. Counter-intuitively, we find that the number of presentations of a given example required for complete memorization can decrease with dataset size even when input-output pairs share no common structure. Although memorizing networks do not learn a universal solution, they exhibit interesting behaviors which call for further investigations into the underlying mechanisms.

Heterogeneous medical image data motivate domain-specialized diffusion experts, but expert specialization alone does not determine how their predictions should be combined during sampling. Learning a global router is particularly challenging when the server has limited, potentially unrepresentative images. We study two router-learning strategies for a fixed expert pool. Server-Only learns input- and timestep-dependent expert assignments from server images with component labels. Anchor-Delta (AD) uses the same images and frozen experts to obtain additional evidence from denoising losses. It distills targets that combine this evidence with the current router's assignments, while a proxy gate controls a subsequent refinement step. Component labels describe data origin; expert losses measure empirical prediction performance. We evaluate these distinct signals through routing analyses and paired medical image generation. An AD-trained router improves generation metrics over Server-Only in a heterogeneous four-domain setting, and a separate server-data study shows gains in held-out routing over a corrected-routing comparator. Additional paired evaluations show that the effect of inference-time evidence mixing varies across benchmarks. These findings establish server-data availability and expert performance as distinct considerations in learning routers for multi-domain diffusion models.

Co-circulating influenza, COVID-19, and respiratory syncytial virus (RSV) create a shared hospital-admission burden, because beds, staff, testing capacity, and respiratory support are allocated to total demand rather than to one pathogen at a time. This makes respiratory forecasting a target-construction problem as well as a modeling problem: disease-specific forecasts preserve attribution, whereas hospitals also need a single planning signal for combined burden. We developed a supervised 1- to 4-week-ahead framework for weekly admissions and compared two total-burden strategies: summing independently generated pathogen-specific forecasts and forecasting the combined admission target directly. Evaluation included retrospective held-out testing and prospective operational forecasts issued before the corresponding admission weeks were observed, with performance summarized using MAE and Percent Agreement (PA; the smaller of model-generated and observed counts divided by the larger). In retrospective total-burden evaluation, the combined-burden model had PA values of 0.85, 0.78, 0.74, and 0.76 and MAE values of 80.63, 200.09, 183.97, and 156.13 from week 1 to week 4 ahead, compared with summed-forecast PA values of 0.85, 0.80, 0.69, and 0.69 and MAE values of 100.14, 200.06, 205.65, and 217.57. Prospectively, the combined-burden model had higher PA and lower MAE at all four horizons, with the largest separations at weeks 3 and 4 ahead. These results support layered respiratory forecasting: pathogen-specific forecasts remain necessary for etiologic interpretation, while combined-burden forecasts better align with hospital preparedness and shared-capacity planning.

Reliable deployment of LLM-based multi-agent systems calls for runtime auditors that predict terminal outcomes from observed execution prefixes. These auditors typically learn from a limited corpus of trajectories with verified outcomes. Expanding this corpus requires costly execution and verification, yet broader failure coverage alone provides no explicit criterion for identifying prefixes that the current auditor finds difficult to classify. To this end, we propose Dream2Discern, a runtime auditor trained through implicit augmentation guided by its own prediction loss. During Dream, we select perturbations of the auditor's representation operators to increase prediction loss on recorded prefixes, yielding hard latent variants. During Discern, the auditor learns to predict execution outcomes from these variants using the original labels. The variants are regenerated as the auditor learns, turning a limited corpus into a renewable source of hard examples and requiring no additional trajectory collection or outcome verification. Extensive evaluations on AFTraj-2K and StepShield show that Dream2Discern outperforms competitive failure auditors and achieves over 90% trajectory-level weighted accuracy on both benchmarks while balancing prediction accuracy and timeliness. Project page: https://zbox1005.github.io/dream2discern-project/

Frontier models are now trained for complex multi-agent tasks, from running a company to engaging in negotiations. These tasks necessitate training models to be persuasive. But what does this optimization pressure mean for model alignment? In this work, we study how reinforcement learning (RL) training for persuasion affects models' use of misaligned strategies such as evidence fabrication and belief coercion, and what techniques can reduce such misalignment. We find that optimizing sender models solely for receiver belief shifts causes the average number of distinct misaligned strategies used per game to increase by more than 25%, and fabrications per game increase to more than 4 times relative to pre-RL models. We compare various mitigation approaches, including stronger strategy guidance, clean demonstrations, penalizing misaligned strategy usage, and improving the model's ability to detect fabrications. We find that navigating the persuasiveness-misalignment trade-off in this setting is challenging, with the best trade-offs coming from combining guidance on aligned persuasion strategies with reward penalties on misalignment. Overall, our work provides a testbed for assessing misalignment stemming from RL-based persuasion training and shows concrete paths for reducing misalignment. Importantly, it highlights that without direct intervention, persuasion-involved RL environments are likely to naturally lead to large increases in the use of misaligned strategies.

Spatial prediction estimates unobserved values from spatially distributed observations. Although flexible predictive models are increasingly available, their practical use remains limited when interpretability is essential. In this study, we formulate a graph partitioning problem to construct a global surrogate model that approximates black-box predictions with simpler representations. Specifically, we select connected subgraphs to minimize within-cluster residuals in vertex-level predictions. Although this formulation can be viewed as a variant of spatial regionalization, it operates on model-based predictions at individual locations rather than predefined areal units. With many locations, this point-level formulation may produce artificial connections between regions, reducing interpretability. We therefore introduce a constraint that excludes such connections while retaining flexible partition boundaries. We theoretically analyze how this constraint affects the resulting partition. We also develop a computational procedure for adjusting partition boundaries. Experimental results show that the proposed method yields spatially coherent partitions with flexible boundaries.

Topology optimization places material so that a structure resists deformation under given loads. Machine-learning methods often report large speed-ups over classical, non-learned optimization, but a slow classical code or a less stiff learned design can exaggerate them. We re-ran released code for eight methods on one workstation and studied two in depth: Optimize Any Topology (OAT), a diffusion model that generates designs, and a graph-based neural optimizer. Speeding up only the classical code's finite-element solver cut these two methods' speed-ups by up to 20-fold. We then timed the classical code until its design was at least as stiff as the learned one; for OAT, we read this point off the completed runs. Published timings imply a 134x speed-up for OAT on a 256 x 256 grid; our matched single-design comparisons gave 1.5x to 3.5x. At matched stiffness, the graph-based optimizer was slower than the classical code in all six two-dimensional problems. Of the six other methods, two kept their advantage, one shrank, one reversed and two were not comparable. In an exploratory audit of published claims, language models judged that 8 of 28 speed comparisons in randomly sampled, openly accessible articles lacked evidence of equal design quality; our own review counted 11, so the count depends on who judges. We provide a comparison procedure and a minimum report: classical code, hardware, design quality, what each timing includes, and data-generation and training costs.

Backward conformal prediction (BCP) fixes the size of a prediction set and estimates its miscoverage, but assumes calibration and test data are exchangeable. We extend BCP to covariate shift through importance weighting, preserving its guarantees when the density ratio is known. We show that BCP's tight miscoverage report depends only on the relative weight assigned to missed calibration examples, explaining why a better domain classifier can sometimes produce a worse estimate. Motivated by the fact that BCP depends only on nonconformity scores, we propose estimating density ratios on sorted score vectors rather than high-dimensional inputs. We prove that this approach recovers the same report asymptotically under an interpretable condition and characterize the resulting bias otherwise. Experiments with simulated and benchmark distribution shifts show that the proposed method improves BCP's miscoverage estimates, while score-based weighting consistently yields more accurate estimates than embedding-based weighting.

Large language models (LLMs) have achieved remarkable success across a wide range of language tasks. At their core, autoregressive LLMs are conditional probability models that learn to approximate the next-token distribution given the preceding context. Despite this probabilistic nature, the stochastic dynamics of autoregressive decoding have rarely been investigated from a rigorous probabilistic perspective. In this work, we define the per-step prediction error as the difference between the sampled token indicator and the model's conditional predictive distribution, and prove that the cumulative error process is a martingale with respect to the decoding filtration, implying that its conditional first-order moment is identically zero. Beyond this first-order property, we derive a closed-form expression for the second-order behavior of the martingale sequence: its predictable quadratic variation reduces to $\sum_t\left(1-p(y_t\mid y_{<t})\right)$, computable from realized token probabilities alone. This identity establishes a direct theoretical connection between sequence fluctuations and predictive uncertainty during autoregressive decoding. White-box experiments on Llama-3.x and Qwen2.5 families (1B to 72B parameters) verify that the empirically measured second-order moments closely match the theoretical predictions derived from our martingale analysis. These results provide a probabilistic framework for analyzing autoregressive LLM decoding, reveal intrinsic probabilistic fingerprints of different language models, and establish a rigorous mathematical foundation for LLM-generated text detection and uncertainty quantification.

We propose a conservative local neural operator (CLNO) for accurate and generalizable autoregressive prediction of compressible flows. Building upon the locality and translation invariance of the local neural operator (LNO), CLNO introduces a conservative evolution framework based on local subdomains of fixed physical size, separating local mean-state evolution from fine-scale reconstruction. The flow is encoded into subdomain-integrated conservative states and fine-scale features, while a shared local network predicts interface fluxes for finite-volume updates that enforce discrete conservation. A convolutional reconstruction network recovers fine-scale structures using zero-mean basis functions, preserving the updated local means and providing information for subsequent evolution. To accommodate different physical parameters, Reynolds–Mach number conditioning modulates the encoding, flux-prediction, and reconstruction networks, enabling a single model to learn parameter-dependent dynamics and generalize to unseen parameter combinations without retraining. CLNO supports prediction across spatial resolutions, computational domain sizes, and boundary conditions. Numerical experiments demonstrate improved generalization and competitive or higher accuracy compared with LNO, including order-of-magnitude error reductions in representative cases, at approximately three to five times the inference cost on periodic benchmarks. These results demonstrate accurate, generalizable, and physically consistent flow prediction through conservative local evolution and fine-scale reconstruction.

Constant-step stochastic gradient descent with momentum (SGDM) augments the parameter recursion with a momentum state that carries geometrically decaying memory of past stochastic gradients. We study this coupled recursion as a nonlinear time series and develop a framework for its stationary behavior and distributional limits. The joint parameter-momentum process forgets its initialization geometrically under a finite $p$th moment for $p > 1$, yielding a unique stationary causal solution. Under a second-moment condition, this contraction leads to an invariance principle and quenched Gaussian limit for averaged SGDM, while a small-learning-rate analysis gives a Gaussian limit for the last iterate. In a stochastic-curvature quadratic model, we further study how the rate of initialization forgetting relates to stationary variance and averaged fluctuations.

Expectiles are tail-sensitive distributional summaries with important applications in financial risk measurement and forecasting, yet statistical theory for their online estimation from dependent data remains limited. We study a one-pass, stochastic-gradient algorithm with constant learning rate $\gamma$ for estimating the marginal expectile of a stationary causal process. The recursion admits a unique stationary causal solution and geometrically forgets its initialization. Under short-range dependence, its last iterate is asymptotically Gaussian at the $\sqrt{\gamma}$ scale, with variance governed by the population curvature and the long-run variance of the expectile score. For long-memory linear processes the scale changes to $\gamma^{\beta-1/2}\ell(1/\gamma)$, with a Gaussian limit arising from a fractional Brownian motion integral. We further derive the joint long-memory limit for multiple expectile levels and characterize its limiting covariance structure. Various numerical experiments corroborate the predicted Gaussian approximations and regime-specific fluctuation scales.

Layer-wise post-training quantization methods let each layer compensate for the quantization error inherited from earlier layers. We ask under what assumptions such compensation guarantees a network output error no larger than without it. For linear networks of any depth, with the required calibration Gram matrices invertible, if the rounding error of each layer has zero conditional mean and a deterministic second-moment map common to both schemes, as under subtractive dither on an unbounded grid, then compensation with strengths in $[0,1]$ fixed in advance gives a mean-square output error no larger than no compensation, each scheme being followed along its own trajectory. The exact gain separates error removed directly from reduced excitation of later rounding noise. Outside this setting the comparison can fail. With rectified linear activations it fails for two independent reasons: noise changes activation states, and, even when no state changes, the correction moves error from samples on which a unit is inactive to samples on which it is active. With stable activations, commutators between the calibration projection and the activation patterns bound the excess. With a bounded codebook, even unique exact layer-wise minimizers with no overload can give a larger output error. For fixed rounding errors it already fails for two scalar linear layers, refuting a published universal guarantee. An exact residual identity gives the necessary and sufficient condition on a common layer input. All these comparisons hold the calibration input fixed. On two 8B models at 4 bits, fixed half-strength compensation beat uncompensated rounding in every model, text and draw tested, yet no rule for choosing the strength beat a layer-wise rule consistently, and lowering the full-network loss on the data used to choose did not reliably lower it on new text.

Certified robustness is critical for deploying neural networks in safety-sensitive settings, but computing exact certificates typically requires solving nonconvex optimization problems. Practical verifiers use convex relaxations: linear programming (LP) relaxations are scalable but often too conservative, whereas semidefinite programming (SDP) relaxations can be substantially tighter but are computationally expensive. We introduce a hierarchy of second-order cone programming (SOCP) relaxations that provide stronger certificates than LP while avoiding the cost of global positive-semidefinite constraints. Our construction combines convex-hull relaxations of ReLU activations, selective McCormick lifting over certified variable bounds, and second-order cone constraints induced by 2-by-2 principal minors. Adding these constraints progressively yields a monotone hierarchy of increasingly tight bounds with a controllable computation-tightness trade-off. On fully connected and convolutional networks on MNIST and CIFAR-10, SOCP-based verification improves certified robust accuracy over LP-based verification and is competitive with alpha,beta-CROWN, while remaining orders of magnitude faster than SDP-based verification. These results position selective second-order constraints as a practical middle ground between scalable LP-based and tight SDP-based certification. Finally, we introduce class-optimized robust evaluation (CORE), which applies tighter relaxations only to target classes not certified by cheaper ones. On the fully connected MNIST benchmark, CORE variants match SDP-cert's certified robust accuracy while reducing average verification time by up to 86.30%.

Retrieval-Augmented Generation (RAG) systems are increasingly used for biomedical Question Answering (QA). However, evaluating their effectiveness remains challenging due to the lack of reliable reference resources and the frequent misalignment between automatic evaluation metrics and human judgments. One practical question is how evaluation results should be used to guide development decisions, especially when they correlate weakly with human judgments and may even lead to different conclusions. We explore how to use a small amount of human preference signals to calibrate direct assessment evaluation metrics, and show that calibrated metrics consistently outperform baselines that either treat all metrics equally or rely on a single best metric across multiple datasets.

DNA methylation is a critical epigenetic modification that regulates gene expression and genome stability, primarily catalyzed by the DNA methyltransferase 3 (DNMT3) family of enzymes. Distinguishing precise sequence-specific flanking preferences between DNMT3A and DNMT3B remains computationally challenging due to the complex and high-dimensional nature of biochemical sequence data. In this study, we propose a novel Quantum Machine Learning (QML) approach to classify DNA methylation preferences, leveraging a Variational Quantum Classifier (VQC). Using experimentally derived deep enzymology data, we constructed a dataset containing 4,096 unique NNNCGNNN motifs. We evaluated a four-qubit VQC using a custom NumPy exact-statevector simulator. The described two-layer circuit contains 24 trainable rotations, eight CNOT gates, and eight input-encoding rotations, totaling 40 gates before measurement. Across five initialization seeds, the revised VQC evaluation achieved a ROC-AUC of 0.661 pm 0.009, where variability denotes sample standard deviation. The held-out test partition is approximately balanced, containing 200 DNMT3A-labeled motifs and 210 DNMT3B-labeled motifs. An always-DNMT3B predictor achieves 51.2% accuracy and a balanced accuracy of 0.500 on this test partition. This proof-of-concept establishes a foundation for utilizing near-term quantum architectures in complex epigenetic classification tasks, paving the way for hybrid quantum-classical pipelines in bioinformatics.

A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the value was corrupted between the instrument and the software. The three call for different responses and look alike in a single sample. We state the three-cause problem, prove which pairs of causes one channel cannot separate, and propose a bank of online detectors - a Kalman tracker with CUSUM and EWMA charts on its innovation, a Hampel median with generalized-ESD confirmation, a robust noise statistic with ADWIN, exact-value checks and an optional two-channel complementary filter - combined by a deterministic, inspectable attribution rule. The detectors were merged into a public algorithm library. On 2,942 injected events of thirteen fault types the method detects 96.3% (95% interval 95.7-97.0) at 0.22 false alarms per hour and attributes 91.2% (90.6-91.8) of detected events correctly from one channel and 99.0% (98.6-99.4) with a second, rate channel; from one channel only 2 of 207 detected sensor drifts are attributed to the instrument, as the problem statement predicts. Injected into recorded industrial signals, the same catalogue is detected at 80.8% and attributed correctly at 84.5%, at 3.6 false alarms per hour. On the Skoltech Anomaly Benchmark the method detects 32 of 33 experiments with a median delay of 15 s, at 46 false alarms per hour against 11 for a rolling z-score.

Knowledge distillation transfers representational knowledge from a large teacher network to a compact student. Existing relational and contrastive distillation methods treat all samples uniformly, spending equal gradient on well-matched and poorly-matched regions of the representation space alike. We propose Representational Difference Distillation (RDD), which diagnoses where teacher and student representations structurally disagree and concentrates the distillation signal on those regions. RDD uses two asymmetric affinity matrices that capture opposing directions of misalignment: one identifies samples the student incorrectly considers similar, the other identifies teacher relationships the student has yet to learn. These affinities determine which sample pairs the distillation loss aligns and how strongly it corrects each misaligned sample. On CIFAR-100 with resnet32x4 to resnet8x4, RDD reaches 75.58% test accuracy in our runs, on par with CRD (75.45%) and above KD and RKD. We additionally explore curriculum-based and sampling-based variants, finding that weighting the loss is more effective than filtering the data.

In this paper, we propose the Dissipative Lagrangian Neural Network (dLNN), a continuous-time system identification technique that separately learns the conservative and dissipative dynamics of a mechanical system directly from data. A key contribution is a novel parametrisation of dissipative forces that (i) is structurally constrained to be dissipative, enforcing passivity of the learned model, (ii) cannot absorb conservative mechanics, and (iii) retains sufficient expressivity to represent a broad family of state-dependent damping and regularised friction forces. The model is validated on prediction tasks on two real-world mechanical systems: trajectory prediction of a chaotic double pendulum and inverse-dynamics modelling of a six-degree-of-freedom industrial robot. Across these tasks, the dLNN demonstrates improved predictive performance over existing Lagrangian architectures while remaining competitive with a black-box neural model. The imposed physical structure also improves generalisation to unseen states, with the benefits becoming particularly pronounced when training data is limited. Finally, we introduce a symbolic distillation procedure that extracts closed-form expressions from the learned physical components. Applied to the industrial robot, this produces interpretable symbolic models that retain high predictive accuracy while reducing the computational overhead associated with evaluating neural Lagrangian models.

In this work we present a probabilistic model for predicting the temporal evolution of wildfire size using conditional generative algorithms, namely flow matching. Rather than predicting spatially resolved fire progression, the proposed model forecasts a time series of fire acreage, substantially reducing computational cost and enabling efficient integration with downstream applications such as early warning systems and risk assessment tools. Given estimates of the fire size at the current and previous forecasting steps, alongside spatially averaged weather conditions, terrain characteristics, and fuel composition, the model predicts the change in fire area over a 3-hour interval. Training data are drawn from 140 coupled atmosphere-wildfire simulations generated using WRF-SFIRE, paired with weather data from the North American Mesoscale (NAM) model. Sensitivity analysis demonstrates that the model captures physically-consistent responses to variations in wind speed, temperature, and relative humidity. Model performance is evaluated against reference solutions from WRF-SFIRE. For single-step predictions, the model achieves a symmetric mean absolute percentage error (sMAPE) of 0.39 relative to the reference WRF-SFIRE solutions. Recursive multi-step predictions over 24 hours are performed for 12 held-out test cases, where the flow matching model achieves a final fire area sMAPE of 0.39 while further providing meaningful quantification of forecast uncertainty.

Spiking Transformers achieve energy efficiency by discarding softmax normalization in favor of thresholded, binary attention. However, this removal breaks the thermodynamic framework used to analyze attention networks, as the normalized measure and partition function cease to exist. We restore this theoretical foundation by proving that softmax-free spiking attention operates exactly as the zero-temperature limit of a product of independent two-level systems. This correspondence places spiking attention strictly in the frozen ordered phase of the thermodynamic phase diagram for all inputs. Consequently, the entropic cost of organizing attention collapses, replaced by a Landauer-type physical floor governed by action-potential generation. Furthermore, this ordered-phase pinning mathematically explains the optimization difficulty caused by vanishing surrogate gradients. We validate this theory with forward-relaxation experiments, placing both standard and neuromorphic attention mechanisms on a unified thermodynamic axis.

We study pretraining when compute is abundant but unique training data are finite, requiring repeated training on the same data. We first revisit infinite-compute pretraining (Kim et al. 2026) and show that its frontier depends critically on choices previously fixed or studied only in isolation. In particular, optimizing batch size together with epoch counts reveals a U-shaped relationship between validation loss and batch size, favoring smaller batches than typical pretraining, likely because small-batch gradient noise provides implicit regularization. We further find that training recipes, previously optimized under a single-pass assumption, can be tuned, e.g., repacking documents into new sequences every epoch, weight tying, and positive embedding weight decay. We then relax the infinite-compute assumption. In particular, smaller-than-typical batch sizes can require prohibitively long sequential training, motivating to treat GPUs and time as separate resources. Tighter deadlines favor larger batches, and latency-oriented parallelism can trade additional GPUs for shorter training. Nevertheless, even under a fixed time budget, a smaller batch with fewer GPUs can achieve lower validation loss than a larger batch with more GPUs, challenging the common practice of increasing data parallelism to maximize GPU utilization under a deadline. Together, our results identify underexplored factors in repeated-data training and provide concrete guidance on how training recipes and resource allocation should change from single-pass pretraining.

High-pileup conditions in CMS during the HL-LHC era make charged-particle tracking increasingly challenging as detector occupancy and combinatorics grow. We present a hybrid approach that exploits Line Segment Tracking (LST) objects rather than individual hits to enable the first CMS ML-based track reconstruction algorithm. The LST objects, namely the triplets used in this study, are built according to geometry- and physics-driven criteria and carry richer local structure, geometrical cues, and physics quantities such as transverse momentum, which helps the model incorporate detector-level information from the start. We apply a Transformer encoder with geometrically masked sparse self-attention to embed these objects in a learned latent space, and use object condensation to assemble full tracks in a single inference step. This object-based representation reduces complexity in dense environments and makes the model less sensitive to ambiguities introduced by high pileup. Evaluated on CMS Phase-2 simulation of $t\bar{t}$ events at $\langle \mu \rangle = 200$, the approach achieves reconstruction efficiency comparable to the LST quintuplet baseline while reducing the fake rate by roughly a factor of three and suppressing duplicates almost entirely, demonstrating the promise of advanced ML architectures for next-generation particle tracking.

Emergent misalignment (EM) occurs when fine-tuning a language model to a narrow task makes it misaligned on unrelated ones. Because organizations now routinely fine-tune models on their own data, it matters which kinds of training data cause EM. So far, studies have induced EM with data containing recognized safety failures, such as insecure code or harmful advice, while biased training data is typically examined by fairness research for the bias models learn from it rather than for its effects on safety. In real-world text, much of this bias, such as demographic stereotypes, appears not as derogatory language but politely, embedded in judgments about people. We ask whether learning such polite stereotypes can itself cause EM. We find that fine-tuning on polite answers that rely on stereotypes makes models less safe and less truthful on unrelated tasks. The effect holds across four open-weight model families (7B to 12B), both gender and race stereotypes, and repeated training runs, relative to controls trained on the same questions without the stereotype. We further show that the learned stereotype has its own direction in model activations, distinct from directions for generic misaligned traits such as deception. This direction exists before fine-tuning and can both control the learned stereotype and monitor its acquisition. Existing defenses against EM improve safety while leaving the stereotype largely intact. Beyond this controlled setting, fine-tuning on political news coverage also triggers EM, apparently amplified by partisan framing, and stereotype models acting as tool-using agents can carry out harmful tasks and blame people without evidence. Our results show that biased training data is not only a fairness problem but also a safety one. We argue that EM is studied too narrowly, and that fairness concerns and methods offer a natural way to broaden the search for its triggers.

Transformer models trained on symbolic music have become remarkably capable. However, it is yet unclear how they represent music structure, and whether understanding their internal representations can be used to unlock methods for model control. We examine this through a case study of harmonic analysis: when a music transformer navigates a cadence, what does it internally "know" about its harmonic context, and can that knowledge be used to affect the generation? Using Aria, a LLaMA-based symbolic music transformer evaluated on Bach chorales, we find that harmonic decisions are causally localized progressively from middle layers onward. Linear probes show that mode becomes decodable in the same layer range where patching identifies growing causal effect, while key identity emerges later. We then confirm these representations are actionable for music generation control: steering via latent directions corresponding to harmonic changes reliably produces a deceptive or minor resolution, while preserving structural coherence and generation quality. This demonstrates that, at inference time, the model's internal harmonic knowledge can be directly manipulated to control the harmonic output.

We study online multiclass multicalibration over $T$ rounds, and give an algorithm with beyond-worst-case guarantees. With $n$ possible outcomes, forecasts lie in a simplex of dimension $d=n-1$. In the worst case, cumulative $\ell_1$ multicalibration error scales with $\widetilde\Theta(T^{(d+1)/(d+2)})$, which becomes untenable once $d$ exceeds a small constant. Our algorithm adapts to the geometry of the realized sequence: if the sequence of adversarially selected outcome means lies in an affine subspace of dimension $s$, it obtains error $\widetilde O(n^{1/(s+2)}T^{(s+1)/(s+2)})$ with high probability. The algorithm adapts to this structure without knowing the subspace or its dimension. For $T\ge n\ge2$, this rate is minimax optimal up to polylogarithmic factors. For a stochastic instance with a fixed conditional mean, the bound specializes to $\widetilde O(\sqrt{nT})$, giving a square-root dependence on the horizon.

Small area estimation (SAE) traditionally borrows strength across domains through a prespecified working model. We introduce small area learning (SAL), a learner-agnostic framework that formulates SAE as a multi-target semi-supervised learning problem and treats outcome prediction as a nuisance task. Areas may differ through unrestricted covariate shift and structured conditional shift, with information shared through common and area-specific components. We develop standard and cross-fitted estimators for low-complexity and flexible learners, unifying synthetic, model-based, and fully residual-corrected estimation. For area means, we establish consistency and asymptotic normality and provide variance estimators and confidence intervals. We extend the theory to settings with only aggregated population covariates and characterize the resulting aggregation bias. SAL reveals a bias–variance trade-off: partial residual correction can reduce mean squared error, whereas full correction yields estimators robust to working-model misspecification, potentially at higher variance. Simulations yield RMSE reductions of 34%–49% relative to direct estimation, with the best configuration varying across settings. An empirical application favours a cross-fitted estimator using a flexible learner under the chosen validation criterion. The numerical studies also illustrate how shrinkage can produce overly optimistic confidence intervals at the expense of strong modeling assumptions.

Ransomware is a fast-escalating privacy threat causing significant financial and data losses worldwide. As attacks grow more sophisticated, traditional static signatures and dynamic behavior analysis struggle to defend. In this paper, we propose an novel framework that couples software-level scanning with hardware-level microprocessor activity monitoring to achieve efficient ransomware detection and mitigation. The contributions are threefold: (1) we employ adversarial training to harden static analysis against obfuscation and leverage hardware-assisted monitoring for real-time detection; (2) we integrate neural architecture search (NAS) to automatically tailor models for stronger generalizability; and (3) We validate the proposed approach through extensive experiments, demonstrating substantially improved detection accuracy and reduced latency compared with existing methods. Moreover, it provides trace-level interpretability that explains the underlying decision process.

Heap-leaching hydraulics vary between regions of an ore bed, motivating fast prediction models that accommodate different material properties. This study develops a parameter-conditioned physics-guided machine-learning (PGML) surrogate and demonstrates its use in simulated saturation control. A dataset of 4096 numerical Richards-equation trajectories was divided into 2867 training, 614 validation and 615 evaluation cases. The network predicts hourly saturation profiles from the current profile, irrigation rate and four hydraulic parameters, and is trained with an approximate Gardner mass-balance penalty. A daily predictive controller compared 200 constant irrigation rates over a 24-hour horizon. Across the evaluation cases, the median full-cycle tracking deviation was 4.9% of a 0.70 saturation target, equal to that of a tuned proportional-integral (PI) controller, while the PGML controller reached the target without overshoot in the median and applied approximately 4% less irrigation. Moderate physics guidance reduced long-horizon prediction error by approximately 30% relative to data-only training and reduced irrigation-bound activity, but slightly worsened closed-loop tracking. For a matched prediction workload, the surrogate was approximately 70 times faster than the numerical solver, and batched evaluation cost under 2 ms per column for 5000 columns. The results support a reusable physics-guided surrogate for large-scale saturation control in simulation.

Predictive coding networks (PCNs) promise properties not offered by backpropagation-trained networks: learning from purely local information, a single network that both discriminates and generates, and flexible inference by energy minimization. However, their iterative inference is expensive, making hardware acceleration desirable, and a useful, efficient accelerator needs both depth and aggressive quantization. Canonical state-based PC delivers neither: its inference signal decays exponentially with depth, so deep models fail to train, and low precision makes it worse. We show that pairing two recent methods carries the full PCN property set into the deep, 4-bit regime: bidirectional PC (bPC), which unifies discrimination and generation, and the error-based reparameterization (ePC), which optimizes prediction errors rather than states during inference. Error optimization delivers the output signal to every layer in a single backward pass, removing the signal decay; one additional element, global-norm clipping of the error gradients, stabilizes the error dynamics at depth. The resulting ePC+bPC model trains stably up to at least eight layers and retains its full-precision accuracy and clean class-conditional generation under W4A4E4 quantization-aware training, across FP4 and three posit formats. On six-layer networks, where state-based bPC collapses to chance, ePC+bPC classifies more than 50 accuracy points above it after one epoch of training, with class-conditional generations 5-9x better in RMSE. 4-bit quantization is nearly free: full-precision and 4-bit accuracies differ by at most ~5 points, and by under two in the best-performing formats. We characterize the two failure modes the method overcomes, bounded signal decay in state-based PC and error divergence in unregularized error optimization, and discuss implications for a 4-bit-native PCN accelerator.