Machine Learning
Optimal trajectory-guided stochastic co-optimization for e-fuel system design and real-time operation
Jeongdong Kim, Minsu Kim, Jonggeol Na et al. (+1 more)
E-fuels are promising long-term energy carriers supporting the net-zero transition. However, the large combinatorial design-operation spaces under renewable uncertainty make the use of mathematical programming impractical for co-optimizing e-fuel production systems. Here, we present MasCOR, a machine-learning-assisted co-optimization framework that learns from global operational trajectories. By encoding system design and renewable trends, a single MasCOR
agent generalizes dynamic operation across diverse configurations and scenarios, substantially simplifying design-operation co-optimization under uncertainty.
Benchmark comparisons against state-of-the-art reinforcement learning baselines demonstrate near-optimal performance, while computational costs are substantially lower than those of mathematical programming, enabling rapid parallel evaluation of designs within the co-optimization loop. This framework enables rapid screening of feasible design spaces together with corresponding operational policies. When applied to four potential European sites targeting e-methanol production, MasCOR shows that most locations benefit from reducing system load below 50 MW to achieve carbon-neutral methanol production, with production costs of 1.0-1.2 USD per kg. In contrast, Dunkirk (France), with limited renewable availability and high grid prices, favors system loads above 200 MW and expanded storage to exploit dynamic grid exchange and hydrogen sales to the
market. These results underscore the value of the MasCOR framework for site-specific guidance from system design to real-time operation.
Read the full abstract ▾
All authors Jeongdong Kim, Minsu Kim, Jonggeol Na, Junghwan Kim
Machine Learning · Cross-listed from cs.CV
Beyond Pixel Histories: World Models with Persistent 3D State
Samuel Garcin, Thomas Walker, Steven McDonagh et al. (+5 more)
Interactive world models continually generate
video by responding to a user's actions, enabling open-ended generation capabilities. However, existing models typically lack a 3D representation of the environment, meaning 3D consistency must be implicitly learned from data, and spatial
memory is restricted to limited temporal context windows. This results in an unrealistic user experience and presents significant obstacles to downstream tasks such as training
agents. To address this, we present PERSIST, a new paradigm of world model which simulates the
evolution of a latent 3D scene: environment, camera, and renderer. This allows us to synthesise new frames with persistent spatial
memory and consistent geometry. Both quantitative metrics and a qualitative user study show substantial improvements in spatial
memory, 3D consistency, and long-horizon stability over existing methods, enabling coherent, evolving 3D worlds. We further demonstrate novel capabilities, including synthesising diverse 3D environments from a single image, as well as enabling fine-grained, geometry-aware control over generated experiences by supporting environment editing and specification directly in 3D space. Project page: https://francelico.github.io/persist.github.io
Read the full abstract ▾
All authors Samuel Garcin, Thomas Walker, Steven McDonagh, Tim Pearce, Hakan Bilen, Tianyu He, Kaixin Wang, Jiang Bian
Machine Learning
Minimax Optimal Strategy for Delayed Observations in Online Reinforcement Learning
Harin Lee and Kevin Jamieson
We study reinforcement learning with delayed state observation, where the
agent observes the current state after some random number of time steps. We propose an
algorithm that combines the augmentation method and the upper confidence bound approach. For tabular Markov decision processes (MDPs), we derive a regret bound of $\tilde{\mathcal{O}}(H \sqrt{D_{\max} SAK})$, where $S$ and $A$ are the cardinalities of the state and action spaces, $H$ is the time horizon, $K$ is the number of episodes, and $D_{\max}$ is the maximum length of the delay. We also provide a matching lower bound up to logarithmic factors, showing the optimality of our approach. Our analytical framework formulates this problem as a special case of a broader class of MDPs, where their transition dynamics decompose into a known component and an unknown but structured component. We establish general results for this abstract setting, which may be of independent interest.
Read the full abstract ▾
Machine Learning
When Shallow Wins: Silent Failures and the Depth-Accuracy Paradox in Latent Reasoning
Subramanyam Sahoo, Aman Chadha, Vinija Jain et al. (+1 more)
Mathematical
reasoning models are widely deployed in education, automated tutoring, and decision support systems despite exhibiting fundamental computational instabilities. We demonstrate that state-of-the-art models (Qwen2.5-Math-7B) achieve 61% accuracy through a mixture of reliable and unreliable
reasoning pathways: 18.4% of correct predictions employ stable, faithful
reasoning while 81.6% emerge through computationally inconsistent pathways. Additionally, 8.8% of all predictions are silent failures – confident yet incorrect outputs. Through comprehensive analysis using novel faithfulness metrics, we reveal: (1)
reasoning quality shows weak negative correlation with correctness (r=-0.21, p=0.002), reflecting a binary classification threshold artifact rather than a monotonic inverse relationship; (2) scaling from 1.5B to 7B parameters (4.7x increase) provides zero accuracy benefit on our evaluated subset (6% of GSM8K), requiring validation on the complete
benchmark; and (3) latent
reasoning employs diverse computational strategies, with ~20% sharing CoT-like patterns. These findings highlight that
benchmark accuracy can mask computational unreliability, demanding evaluation reforms measuring stability beyond single-sample metrics.
Read the full abstract ▾
All authors Subramanyam Sahoo, Aman Chadha, Vinija Jain, Divya Chaudhary
Machine Learning
Biased Generalization in Diffusion Models
Jerome Garnier-Brun, Luca Biggio, Davide Beltrame et al. (+2 more)
Generalization in generative modeling is defined as the ability to learn an underlying distribution from a finite
dataset and produce novel samples, with evaluation largely driven by held-out performance and perceived sample quality. In practice, training is often stopped at the minimum of the test loss, taken as an operational indicator of generalization. We challenge this viewpoint by identifying a phase of biased generalization during training, in which the model continues to decrease the test loss while favoring samples with anomalously high proximity to training data. By training the same network on two disjoint
datasets and comparing the mutual distances of generated samples and their similarity to training data, we introduce a quantitative measure of bias and demonstrate its presence on real images. We then study the mechanism of bias, using a controlled hierarchical data model where access to exact scores and ground-truth statistics allows us to precisely characterize its onset. We attribute this phenomenon to the sequential nature of feature learning in deep networks, where coarse structure is learned early in a data-independent manner, while finer features are resolved later in a way that increasingly depends on individual training samples. Our results show that early stopping at the test loss minimum, while optimal under standard generalization criteria, may be insufficient for
privacy-critical applications.
Read the full abstract ▾
All authors Jerome Garnier-Brun, Luca Biggio, Davide Beltrame, Marc Mézard, Luca Saglietti
Machine Learning
Graph Hopfield Networks: Energy-Based Node Classification with Associative Memory
Abinav Rao, Alex Wa and Rishi Athavale
We introduce Graph Hopfield Networks, whose energy function couples associative
memory retrieval with graph
Laplacian smoothing for node classification. Gradient descent on this joint energy yields an iterative update interleaving Hopfield
retrieval with
Laplacian propagation.
Memory retrieval provides regime-dependent benefits: up to 2.0~pp on sparse citation networks and up to 5 pp additional robustness under feature masking; the iterative energy-descent architecture itself is a strong inductive bias, with all variants (including the
memory-disabled NoMem ablation) outperforming standard baselines on Amazon co-purchase graphs. Tuning enables graph sharpening for heterophilous
benchmarks without architectural changes.
Read the full abstract ▾
Machine Learning
Half the Nonlinearity Is Wasted: Measuring and Reallocating the Transformer's MLP Budget
Peter Balogh
We investigate when
transformer MLP nonlinearity is actually necessary. A gate with $d+1$ parameters decides when to replace the full MLP with a linear surrogate. Through systematic investigation across six models (162M-2.8B parameters), two architectures, and three corpora, we establish that nonlinearity need cannot be predicted from
token identity: cross-corpus correlation is zero ($r < 0.05$). The routing decision is fully contextual. Despite weak per-instance predictability, the gate exploits a heavily skewed distribution where most MLP computations are near-linear, achieving 25-56% linear routing at <1% perplexity cost in GPT-2. In GPT-2 Large, 11 of 36 layers beat baseline with gating and no layer exceeds 3.7% all-linear cost. This success is architecture-dependent: Pythia models show higher costs, though Pythia-2.8B's full 32-layer sweep reveals one layer that narrowly beats baseline. As a proof of concept, we progressively replace middle-layer MLPs with frozen linear
matrices: 5 of 24 layers linearize at zero cost. With a full training budget, 4 linearized layers yield a 10.2% perplexity improvement – and a two-phase gated approach pushes this to 17.3%, beating a vanilla fine-tuning control and confirming that the nonlinear MLPs at these layers were actively harmful.
Read the full abstract ▾
Machine Learning
[Re] FairDICE: A Fair Tradeoff in Multi-objective Offline RL
Peter Adema, Karim Galliamov, Aleksey Evstratovskiy et al. (+1 more)
Offline Reinforcement Learning (RL) is an emerging field of RL in which policies are learned solely from demonstrations. Within offline RL, some environments involve balancing multiple objectives, but existing multi-objective offline RL
algorithms do not provide an efficient way to find a fair compromise. FairDICE (see arXiv:2506.08062v2) seeks to fill this gap by adapting OptiDICE (an offline RL
algorithm) to automatically learn weights for multiple objectives to e.g. incentivise
fairness among objectives. As this would be a valuable contribution, this replication study examines the replicability of claims made regarding FairDICE. We find that many theoretical claims hold, but an error in the code reduces FairDICE to standard behaviour cloning in continuous environments, and many important hyperparameters were originally underspecified. After rectifying this, we show in experiments extending the original paper that FairDICE can scale to complex environments and high-dimensional rewards, though it can be reliant on (online) hyperparameter tuning. We conclude that FairDICE is a theoretically interesting method, but the experimental justification requires significant revision.
Read the full abstract ▾
All authors Peter Adema, Karim Galliamov, Aleksey Evstratovskiy, Ross Geurts
Machine Learning · Cross-listed from stat.ML
Scalable Contrastive Causal Discovery under Unknown Soft Interventions
Mingxuan Zhang, Khushi Desai, Sopho Kevlishvili et al. (+1 more)
Observational causal discovery is only identifiable up to the Markov equivalence class. While interventions can reduce this ambiguity, in practice interventions are often soft with multiple unknown targets. In many realistic scenarios, only a single intervention regime is observed. We propose a scalable causal discovery model for paired observational and interventional settings with shared underlying causal structure and unknown soft interventions. The model aggregates subset-level PDAGs and applies contrastive cross-regime orientation rules to construct a globally consistent maximal PDAG under Meek closure, enabling generalization to both in-distribution and out-of-distribution settings. Theoretically, we prove that our model is sound with respect to a restricted $\Psi$ equivalence class induced solely by the information available in the subset-restricted setting. We further show that the model asymptotically recovers the corresponding identifiable PDAG and can orient additional edges compared to non-contrastive subset-restricted methods. Experiments on synthetic data demonstrate improved causal structure recovery, generalization to unseen graphs with held-out causal mechanisms, and scalability to larger graphs, with ablations supporting the theoretical results.
Read the full abstract ▾
All authors Mingxuan Zhang, Khushi Desai, Sopho Kevlishvili, Elham Azizi
Machine Learning
A Short Note on a Variant of the Squint Algorithm
Haipeng Luo
This short note describes a simple variant of the Squint
algorithm of Koolen and Van Erven [2015] for the classic expert problem. Via an equally simple modification of their proof, we prove that this variant ensures a regret bound that resembles the one shown in a recent work by Freund et al. [2026] for a variant of the NormalHedge
algorithm [Chaudhuri et al., 2009].
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Surprisal-Rényi Free Energy
Shion Matsumoto and Ankur Arjun Mali
The forward and reverse Kullback-Leibler (KL) divergences arise as limiting objectives in learning and inference, yet they induce markedly different inductive biases that cannot be explained by expectation terms alone. We introduce the Surprisal-Rényi Free Energy (SRFE), a normalized Rényi-Chernoff log-moment functional of the likelihood ratio. Although SRFE is algebraically related to Rényi divergence for fixed values of the order parameter, its endpoint-symmetric normalization yields a distinct free-energy viewpoint for analyzing how forward- and reverse KL behavior emerges near singular limits. We show that SRFE recovers the forward and reverse KL divergences as endpoint limits and derive local expansions around both regimes in which the variance of the log-likelihood ratio appears as a first-order correction away from the KL objective. These expansions reveal an explicit mean-variance tradeoff governing departures from KL-dominated behavior. We further establish a Gibbs-type variational characterization in terms of weighted KL projections and prove that SRFE controls large deviations of excess code length through Chernoff-type bounds, giving a Minimum Description Length interpretation of the functional. We evaluate SRFE on two synthetic settings (two-dimensional Gaussian-mixture variational inference and multi-class classification) and the long-tailed CIFAR-10 and CIFAR-100 (CIFAR-LT)
datasets. On CIFAR-LT, retraining the best configuration of each loss over five seeds, SRFE matches cross-
entropy accuracy on CIFAR-10-LT and gives mixed results on CIFAR-100-LT, improving few-shot accuracy at one of three imbalance factors. Together, these results position SRFE as a variance- and tail-sensitive, normalized Rényi-Chernoff free energy that clarifies the
geometric, statistical, and large-deviation structure underlying the transition between forward- and reverse-KL limits.
Read the full abstract ▾
Machine Learning
Heterogeneous Time Constants Improve Stability in Equilibrium Propagation
Yoshimasa Kubo, Suhani Pragnesh Modi and Smit Patel
Equilibrium propagation (EP) is a biologically plausible alternative to backpropagation for training neural networks. However, existing EP models use a uniform scalar time step dt, which corresponds biologically to a membrane time constant that is heterogeneous across
neurons. Here, we introduce heterogeneous time steps (HTS) for EP by assigning
neuron-specific time constants drawn from biologically motivated distributions. We show that HTS improves training stability while maintaining competitive task performance. These results suggest that incorporating heterogeneous temporal dynamics enhances both the biological realism and robustness of
equilibrium propagation.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Beyond Cross-Validation: Adaptive Parameter Selection for Kernel-Based Gradient Descents
Xiaotong Liu, Yunwen Lei, Xiangyu Chang et al. (+1 more)
This paper proposes a novel parameter selection strategy for kernel-based gradient descent (KGD)
algorithms, integrating bias-variance analysis with the splitting method. We introduce the concept of empirical effective dimension to quantify iteration increments in KGD, deriving an adaptive parameter selection strategy that is implementable. Theoretical verifications are provided within the framework of learning theory. Utilizing the recently developed integral operator approach, we rigorously demonstrate that KGD, equipped with the proposed adaptive parameter selection strategy, achieves the optimal generalization error bound and adapts effectively to different kernels, target functions, and error metrics. Consequently, this strategy showcases significant advantages over existing parameter selection methods for KGD.
Read the full abstract ▾
All authors Xiaotong Liu, Yunwen Lei, Xiangyu Chang, Shao-Bo Lin
Machine Learning
Towards Improved Sentence Representations using Token Graphs
Krishna Sri Ipsit Mantri, Carola-Bibiane Schönlieb, Zorah Lähner et al. (+1 more)
Obtaining a single-vector representation from a Large
Language Model's (LLM)
token-level outputs is a critical step for nearly all sentence-level tasks. However, standard pooling methods like mean or max aggregation treat
tokens as an independent set, discarding the rich relational structure captured by the model's self-
attention layers and making them susceptible to signal dilution. To address this, we introduce GLOT, a lightweight, structure-aware pooling module that reframes pooling as relational learning followed by aggregation. Operating on the outputs of a frozen LLM, GLOT first constructs a latent
token-similarity graph, then refines
token representations with a graph neural network, and finally aggregates them using a readout layer. Experimentally, our approach is remarkably robust and efficient: on a diagnostic stress test where 90% of
tokens are random distractors, GLOT maintains over 97% accuracy while baseline methods collapse. Furthermore, it is competitive with state-of-the-art techniques on
benchmarks like GLUE and MTEB with 20x fewer trainable parameters and speeds up the training time by over 100x compared with parameter-efficient fine-tuning methods. Supported by a theoretical analysis of its expressive power, our work shows that learning over
token graphs is a powerful paradigm for the efficient adaptation of frozen LLMs. Our code is published at https://github.com/ipsitmantri/GLOT.
Read the full abstract ▾
All authors Krishna Sri Ipsit Mantri, Carola-Bibiane Schönlieb, Zorah Lähner, Moshe Eliasof
Machine Learning
RADAR: Learning to Route with Asymmetry-aware DistAnce Representations
Hang Yi, Ziwei Huang, Yining Ma et al. (+1 more)
Recent neural solvers have achieved strong performance on vehicle routing problems (VRPs), yet they mainly assume symmetric Euclidean distances, restricting applicability to real-world scenarios. A core challenge is encoding the relational features in asymmetric distance
matrices of VRPs. Early attempts directly encoded these
matrices but often failed to produce compact
embeddings and generalized poorly at scale. In this paper, we propose RADAR, a scalable neural framework that augments existing neural VRP solvers with the ability to handle asymmetric inputs. RADAR addresses asymmetry from both static and dynamic perspectives. It leverages Singular Value Decomposition (SVD) on the asymmetric distance
matrix to initialize compact and generalizable
embeddings that inherently encode the static asymmetry in the inbound and outbound costs of each node. To further model dynamic asymmetry in
embedding interactions during encoding, it replaces the standard softmax with Sinkhorn normalization that imposes joint row and column distance awareness in
attention weights. Extensive experiments on synthetic and real-world
benchmarks across various VRPs show that RADAR outperforms strong baselines on both in-distribution and out-of-distribution instances, demonstrating robust generalization and superior performance in solving asymmetric VRPs.
Read the full abstract ▾
All authors Hang Yi, Ziwei Huang, Yining Ma, Zhiguang Cao
Machine Learning · Cross-listed from stat.ML
Learning Order Forest for Qualitative-Attribute Data Clustering
Mingjie Zhao, Sen Feng, Yiqun Zhang et al. (+3 more)
Clustering is a fundamental approach to understanding data patterns, wherein the intuitive Euclidean distance space is commonly adopted. However, this is not the case for implicit cluster distributions reflected by qualitative attribute values, e.g., the nominal values of attributes like symptoms, marital status, etc. This paper, therefore, discovered a tree-like distance structure to flexibly represent the local order relationship among intra-attribute qualitative values. That is, treating a value as the vertex of the tree allows to capture rich order relationships among the vertex value and the others. To obtain the trees in a clustering-friendly form, a joint learning mechanism is proposed to iteratively obtain more appropriate tree structures and clusters. It turns out that the latent distance space of the whole
dataset can be well-represented by a forest consisting of the learned trees. Extensive experiments demonstrate that the joint learning adapts the forest to the clustering task to yield accurate results. Comparisons of 10 counterparts on 12 real
benchmark datasets with significance tests verify the superiority of the proposed method.
Read the full abstract ▾
All authors Mingjie Zhao, Sen Feng, Yiqun Zhang, Mengke Li, Yang Lu, Yiu-ming Cheung
Machine Learning
AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis
Pei Yang, Wanyi Chen, Asuka Yuxi Zheng et al. (+13 more)
Large
language model (LLM)
agents offer a promising data-driven approach to automating Site Reliability Engineering (SRE), yet their enterprise deployment is constrained by three challenges: restricted access to proprietary data, unsafe action execution under permission-governed environments, and the inability of closed systems to improve from failures. We present AOI (Autonomous Operations Intelligence), a trainable multi-
agent framework formulating automated operations as a structured trajectory learning problem under security constraints. Our approach integrates three key components. First, a trainable diagnostic system applies Group Relative Policy Optimization (GRPO) to distill expert-level knowledge into locally deployed open-source models, enabling preference-based learning without exposing sensitive data. Second, a read-write separated execution architecture decomposes operational trajectories into observation,
reasoning, and action phases, allowing safe learning while preventing unauthorized state mutation. Third, a Failure Trajectory Closed-Loop Evolver mines unsuccessful trajectories and converts them into corrective supervision signals, enabling continual data augmentation. Evaluated on the AIOpsLab
benchmark, our contributions yield cumulative gains. (1) The AOI runtime alone achieves 66.3% best@5 success on all 86 tasks, outperforming the prior state-of-the-art (41.9%) by 24.4 points. (2) Adding Observer GRPO training, a locally deployed 14B model reaches 42.9% avg@1 on 63 held-out tasks with unseen fault types, surpassing Claude Sonnet 4.5. (3) The Evolver converts 37 failed trajectories into diagnostic guidance, improving end-to-end avg@5 by 4.8 points while reducing variance by 35%.
Read the full abstract ▾
All authors Pei Yang, Wanyi Chen, Asuka Yuxi Zheng, Xueqian Li, Xiang Li, Haoqin Tu, Jie Xiao, Yifan Pang, Dongdong Zhang, Fuqiang Li, Alfred Long, Lynn Ai, Eric Yang, Bill Shi, Yifan Chen, Shuhan Sun
Machine Learning · Cross-listed from cs.CV
CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance
Hanyang Wang, Yiyang Liu, Jiawei Chi et al. (+3 more)
Classifier-Free Guidance (CFG) has emerged as a central approach for enhancing semantic
alignment in flow-based
diffusion models. In this paper, we explore a unified framework called CFG-Ctrl, which reinterprets CFG as a control applied to the first-order continuous-time generative flow, using the conditional-unconditional discrepancy as an error signal to adjust the velocity field. From this perspective, we summarize vanilla CFG as a proportional controller (P-control) with fixed gain, and typical follow-up variants develop extended control-law designs derived from it. However, existing methods mainly rely on linear control, inherently leading to instability, overshooting, and degraded semantic fidelity especially on large guidance scales. To address this, we introduce Sliding Mode Control CFG (SMC-CFG), which enforces the generative flow toward a rapidly convergent sliding
manifold. Specifically, we define an exponential sliding mode surface over the semantic prediction error and introduce a switching control term to establish nonlinear feedback-guided correction. Moreover, we provide a Lyapunov stability analysis to theoretically support finite-time convergence. Experiments across text-to-image generation models including Stable
Diffusion 3.5, Flux, and Qwen-Image demonstrate that SMC-CFG outperforms standard CFG in semantic
alignment and enhances robustness across a wide range of guidance scales. Project Page: https://hanyang-21.github.io/CFG-Ctrl
Read the full abstract ▾
All authors Hanyang Wang, Yiyang Liu, Jiawei Chi, Fangfu Liu, Ran Xue, Yueqi Duan
Machine Learning · Cross-listed from cs.RO
How to Peel with a Knife: Aligning Fine-Grained Manipulation with Human Preference
Toru Lin, Shuying Deng, Zhao-Heng Yin et al. (+2 more)
Many essential manipulation tasks - such as food preparation, surgery, and craftsmanship - remain difficult for autonomous
robots. These tasks are characterized not only by contact-rich, force-sensitive dynamics, but also by their "implicit" success criteria: unlike pick-and-place, task quality in these domains is continuous and subjective (e.g. how well a potato is peeled), making quantitative evaluation and reward design difficult. We present a learning framework for such tasks, using peeling with a knife as a representative example. Our approach follows a two-stage pipeline: first, we learn a robust initial policy via force-aware data collection and imitation learning, enabling generalization across object variations; second, we refine the policy through preference-based finetuning using a learned reward model that combines quantitative task metrics with qualitative human feedback, aligning policy behavior with human-evaluated task quality. Using only 50~200 peeling trajectories, our system achieves 100% same-produce success on cucumbers, apples, and potatoes, with performance improving by up to 40% through preference-based finetuning. Remarkably, policies trained on one produce category also transfer zero-shot to selected held-out produce categories, achieving 50% success on zucchini, 90% on pear, and 80% on daikon radish.
Read the full abstract ▾
All authors Toru Lin, Shuying Deng, Zhao-Heng Yin, Pieter Abbeel, Jitendra Malik
Machine Learning
Learning Demographic-Conditioned Mobility Trajectories with Aggregate Supervision
Jessie Z. Li, Zhiqing Hong, Toru Shirakawa et al. (+1 more)
Human mobility trajectories are widely studied in public health and social science, where different demographic groups exhibit significantly different mobility patterns. However, existing trajectory generation models rarely capture this heterogeneity because most trajectory
datasets lack demographic labels. To address this gap in data, we propose ATLAS, a weakly supervised approach for demographic-conditioned trajectory generation using only (i) individual trajectories without demographic labels, (ii) region-level aggregated mobility features, and (iii) region-level demographic compositions from census data. ATLAS trains a trajectory generator and fine-tunes it so that simulated mobility matches observed regional aggregates while conditioning on demographics. Experiments on real trajectory data with demographic labels show that ATLAS substantially improves demographic realism over baselines (JSD $\downarrow$ 12%–69%) and closes much of the gap to strongly supervised training. We further develop theoretical analyses for when and why ATLAS works, identifying key factors including demographic diversity across regions and the informativeness of the aggregate feature, paired with experiments demonstrating the practical implications of our theory. We release our code at https://github.com/schang-lab/ATLAS.
Read the full abstract ▾
All authors Jessie Z. Li, Zhiqing Hong, Toru Shirakawa, Serina Chang
Machine Learning · Cross-listed from cs.CR
Gravity Falls: A Comparative Analysis of Domain-Generation Algorithm (DGA) Detection Methods for Mobile Device Spearphishing
Adam Dorian Wong and John D. Hastings
Mobile devices are frequent targets of eCrime threat actors through SMS spearphishing (smishing) links that leverage Domain Generation
Algorithms (DGA) to rotate hostile infrastructure, avoid individual domain blocks, and bypass perimeter enterprise defenses. Despite this, DGA research and evaluation largely emphasize malware C2 and email phishing
datasets, leaving limited evidence on how well detectors generalize to smishing-driven domain tactics outside enterprise perimeters. This work addresses that gap by evaluating traditional and machine-learning DGA detectors against
Gravity Falls, a new
dataset derived from smishing links delivered between 2022 and 2025.
Gravity Falls captures a single threat actor's
evolution across four technique clusters, shifting from short randomized strings to dictionary concatenation and themed combo-squatting variants used for credential theft and fee/fine fraud. Two string-analysis approaches (Shannon
entropy and Exp0se) and two ML-based detectors (an LSTM classifier and COSSAS DGAD) are assessed using Top-1M domains as benign baselines. Results are strongly tactic-dependent: performance is highest on randomized-string domains but drops on dictionary concatenation and themed combo-squatting, with generally low recall across multiple tool/cluster pairings. Overall, both traditional heuristics and some common ML detection methods are ill-suited for consistently evolving DGA tactics observed in
Gravity Falls, motivating more context-aware approaches and providing a reproducible
benchmark for future evaluation.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory
Junyi Zhang, Charles Herrmann, Junhwa Hur et al. (+5 more)
Feedforward
geometric foundation models achieve strong short-window reconstruction, yet scaling them to minutes-long
videos is bottlenecked by quadratic
attention complexity or limited effective
memory in recurrent designs. We present LoGeR (Long-context
Geometric Reconstruction), a novel architecture that scales dense 3D reconstruction to extremely long sequences without post-optimization. LoGeR processes
video streams in chunks, leveraging strong bidirectional priors for high-fidelity intra-chunk
reasoning. To manage the critical challenge of coherence across chunk boundaries, we propose a learning-based hybrid
memory module. This dual-component system combines a parametric Test-Time Training (TTT)
memory to anchor the global coordinate frame and prevent scale drift, alongside a non-parametric Sliding Window
Attention (SWA) mechanism to preserve uncompressed context for high-precision adjacent
alignment. Remarkably, this
memory architecture enables LoGeR to be trained on sequences of 128 frames, and generalize up to thousands of frames during inference. Evaluated across standard
benchmarks and a newly repurposed VBR
dataset with sequences of up to 19k frames, LoGeR substantially outperforms prior state-of-the-art feedforward methods–reducing ATE on KITTI by over 74%–and achieves robust, globally consistent reconstruction over unprecedented horizons.
Read the full abstract ▾
All authors Junyi Zhang, Charles Herrmann, Junhwa Hur, Chen Sun, Ming-Hsuan Yang, Forrester Cole, Trevor Darrell, Deqing Sun
Machine Learning · Cross-listed from math.NA
Physics-informed post-processing of stabilized finite element solutions for transient convection-dominated problems
Süleyman Cengizci, Ömür Uğur and Srinivasan Natesan
The numerical simulation of convection-dominated transient transport phenomena poses significant computational challenges due to sharp gradients and propagating fronts across the spatiotemporal domain. Classical discretization methods often generate spurious oscillations, requiring advanced stabilization techniques. However, even stabilized finite element methods may require additional regularization to accurately resolve localized steep layers. On the other hand, standalone physics-informed neural networks (PINNs) struggle to capture sharp solution structures in convection-dominated regimes and typically require a large number of training epochs. This work presents a hybrid computational framework that extends the PINN-Augmented SUPG with Shock-Capturing (PASSC) methodology from steady to unsteady problems. The approach combines a semi-discrete stabilized finite element method with a PINN-based correction strategy for transient convection-
diffusion-reaction equations. Stabilization is achieved using the Streamline-Upwind Petrov-Galerkin (SUPG) formulation augmented with a YZbeta shock-capturing operator. Rather than training over the entire space-time domain, the neural network is applied selectively near the terminal time, enhancing the finite element solution using the last K_s temporal snapshots while enforcing residual constraints from the governing equations and boundary conditions. The network incorporates residual blocks with random
Fourier features and employs progressive training with adaptive loss weighting. Numerical experiments on five
benchmark problems, including boundary and interior layers, traveling waves, and nonlinear Burgers dynamics, demonstrate significant accuracy improvements at the terminal time compared to standalone stabilized finite element solutions.
Read the full abstract ▾
Machine Learning
Speculative Speculative Decoding
Tanishq Kumar, Tri Dao and Avner May
Autoregressive decoding is bottlenecked by its sequential nature. Speculative decoding has become a standard way to accelerate inference by using a fast draft model to predict upcoming
tokens from a slower target model, and then verifying them in parallel with a single target model forward pass. However, speculative decoding itself relies on a sequential dependence between speculation and verification. We introduce speculative speculative decoding (SSD) to parallelize these operations. While a verification is ongoing, the draft model predicts likely verification outcomes and prepares speculations pre-emptively for them. If the actual verification outcome is then in the predicted set, a speculation can be returned immediately, eliminating drafting overhead entirely. We identify three key challenges presented by speculative speculative decoding, and suggest principled methods to solve each. The result is Saguaro, an optimized SSD
algorithm. Our implementation is on average 30% faster than optimized speculative decoding baselines and up to 5x faster than autoregressive decoding with open source inference engines.
Read the full abstract ▾
Machine Learning
On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics
Mikhail Osipov
We investigate
geometric regularization strategies for learned latent representations in encoder–decoder reduced-order models. In a fixed experimental setting for the advection–
diffusion–reaction (ADR) equation, we model latent dynamics using a neural ODE and evaluate four regularization approaches applied during autoencoder pre-training: (a) near-isometry regularization of the decoder Jacobian, (b) a
stochastic decoder gain penalty based on random directional gains, (c) a second-order directional
curvature penalty, and (d) Stiefel projection of the first decoder layer. Across multiple seeds, we find that (a)–(c) often produce latent representations that make subsequent latent-dynamics training with a frozen autoencoder more difficult, especially for long-horizon rollouts, even when they improve local decoder smoothness or related sensitivity proxies. In contrast, (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and tends to yield better rollout performance. We discuss the hypothesis that, in this setting, the downstream impact of latent-geometry mismatch outweighs the benefits of improved decoder smoothness.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
The elbow statistic: Multiscale clustering statistical significance
Francisco J. Perez-Reche
Selecting the number of clusters remains a fundamental challenge in unsupervised learning. Existing approaches typically focus on identifying a single "optimal" partition, often overlooking statistically meaningful structure present across multiple resolutions. We introduce ElbowSig, a general inferential framework for assessing clustering structure over a range of resolutions. The method formalizes the elbow heuristic by defining a normalized discrete
curvature statistic based on the sequence of within-cluster heterogeneity values, and evaluates its significance relative to a null distribution of unstructured data. This yields hypothesis tests across resolutions, enabling simultaneous inference at multiple clustering scales. We derive the asymptotic behavior of the null statistic in both large-sample and high-dimensional regimes, characterizing its limiting form and variability. Because it depends only on the heterogeneity sequence, ElbowSig is compatible with a wide range of clustering
algorithms, including hard, fuzzy, and model-based methods. Experiments on synthetic and real
datasets show that the procedure controls Type-I error under unstructured data while providing power to detect multiscale organization, revealing structure that is often missed by single-resolution selection criteria.
Read the full abstract ▾
Machine Learning
Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations
Patrick Inoue, Florian Röhrbein and Andreas Knoblauch
While deep neural networks (DNNs) have achieved remarkable performance in tasks such as image recognition, they often struggle with generalization, learning from few examples, and continuous adaptation - abilities inherent in biological neural systems. These challenges arise due to DNNs' failure to emulate the efficient, adaptive learning mechanisms of biological networks. To address these issues, we explore the integration of neurobiologically inspired assumptions in neural network learning. This study introduces a biologically inspired learning rule that naturally integrates neurobiological principles, including sparsity, lognormal weight distributions, and adherence to Dale's law, without requiring explicit enforcement. By aligning with these core neurobiological principles, our model enhances robustness against
adversarial attacks and demonstrates superior generalization, particularly in few-shot learning scenarios. Notably, integrating these constraints leads to the emergence of biologically plausible neural representations, underscoring the efficacy of incorporating neurobiological assumptions into neural network design. Preliminary results suggest that this approach could extend from feature-specific to task-specific encoding, potentially offering insights into neural resource allocation for complex tasks.
Read the full abstract ▾
Machine Learning
SynthCharge: An Electric Vehicle Routing Instance Generator with Feasibility Screening to Enable Learning-Based Optimization and Benchmarking
Mertcan Daysalilar, Fuat Uyguroglu, Gabriel Nicolosi et al. (+1 more)
The electric vehicle routing problem with time windows (EVRPTW) extends the classical VRPTW by introducing battery capacity constraints and charging station decisions. Existing
benchmark datasets are often static and lack verifiable feasibility, which restricts reproducible evaluation of learning-based routing models. We introduce SynthCharge, a parametric generator that produces diverse, feasibility-screened EVRPTW instances across varying spatiotemporal configurations and scalable customer counts. While SynthCharge can currently generate large-scale instances of up to 500 customers, we focus our experiments on sizes ranging from 5 to 100 customers. Unlike static
benchmark suites, SynthCharge integrates instance geometry with adaptive energy capacity scaling and range-aware charging station placement. To guarantee structural validity, the generator systematically filters out unsolvable instances through a fast feasibility screening process. Ultimately, SynthCharge provides the dynamic benchmarking infrastructure needed to systematically evaluate the robustness of emerging neural routing and data-driven approaches.
Read the full abstract ▾
All authors Mertcan Daysalilar, Fuat Uyguroglu, Gabriel Nicolosi, Adam Meyers
Machine Learning
Inverse Reconstruction of Shock Time Series from Shock Response Spectrum Curves using Machine Learning
Adam Watts, Andrew Jeon, Destry Newton et al. (+1 more)
The shock response spectrum (SRS) is widely used to characterize the response of single-degree-of-freedom (SDOF) systems to transient accelerations. Because the mapping from acceleration time history to SRS is nonlinear and many-to-one, reconstructing time-domain signals from a target spectrum is inherently ill-posed. Conventional approaches address this problem through iterative optimization, typically representing signals as sums of exponentially decayed sinusoids, but these methods are computationally expensive and constrained by predefined basis functions. We propose a conditional variational autoencoder (CVAE) that learns a data-driven inverse mapping from SRS to acceleration time series. Once trained, the model generates signals consistent with prescribed target spectra without requiring iterative optimization. Experiments demonstrate improved spectral fidelity relative to classical techniques, strong generalization to unseen spectra, and inference speeds three to six orders of magnitude faster. These results establish deep generative modeling as a scalable and efficient approach for inverse SRS reconstruction.
Read the full abstract ▾
All authors Adam Watts, Andrew Jeon, Destry Newton, Ryan Bowering
Machine Learning
Coalgebras for categorical deep learning: Representability and universal approximation
Dragan Mašulović
Categorical deep learning (CDL) has recently emerged as a framework that leverages category theory to unify diverse neural architectures. While
geometric deep learning (GDL) is grounded in the specific context of invariants of group actions, CDL aims to provide domain-independent abstractions for
reasoning about models and their properties. In this paper, we contribute to this program by developing a coalgebraic foundation for equivariant representation in deep learning, as classical notions of group actions and equivariant maps are naturally generalized by the coalgebraic formalism. Our first main result demonstrates that, given an
embedding of data sets formalized as a functor from SET to VECT, and given a notion of invariant behavior on data sets modeled by an endofunctor on SET, there is a corresponding endofunctor on VECT that is compatible with the
embedding in the sense that this lifted functor recovers the analogous notion of invariant behavior on the embedded data. Building on this foundation, we then establish a universal approximation
theorem for equivariant maps in this generalized setting. We show that continuous equivariant functions can be approximated within our coalgebraic framework for a broad class of
symmetries. This work thus provides a categorical bridge between the abstract specification of invariant behavior and its concrete realization in neural architectures.
Read the full abstract ▾