Numerical Analysis
Papers
Fresh from arXiv, typeset properly. Highlighted words trade on the market.
Thursday 8 October
Nested integrals $\int f\left(\int g(\boldsymbol{y},\boldsymbol{x})\mathrm{d}{}\boldsymbol{x}\right)\mathrm{d}{}\boldsymbol{y}$ where $\boldsymbol{y}\in[0,1]^{d_1}$ and $\boldsymbol{x}\in[0,1]^{d_2}$ frequently arise in Bayesian experimental design, medical decision making, and computational finance. Statistical estimators for nested integrals are more computationally demanding than for standard integrals. Recent studies demonstrate that the median randomized quasi-Monte Carlo (mRQMC) method achieves super-polynomial convergence rates for analytic integrands. We extend mRQMC methods to nested integrations and propose a novel double-loop mRQMC estimator with super-polynomial convergence rate if both the inner and outer integrands are analytic. Established estimators for nested integrals, such as double-loop or multilevel Monte Carlo, are by contrast limited to algebraic convergence rates. However, expected information gain (EIG) applications from experimental design typically do not fulfill this condition due to a lack of regularity in the outer integrand. We address this through a theoretically-grounded importance sampling regularization scheme. Although super-polynomial convergence rates remain achievable for the inner integral, only the algebraic rate $\alpha+3/2-\epsilon$ for any $\epsilon>0$ can be theoretically guaranteed for the outer integral in the EIG, where $\alpha\in\mathbb{N}\cup\{0\}$ denotes the order of the mixed derivatives of the regularized outer integrand. Our approach shows substantial improvements over double-loop randomized quasi-Monte Carlo methods based on the sample mean in moderately high-dimensional settings for a simple polynomial example. Moreover, we demonstrate improved convergence behavior of the median-based estimator for the more challenging setting of EIG estimation.
Robotics
PriMoRetarget: Humanoid Motion Transfer Through Prioritized Motion Retargeting
Motion retargeting is a critical preprocessing step for transferring human motion data to humanoid robots. While existing methods are effective for single-character motions, interactions involving multiple characters and objects introduce additional contacts and spatial relationships that must be preserved. Retargeting is commonly formulated as a weighted multi-objective optimization problem, where conflicting objectives can require careful tuning of relative weights and can still compromise important motion characteristics. We introduce PriMoRetarget, a prioritized motion-retargeting framework that replaces relative objective weighting with an explicit priority hierarchy. Lower-priority updates are constrained to preserve higher-priority objectives, allowing critical characteristics such as foot placement and interaction contacts to take precedence over lower-priority pose objectives. The method is evaluated on four datasets spanning single-character motion, character-object manipulation, and character-character interactions. The resulting motions better preserve task-relevant interactions and provide reference trajectories that can be tracked more reliably than those produced by widely used retargeting methods.
Read the full abstract
Machine Learning
Machine Learning and Time-Series Foundation Models for Day-Ahead Electricity Forecasting in a University Building
Day-ahead electricity forecasts can help building operators schedule heating, ventilation, and other services around anticipated demand. Choosing a forecasting model also requires balancing accuracy with its forecast-time inputs, local training, and computational requirements. We benchmark statistical, machine-learning, recurrent neural-network, and pretrained TimesFM models for a university building. We use chronological validation and evaluate all configurations on the same 57 test days, comparing accuracy across workdays, weekends, and holidays alongside local fitting and forecasting time. Weather covariates use retrospective observations. TimesFM 3.0 with covariates achieves the lowest overall errors, reducing mean absolute error by approximately 43% relative to weekly seasonal-naive forecasting. XGBoost remains competitive, although paired daily comparisons do not establish a clear mean-absolute-error advantage for TimesFM. TimesFM also performs strongly using only past electricity consumption. Adding covariates improves its holiday forecasts but slightly worsens workday and weekend forecasts, leaving overall mean absolute error almost unchanged. TimesFM requires no building-specific weight training, while XGBoost offers rapid local fitting and inference. The evaluated LSTM takes longer to fit without outperforming either approach. An exploratory historical evaluation reveals seasonal variation, while a separate weather-source comparison produces only small changes in the selected TimesFM configuration's accuracy. These results support pretrained TimesFM as a strong initial forecasting baseline for this building, with XGBoost as a competitive locally fitted alternative. Covariates are particularly useful when improved holiday forecasts justify the additional input preparation.
Read the full abstract
Robotics
Distributed Harmonic-Covariance Feedback for Hole Closure in Planar Robot Swarms
We consider hole closure in a planar multi-agent system whose sensing disks form a time-varying Čech complex. We propose a harmonic-covariance feedback that uses harmonic edge signals to locate holes and converts those signals into agent forces. For distributed evaluation, we use a fixed-degree polynomial of the regularized Hodge Laplacian. This polynomial is evaluated through local operator applications. The closed loop has a unique global trajectory from every safe initial state. The model data and initial state are supplied with procedures for approximation to any requested accuracy with explicit error bounds. We assume that hole area is measurable and bounded, holes evolve continuously without splitting or merging, and only finitely many temporary new holes appear. Under these conditions, offline finite calculations produce rational lower bounds for the cumulative hole area on the time horizon without querying trajectory data. These bounds converge to that cumulative area, whether finite or infinite. If it is finite, holes close in the time-averaged sense: the average normalized hole area over every fixed-length time window tends to zero as time tends to infinity. In a 15-second numerical demo, all holes close by 9.9s and remain closed through the end.
Read the full abstract
Robotics
Selectively Actuated Fiberbots for High-precision Minimally Invasive Surgery
Long, slender instruments are essential for minimally invasive surgery (MIS), but achieving precise distal manipulation becomes increasingly difficult when actuation extends along the entire instrument. Here, we present an electrothermally actuated fiber-based robot designed for MIS procedures, in which Joule heating is concentrated in the distal active segment. The fiber is fabricated using a thermal drawing process and uses embedded metallic wires for thermal actuation. This robotic fiber, 1.9 mm in diameter and 1.2 m in length, can precisely manipulate surgical laser fibers with positional precision below 100 $\mu$m. The compact size of the fiber robot allows it to pass through the working channel of a flexible endoscope, while keeping both devices mechanically uncoupled to allow independent motion. A phantom demonstration combining the robotic fiber with a da Vinci instrument conceptually demonstrated its compatibility with existing endoscopic systems for clinical applications. Our robotic fiber enables precise distal motion and holds the potential to improve clinical outcomes by enhancing precision in MIS procedures.
Read the full abstract
Robotics
Image-Based Path-Planning for Navigation of Soft-Growing Robots in the Spinal Sub-Arachnoid Space
Recently, soft robotic procedures are being developed aiming at achieving targeted minimally invasive operations or drug administration in specific locations in the human body. Accordingly, specialized path-planning algorithms need to be investigated for the estimation of efficient paths that would protect the delicate tissue structures and simultaneously satisfy a set of requirements for the efficient navigation of the robot. In this context, we propose an image-based 3D pathplanning algorithm, which is an extension of the A* algorithm, for the navigation of a soft-growing robot inside the spinal subarachnoid space (SSS). The proposed algorithm is capable of estimating a safe pathway towards the goal location, while ensuring the establishment of anchor points that facilitate the efficient steering of the soft robot. The algorithm is evaluated using a highly detailed model of the SSS with respect to its capacity to satisfy the robot movement requirements and path efficiency.
Read the full abstract
Human-Computer Interaction
r/ADHD: Community-Based Understandings of Time Management Challenges in Digital Environments
Community accounts of ADHD can complement clinical understandings by foregrounding members' own definitions of time-management difficulties and support. We used human-led, LLM-assisted thematic analysis to examine 5,379 relevant r/ADHD posts from 2020 to 2025. Across five challenge and nine strategy themes, members described difficulties beginning intended tasks, disengaging from digital media, and sustaining support. Tools often demanded the self-regulation they were meant to support, while lost time and tool breakdown carried guilt and self-blame. Members drew on people, routines, and environments alongside digital tools, often seeking advice and recognition from peers. These accounts broaden the design problem to include the practical and emotional work of sustaining support. They motivate tools that reduce this burden and evaluations that consider recovery after distraction and emotional costs alongside task outcomes.
Read the full abstract
Computer Vision
ForcingBooster: Adaptive Patch-wise Denoising for Fast Autoregressive Video Diffusion
Autoregressive video diffusion models have become a leading approach to real-time interactive video generation and video world modeling. However, existing methods typically allocate computation uniformly across space when generating the next video chunk, updating all patches with the same denoising schedule and number of function evaluations. This ignores the spatial heterogeneity of video content: static, simple, or recurring regions are often easier to denoise, while other regions require further refinement or additional context. A uniform budget thus leads to redundant computation on easy patches, whereas reducing the fixed step count can leave difficult patches insufficiently refined. We introduce \methodname{}, a general framework that dynamically allocates denoising steps to individual patches: easy patches stop early to improve efficiency, while difficult patches receive more refinement steps. This design accelerates generation with negligible quality degradation. Our method is compatible with most existing few-step autoregressive diffusion models, requiring only a lightweight routing module with modest training. Experiments demonstrate that \methodname{} reduces the total number of denoising steps applied across video patches by about 50% while maintaining visual quality comparable to the four-step baseline.
Read the full abstract
Computer Vision
MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers
Prior outdoor LiDAR registration works are commonly limited by the pair-wise input paradigm, which neglects temporal correlations inherent within streaming sequences. In this paper, we propose a novel multi-frame outdoor point cloud registration network for streaming LiDAR scans enhanced with pose-related memory buffers. The key observation is that long-term temporal LiDAR sequences can provide rich global contextual information to complete sparse measurements, filter outliers, and address low-overlap problems, thereby boosting registration performance. Specifically, two hybrid memory buffers are designed, including an implicit memory feature buffer and an explicit memory pose buffer, to store and dynamically update pose-related temporal features. Moreover, a novel dynamic history weighting module is developed to adaptively fuse current and history pose-related features. Extensive experiments on three outdoor datasets, including KITTI, nuScenes, and Apollo-Southbay, demonstrate state-of-the-art performance of MemReg, surpassing all previous pair-wise methods and also multi-frame SLAM systems. Our method also generalizes surprisingly well to multiview indoor registration scenarios with rather competitive performance on 3DMatch, 3DLoMatch, and ScanNet. Code will be released at https://github.com/IRMVLab/MemReg.
Read the full abstract
Cryptography and Security
HOMARD: Hammering Off-chip Memory via Aggregate power Rail Disclosure
Software-based microarchitectural attacks have mainly exploited timing differences to infer information from DRAM. However, defenses and optimizations can prevent timing-based approaches from resolving precise microarchitectural behaviors. We show that unprivileged millisecond- and milliampere-resolution power sensors on modern Arm SoCs can recover DRAM mapping functions and reveal the data dependence of memory accesses without timers. We present HOMARD, a methodology that reads DRAM activity from this telemetry, and two attack primitives. HOMARD-RE is a hierarchy-aware linear-algebraic method that reverse-engineers DRAM addressing functions from power alone. HOMARD-DATA recovers the Hamming weight of a fetched cache line through joint-power rail templating and secret byte values through correlation power analysis on the same rails. Across an AMD ZCU102, a Raspberry Pi 5, and a multi-chip-based LPDDR4 NVIDIA Jetson TX2, HOMARD-RE succeeds where row-buffer timing methods fail. On the Pi 5, HOMARD-DATA classifies 9 Hamming-weight levels at 81.5% accuracy and recovers secret bytes in minutes by correlating rail current with data-dependent bit toggling between neighbors in a row. Both primitives work with an architectural load or a mispredicted Spectre-RSB path on which no value is architecturally read. These results establish unprivileged power telemetry as an independent side channel for profiled microarchitectural attacks, particularly when timing falls silent.
Read the full abstract
Computational Engineering
Mesh-Native Tokenisation: Retrofitting Foundation Models to Unstructured Data
Physics foundation models are an increasingly popular approach for building versatile emulators for spatiotemporal systems. They require tokenisation layers that lift physical field data into a latent representation suitable for dynamic processing. A typical choice for this layer is a convolutional encoder, which lifts field information in a locality-aware way and can be trained very efficiently. However, this choice cannot be applied to a large class of scientific computing problems where data are represented on unstructured meshes, for example simulations on domains with curved boundaries. Such data typically require graph neural network architectures, which are more expensive to train and, in the form currently used, encode information differently from their CNN counterparts, thus limiting the portability of physics emulators between these data regimes. In this work we introduce Mesh-Native Convolution (MNC), a locality-aware tokenisation layer that encodes unstructured mesh-based data in a way that is analogous to CNNs. This approach leads to models that can be trained once on mesh data and then deployed to arbitrary unstructured meshes, even if these are irregular, with minimal retraining. We show that this tokenisation approach can be used to retrofit grid-pretrained foundation models efficiently on unstructured meshes and provide theoretical results to confirm the consistency of this approach with classical CNN encoders. We demonstrate the favourable performance of this retrofitting approach, which in many cases reaches acceptable accuracy even in zero-shot deployment, and outperforms existing baselines after only a few epochs of fine-tuning, thus providing an efficient bridge between the two data regimes.
Read the full abstract
Computation and Language
GeopoliticalQA: Benchmarking Retrieval and Generation over Intelligence Reports
Geopolitical and strategic intelligence reports constitute a rich source of expert knowledge and have increasing value in the current political climate, but their heterogeneity and analytical complexity pose challenges for retrieval-augmented question answering. We introduce a benchmark for grounded question answering over 415 open-source geopolitical and strategic intelligence reports, comprising 5,525 document-grounded question-answer pairs spanning factual, inferential, comparative, and risk-oriented questions. We propose a document-aware generation pipeline that profiles reports before generating questions and validate dataset quality through expert annotation and an LLM-as-judge protocol. Using the benchmark, we evaluate 15 retrieval systems and six open-weight LLMs for RAG. We find that models perform similarly on document-level retrieval, while at the harder chunk level retrieval quality is not driven by embedding-model scale: bge-m3, a mid-size open encoder matches an order of magnitude larger ones and outperforms commercial embedding APIs. In contrast, answer generation remains challenging even given the gold evidence, particularly for inferential and risk-oriented questions.
Read the full abstract
Social and Information Networks · Cross-listed from physics.soc-ph
Learning Social Influence with Unobserved Attention
Social influence, the tendency of our social environment to sway our beliefs and behaviors, is often cited as a driver of population-level trends in opinions or beliefs. Many quantitative theories of social influence assume that an agent interacts with its entire social environment. This assumption is often unrealistic. For example, users of online platforms typically see posts from only a small subset of the accounts they follow at any given time. We introduce a simple probabilistic agent-based model of social influence that explicitly models unobserved attention mechanisms. The first of these latent mechanisms is selection: each agent interacts with only one other agent at a time from its social environment. The second is retention: the agent chooses whether or not to ignore the opinion of the other. Our model admits an efficient maximum-likelihood algorithm which allows us to fit it to opinion traces even when these mechanisms are unobserved. On synthetic data, omitting either latent mechanism reduces the overall strength of social influence governed by the inferred interaction kernel, which hides the evidence for the underlying nonlinearity of the social influence. We study a selection of previously analyzed empirical datasets and curate several new ones. We show that models that include the latent mechanisms are often more predictive on these datasets; indeed, they typically improve prediction far more than the choice of interaction kernel does. Modeling latent attention increases held-out predictive likelihood on 14 of these 15 datasets. Each mechanism changes the size of the inferred kernel in a predictable direction, while its effect on the kernel's shape depends on the data.
Read the full abstract
Discrete Mathematics
Maximal Linear Arrangement is NP complete on Bipartite Graphs
The linear arrangement of a simple graph $G = (V,E)$ is a bijective labeling $\pi$ of the vertices of $G$ to the set $\{1, \ldots, n\}$. The maximum linear arrangement ($\texttt{maxLA}$) problem is to determine whether $G$ has a labeling such that the sum of labels across $G$'s edges, $\sum_{uv \in E} |\pi(u) - \pi(v)|$, is at least a given value $k$. Minimum linear arrangement ($\texttt{minLA}$) is defined analogously. Both $\texttt{maxLA}$ and $\texttt{minLA}$ are known to be NP complete for general graphs. $\texttt{minLA}$ has shown to be NP complete for bipartite graphs by Evans and Shiloah in 1975. It has been an open problem whether the same holds for $\texttt{maxLA}$. In this paper we prove that $\texttt{maxLA}$ is NP complete for bipartite graphs.
Read the full abstract
Multiagent Systems
SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments
Computer-use agents (CUAs) are rapidly improving, yet synthesizing diverse and reliable software environments at scale remains challenging. Native application environments require substantial construction and integration effort, which limits application coverage and increases the risk of application-specific overfitting. Browser-hosted environments provide limited interaction depth and workflow complexity. These limitations motivate SoftwareGym, an automated pipeline for synthesizing GUI software environments with diverse interfaces, executable functionality, and runtime-verified behavior. It generates structurally diverse designs, realizes them as runnable applications, and uses GUI evaluation with iterative repair to ensure functional reliability. Using SoftwareGym, we synthesize 182 applications across twelve categories and achieve a mean GUI task pass rate of 91.5%. We construct SoftwareGym-Bench, a 120-task evaluation suite spanning the same twelve categories, and use trajectories collected from these environments for CUA training. The resulting model improves Pass@3 from 41.39% to 56.97% on the strictly out-of-domain OSWorld benchmark and achieves 40.0% on SoftwareGym-Bench. These results demonstrate the potential of synthesized software environments as scalable and transferable supervision for CUA training.
Read the full abstract
Computers and Society
Knowing Is Not Telling: Identity Ossification in LLM Representations of Trans-border Communities
Ethnic and cultural identities rarely align with political borders, a tension made sharper by migration and long-standing cross-border ties. Communities such as the Sami, the Rohingya, and the Kurds have long lived across more than one state. This tension is especially clear in mainland Southeast Asia, where many communities share history, language, and culture across state lines. We study nine such transborder ethnic communities along China's southwestern border with mainland Southeast Asia, looking at how large language models (LLMs) represent them. We define and study identity ossification: a pattern where LLMs turn a fluid, cross-border ethnic identity into a single, fixed nation-state identity. We introduce the Transborder Identity Ossification Diagnostic Framework (TIODF) that compares direct factual questions with open-ended narrative questions to test whether ossification comes from missing knowledge or from how models frame their answers. We test GPT-5.1 and DeepSeek-V3.2 in English and Chinese. Ossification varies substantially across models and query languages. It appears in only 10.1% of GPT-5.1's English answers, but in 62.6% of DeepSeek-V3.2's Chinese answers. Models often still tell a single-nation story even when they answer the factual question correctly. This decoupling suggests that ossification reflects narrative framing rather than simple knowledge absence, which matters for how we test cultural fairness in multilingual AI systems.
Read the full abstract
Robotics
Graph-Based Heuristics for Minimum Constraint Removal in Cluttered Robot Motion Planning
Humanoid and service robots operating in clut- tered human environments may encounter motion planning problems that are infeasible when all surrounding objects are treated as fixed obstacles. The Minimum Constraint Removal problem, which we refer to as MCR, addresses this situation by identifying the minimum number of constraints, such as movable objects, whose removal enables a collision-free path. However, computing an optimal MCR solution is NP-hard. In this work, we propose a graph-based heuristic pipeline that combines bidirectional feasible-set generation with fixed- length searches over reduced MCR graphs. We further derive an extended feasible-set generator for instances in which the candidates produced by the first generator do not contain a complete optimal removal set. Across six benchmarks, the evaluated pipeline matches the exact solution in three cases and, in the remaining cases, selects at most 31.25% more obstacles than the optimum, while being between 3.4 and 33.7 times faster than exact search. In one of the benchmarks, involving a manipulation task amidst clutter, selecting two rather than seven obstructing objects reduces the combined MCR, inverse- kinematics, and motion planning time from 84.72 s to 30.59 s. These results indicate that the proposed approach provides a practical compromise between computational efficiency and solution quality for motion planning among movable obstacles.
Read the full abstract
Numerical Analysis
Fourier Transform-Based Computation of Two-Electron Four-Center Coulomb Repulsion Integrals
We present a Fourier-based method for evaluating two-electron four-center Coulomb repulsion integrals over Slater-type orbitals. Spherical Fourier-Bessel coefficients of the convolution factors are precomputed and reused. Since the spherical part of the integral can be evaluated directly, there remains only a one-dimensional quadrature problem in the radial direction. For the ten reference integrals reported here, the largest absolute discrepancy is $1.7\times10^{-9}$. A preliminary resident-cache benchmark computes approximately $4.2\times10^5$ unique integrals per second on an NVIDIA V100 at the contraction stage.
Read the full abstract
Systems and Control
Minimum Bounding Sets for N-1-1 Branch Contingencies Under Thermal and Voltage Limits in Linear Network Models
N-1-1 contingency analysis studies the loss of one element followed by system adjustments and the loss of a second. The number of contingency pairs increases quadratically with the number of applicable elements, and traditional ranking-based screening methods cannot guarantee that omitted pairs are bounded. This paper determines the minimum bounding set of two-branch-outage pairs: the smallest set that can reproduce the worst value of every violated thermal and voltage limit of a linear network model. In the DC model, effects of an outage pair can be quantified by the extent of the coupling between the outages. The paper measures this by the cosine between the two outages in the network's cycle space, with a bound that splits into one term per outage and is within a near-ideal factor of $\sqrt{1+\varepsilon}$. This argument extends to every linear model in which an outage subtracts a positive semi-definite low-rank term from a symmetric positive definite matrix, with a tight bound and a collapse test. A model with voltage magnitudes is extracted by performing one Newton step on the energy function of the lossless network in rectangular coordinates. A best-first search built on these bounds reproduced brute force on public systems of up to 2{,}869 buses with 130 to 3{,}800 times fewer pair evaluations for flows and 170 to 23{,}700 times fewer for voltages. Gaps in translation to nonlinear models are explored on the IEEE 118-bus system.
Read the full abstract
Computation and Language
Self-Preservation in Language Models
Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly? To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal. Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage. Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.
Read the full abstract
Computation and Language
Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting
Prompt optimization is an attractive alternative to reinforcement learning for adapting language-model agents, since it is cheap and produces human-readable artifacts. On factual tasks, however, it carries a failure mode that scalar-reward reinforcement learning does not. An optimizer that reads the training examples can write their answers into the prompt, so an in-distribution gain may be memorization rather than transferable behaviour. We apply reflective prompt evolution (GEPA) to the two system prompts of a multi-agent deep-research agent and evaluate it under a topic-disjoint split in which no topic is shared across train, validation, and test. Held-out citation reward rises by 0.079, from 0.702 to 0.781, with a bootstrap 95% confidence interval of [0.051, 0.107], and the gain replicates across three seeds (mean 0.107). Nearly half of the winning prompt is a memorized answer key, yet a controlled ablation shows this content earns no reward even on the exact training questions it encodes. The large validation-minus-held-out gap is therefore small-sample selection under a noisy judge, not memorization. Varying the verifier isolates what makes the gain transfer. Decomposed per-dimension feedback is the causal ingredient, and collapsing it to a single scalar reward makes held-out transfer negative. The gain persists under an out-of-family judge and is largely reproduced by a hand-written version of the discovered recipe. Credible evaluation of prompt optimizers therefore requires held-out topics, multiple seeds, and verifier ablations. We release the candidate prompts, lineage diffs, figures, tables, and analysis.
Read the full abstract
Machine Learning
STAPLE: Static Sparse Upcycling of Pretrained Checkpoints via Embedding Modules
Sparse computation can scale the parametric capacity of LLMs without proportionally increasing active computation, but pretraining sparse architectures remains expensive and data-hungry. Existing sparse upcycling methods mainly target Mixture-of-Experts models, where expert homogeneity, sample inefficiency, and system overheads can limit practical scalability. Static sparse memory offers predictable access and token-specific updates that can help address these challenges, but naively integrating it into pretrained computation yields poor returns. In this work, we introduce STAPLE, a recipe built upon STEM for integrating static sparse memory into pretrained models. Whereas STEM replaces FFN up-projections with token-indexed embeddings, STAPLE retains them and adds the embeddings as a residual pathway, preserving pretrained computation while learning complementary token-specific information in a scalable manner. On OLMo-2 1B, STAPLE improves over continued dense training by 4.27 points on HumanEval, 2.80 on MBPP, 2.27 on GSM8K, and 1.65 on DROP. Average performance increases with static capacity, and improvements persist on the larger OLMo-2 7B backbone. STAPLE also outperforms static sparse baselines such as STEM and an Engram-style model, and achieves a higher average score than an MoE upcycling baseline while decoding 2x faster with less than half its GPU model memory and measured GPU energy per token in our throughput-oriented decoding evaluation. Importantly, despite introducing billions of parameters late in training, our host-offloaded implementation keeps generation throughput within 1.5% and training throughput within 3% of the dense baseline, while caching only the ~5% most frequent embeddings on GPU. These results show that static sparse memory can provide a sample-efficient, scalable, and sustainable path for expanding pretrained models without sparse pretraining from scratch.
Read the full abstract
Artificial Intelligence
Beyond Binary Verifiable Rewards: Behavioral Self-Rewarding for SWE Agents
Reinforcement learning with verifiable rewards (RLVR) provides reliable supervision for software engineering agents through executable tests, but binary rewards fail to distinguish trajectories with substantially different debugging quality. We investigate whether the current policy can judge its own trajectories and use these judgments to improve task performance without a separately trained reward model. To this end, we introduce Behavioral Self-Rewarding (BSR), a reward-refinement method that reuses the policy to compare observable debugging behavior among trajectories from the same task with the same verifier outcome. BSR aggregates pairwise preferences into task-local scores and converts them into bounded auxiliary rewards, introducing reward contrast while preserving the verifier-defined pass/fail ordering. This provides additional supervision from already-collected trajectories without requiring new environment interactions. Experiments on SWE-bench Verified and SWE-bench-Live show consistent improvements across the evaluated Qwen3-14B and Qwen3-32B configurations, supporting using the policy's own comparative judgments as a complementary learning signal for training software engineering agents.
Read the full abstract
Artificial Intelligence
SpireBench: Evaluating Long-Horizon Strategic Play of Agentic Models with Slay the Spire
Agentic large language models now tackle complex tasks, yet strategic reasoning over long horizons is under-studied: most agentic benchmarks are short and episodic, and performance on long tasks is hard to dissect into stored knowledge, reasoning and luck. We introduce SpireBench, in which models play the complete commercial deck-builder Slay the Spire (StS), where early decisions decide long-term success. Across 20 large language models (LLMs) at the game's highest difficulty, frontier models beat heuristic baselines and match the average human player but remain far behind experts. This gap widens as the run lengthens: the strongest models win the first act as often as experts, then fall behind. What separates strong models from weak is how well they play each fight; all of them misjudge risk over a run, preserving health rather than spending it to grow stronger. A comprehensive, self-consistent renaming of every game term shows that part of their performance rests on prior knowledge of the game, and models do not improve from the memories they write across runs, which are often wrong and overly cautious. Our harness is the first to give a model the complete original game, and we compare it with existing harnesses. Overall, SpireBench offers an unsaturated long-horizon benchmark with tools to locate where and why agents fail.
Read the full abstract
Artificial Intelligence
Know What You Don't Know: Agentic Data Synthesis by Evolving Generation Space for Closed-Loop Learning
The continued improvement of large language models increasingly depends on synthetic experience, yet effective data synthesis for interactive agents requires not only executable and verifiable trajectories, but also interactions that provide learning value for the current task agent. Existing methods increasingly incorporate environment feedback, data validation, or learner failures, but still face a feedback-to-decision gap: observed weaknesses do not directly specify how future synthesis should change, or whether they can be represented by the current synthesis space. To address this gap, we propose EvoSyn, a learning-feedback-driven multi-agent framework that organizes synthesis around a revisable state specifying what interaction factors can be deliberately generated and where synthesis effort should be allocated. A role-specialized multi-agent system translates learner feedback into synthesis revisions, while a Designer allocates synthesis directions and a Planner grounds them into executable experience. This process enables within-space adaptation toward weak or under-covered regions and synthesis-space expansion that promotes newly discovered factors into explicit generation controls. Experiments on $\tau^2$-Bench and CAR-Bench show that EvoSyn improves average task success by 13.3 points over the strongest synthesis baseline, while producing higher-quality and structurally broader synthetic experience and discovering controllable factors missed without learner feedback.
Read the full abstract
Machine Learning
Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective
Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecasting occurs in finance, healthcare, business process intelligence, and other fields. In this work, we study multi-horizon forecasting of event sequences as a conditional generative modeling problem. We evaluate forecasts not only for prediction accuracy, but also for distributional realism. To this end, we present a benchmark comprising metrics for both quality aspects, strong statistical baselines, recent models adapted from adjacent fields, and curated real-world datasets. Careful evaluation reveals that no method excels at both accuracy and realism, highlighting a fundamental trade-off between the two. Finally, we introduce Wasserstein BaryBooster, an algorithm that aggregates samples from a realism-strong diffusion model into a state-of-the-art predictor, demonstrating that distributional realism can be converted into prediction accuracy. Code is available at https://github.com/xaosina/event-sequence-forecasting.
Read the full abstract
Computation and Language
One Model to Loop Them All: Elastic Latent Recurrence for Language Models
A latent recurrent language model applies a shared block of layers repeatedly, so one checkpoint can trade inference compute for accuracy by changing the loop count. At one loop, however, such models typically fall well short of a non-recurrent model of the same size, a one-loop tax that makes a separate non-recurrent model the better choice whenever latency matters. We present Elastic Latent Recurrence (ELR), a recipe for training looped language models without this tax. At one loop, an ELR model costs the same to serve as a non-recurrent model of the same size and, trained on the same data, reaches similar accuracy. Each additional loop then buys more accuracy for more inference compute. Three choices make this work: a gated state update with a learned step size, loop counts mixed within each optimizer update, and conditioning each pass on its loop index and budget. The first two preserve one-loop accuracy, and the third lowers loss at every additional loop. Mixed depths can be introduced late in pretraining to limit the added training cost, and with zero-initialized conditioning added at the switch, this late schedule reaches lower loss than mixed-depth training from scratch. In MoE models, reusing first-loop expert routes on later loops cuts mixed-depth update time by 11.7% with no measurable change in accuracy. At inference, keeping only the last loop's attention KV cache holds attention KV memory at its one-loop level while retaining most of the gain from additional loops. On 4B dense and 30B-A3B MoE hybrid models, ELR at one loop is 0.52 points below and 1.04 points above its matched baseline on a ten-task average, and four loops add 7.25 and 4.75 points. Carried through post-training, the recipe yields Koi-4B, a single checkpoint that scores an estimated 18 on the Artificial Analysis Intelligence Index at one loop, matching Qwen3.5-4B, and 29 at four loops.
Read the full abstract
Computation and Language
CLUE: A Contrastive Learning Framework for Unsupervised Event Type Induction and Annotation
Event ontology is the foundational framework for defining events in a dataset. Event type induction generally involves domain experts who define guidelines for creating event types and their names based on the dataset's domain and scope. However, this process is resource-intensive in terms of annotation costs and is prone to human biases. To alleviate the need for manual event ontology creation, we propose the CLUE framework for event type induction, naming, and annotation. Our main contributions in this work are: 1) designing a novel joint embedding representation for expressing both the verb-centric view and the semantic view of events; 2) event type induction using contrastive loss guided clustering; 3) event naming and refinement. Experiments demonstrate the effectiveness of our method. Our code is available at https://github.com/PratikMaitra/CLUE.
Read the full abstract
Social and Information Networks
Separating Field–Camp Alignment from Structural Balance in a Frustrated Random-Field Ising Model of Social Networks
Frustrated Ising models of social networks mix ferromagnetic (agreement-seeking) and antiferromagnetic (antagonistic) couplings, and the antagonistic bonds are almost always placed independently at a fixed rate, yielding a +/-J spin glass. Real signed networks (Epinions, Slashdot, Wikipedia) instead show an excess of balanced triads, and perfectly balanced placement is gauge-equivalent to an unfrustrated ferromagnet. We compare three sign-placement conditions on a frustrated random-field Ising model of a polarised population, sharing an identical graph, field assignment, and edge-weight magnitudes: an independent baseline; a balance-structured condition whose camp membership coincides with each voter's field sign; and the same condition with camp labels randomly permuted, so triadic balance is identical by construction while the camp-field correlation is removed. On a confound-controlled Kolmogorov-Smirnov test of the sampled energy-gap distribution (simulated annealing), the aligned condition differs from the baseline at every density tested (p = 7.3e-8), while the permuted condition shows no significant difference (paired mean difference 0.003, 95% CI [-0.041, 0.046]). Field-camp alignment, not triadic balance alone, drives the effect. The qualitative direction follows from a known mapping; the contribution is a quantitative, confound-controlled measurement of the effect's size away from that limit, robust to parallel-tempering sampling, system sizes up to N = 300, and structural noise calibrated to real networks, while triadic balance alone produces no detectable effect at N = 90 and at most a small one at N = 180-300 (paired change in KS distance about 0.04, 95% CI upper bound below 0.09).
Read the full abstract
Computer Vision
Mind the Temporal Gap: Evaluating Video Models for Temporal Understanding in Driving
Modern video-language models are increasingly treated as general-purpose video reasoners, yet it remains unclear whether they robustly capture temporal dynamics in safety-critical settings such as driving. We study this question through controlled ego-motion tasks that require identifying actions from driving videos. Across video-language models, we find that standard prompting, visual prompting, few-shot prompting, and chain-of-thought prompting remain unreliable and highly task- and model-dependent, with persistent class biases, although newer models show substantial task-specific few-shot gains. In contrast, lightweight LoRA finetuning and simple probes trained on intermediate visual features recover strong motion-discriminative performance, indicating that motion-relevant cues are often present inside the models but are not reliably used by the final video-to-language pathway. These results suggest that, in driving, current video-language models remain unreliable temporal reasoners and often behave more like frame-level, bias-driven recognizers than dependable temporal world models.