Machine Learning
Fine Until Fine-Tuned: Repeated Solutions Make Reasoning Fragile
Ely Sheikh
Recipes such as s1 and LIMO teach a model to reason with little data by showing it the same thousand or fewer worked solutions many times over. Judged when that training ends, the repetition looks harmless. But
reasoning models are often trained again, and we find that repetition leaves their
reasoning fragile to that next stage, even when the stage has nothing to do with
reasoning. We fine-tuned Qwen3.5-9B-Base on its own correct solutions to competition math problems, either drilling a few hundred of them about eight times each or showing many more once; with the same amount of training, both solve about 95% of held-out problems. A single pass of ordinary instruction tuning leaves the once-trained model where it was, while the drilled one falls to 86.0%, and harsher later stages take it to 59.3% or below. A third model that visited the drilled problems just as often, with a new solution at every visit, was unharmed, so the damage comes from seeing the same texts again rather than from having few problems. The break recurs with a stronger model's traces, in further training runs and on other models and tasks. It is also cheap to undo: the
reasoning is suppressed rather than erased, and five updates of
reasoning training bring almost all of it back, as does brief training on the
reasoning format with almost no mathematics. Fresh solutions prevented the damage, and so did replaying 6.25% of the original solutions in a gentler later stage, so our claim concerns later training without such replay. Sharpening alone does not explain the break, since a model sharpened three-quarters as much without repetition was unharmed. On a skill the base model could not perform within a
token budget, repetition mainly cost learning.
Read the full abstract ▾
Computer Vision
PGL-3D: Towards Progressive Geometric Learning for 3D Visual Query Localization
Liang Peng, Shizhuo Mu, Bohan Tan et al. (+6 more)
3D Visual Query Localization (3DVQL) retrieves the latest contiguous occurrence of a queried object in an RGB–point-cloud sequence and predicts a 9-DoF cuboid for every response frame. The query is captured independently of the search sequence, so its annotated pose may differ from how the object appears in the search frames. The
benchmark baseline predicts cuboids after feature modeling, leaving their geometry unused for subsequent feature refinement. We investigate whether complete intermediate cuboids can improve query and proposal representations before final decoding. We introduce Progressive
Geometric Learning for 3DVQL (PGL-3D), a predict–select–refine–re-predict framework that uses intermediate cuboids to guide the aggregation of search evidence and update query and proposal representations. A shared head first predicts a complete cuboid for every proposal. Query–Tube–
Memory (QTM) then selects reference observations by combining proposal association, cuboid quality, frame response, and target absence, since association confidence alone establishes neither target presence nor
geometric accuracy. The center, size, and orientation of each selected cuboid define soft pooling weights over query-conditioned proposal features. The pooled
memory updates the query and proposal representations, and the head re-predicts from the updated features. A training-only objective, ST-D9O, supervises cuboid geometry at every stage by adding boundary, signed-distance, and soft-overlap terms to parameter regression. PGL-3D achieves a mean stAP of $0.270 \pm 0.004$ on 3DVQL, compared with $0.044$ reported for LaF. Ablations support the benefits of geometry-guided feature updates, while stage-wise analyses show improved cuboid accuracy. Replacing the geometry objective in our PROT3D reproduction with ST-D9O improves mAO on GSOT3D from $21.63\%$ to $25.78\%$. Our code and models will be released.
Read the full abstract ▾
All authors Liang Peng, Shizhuo Mu, Bohan Tan, Wenyuan Wang, Chen Zhao, Xingping Dong, Heng Fan, Libo Zhang, Bo Du
Sound · Cross-listed from eess.AS
An Efficient Parametric Codec for Low-Bitrate First-Order Ambisonics
Wei-Ting Lai, Amy Bastine, Lachlan Birnie et al. (+2 more)
Driven by the rapid growth of immersive teleconferencing and generative spatial audio, efficient low-bitrate coding of first-order Ambisonics (FOA) has become increasingly important. In this work, we develop a lightweight parametric FOA codec that retains the standard Directional Audio Coding analysis and synthesis while redesigning the spatial metadata quantization scheme. Rather than quantizing direction-of-arrival (DOA) and diffuseness independently, we combine them into a 3-D directivity vector and jointly quantize these vectors across frequency bands via residual vector quantization (RVQ). The RVQ codebooks are optimized within minutes via stage-wise k-means without backpropagation, yielding a constant-bitrate (CBR) representation that decouples metadata rate from the number of frequency bands. Evaluations show that our approach outperforms low-bitrate perceptual codecs in FOA reconstruction, remains competitive with neural codecs on the downstream Sound Event Localization and Detection task, and maintains robust performance when paired with different external monaural codecs. Given its lightweight training, strong performance, and CBR design, we consider the proposed method to be a favorable and reproducible baseline for future FOA codec research.
Read the full abstract ▾
All authors Wei-Ting Lai, Amy Bastine, Lachlan Birnie, Thushara D. Abhayapala, Prasanga N. Samarasinghe
Systems and Control
Grid-Forming E-STATCOMs for Stable Integration of Large-Scale Data Centers: Modeling and Control
Prabhat Ranjan Bana, Novan Zakkia, Jean-Philippe Hasler et al. (+1 more)
The rapid expansion of large-scale AI data centers (AIDC) is introducing new stability challenges, particularly in weak or low-inertia networks characterized by fast, step-like demand variations and strict requirements on voltage and dynamic performance. This paper investigates the use of grid-forming (GFM) Enhanced STATCOMs (E-STATCOMs) to support reliable integration of such facilities. A power-admittance-based linear modelling framework is developed to capture system interactions and is validated through detailed EMT simulations. The results demonstrate that E-STATCOMs provide fast, well-damped responses to abrupt load changes while effectively mitigating low-frequency oscillations and interactions with network resonances. By enabling tunable dynamic behavior via a load balancer, virtual impedance, and coordinated active-reactive power support, the proposed approach allows precise shaping of system response and improved regulation at the point of connection. These features make E-STATCOMs a flexible and scalable solution for integrating large data centers into weak grids and long transmission systems, supported by a design-oriented framework that facilitates parameter selection and performance assessment without extensive reliance on EMT studies to meet grid codes and AIDC interconnection requirements.
Read the full abstract ▾
All authors Prabhat Ranjan Bana, Novan Zakkia, Jean-Philippe Hasler, Christer Danielsson
Robotics
FoLD: Force-Informed Learning for Dexterous Articulated Object Manipulation
Haowei Shen, Tingai Li, Yumeng Liu et al. (+6 more)
Transferring human demonstrations to dexterous
robots remains challenging because differences in hand morphology and contact dynamics often cause retargeted motions to fail at producing the intended object behavior. We present FoLD, a framework for learning dexterous manipulation of articulated objects through explicit force guidance. FoLD compute compensatory force fields from human demonstrations together with the
robot's current interaction state, yielding a force prior that promotes the demonstrated object motion. This force prior informs a residual policy that adapts retargeted hand motions to the contact requirements of the task. We evaluate FoLD on a public
benchmark for articulated object manipulation, where it consistently outperforms state-of-the-art baselines across tasks and embodiments. We further validate FoLD on real dexterous
robot platforms, demonstrating successful transfer of human manipulation skills to
robot execution. Here is the link of our project page: https://gghgghgghgg.github.io/FoLD-project-page/.
Read the full abstract ▾
All authors Haowei Shen, Tingai Li, Yumeng Liu, Wenyuan Guang, Xuanze Yang, Qing Fang, Kai Xu, Ligang Liu, Ruizhen Hu
Machine Learning · Cross-listed from stat.ML
Sparsity by Default: The Theory and Practice of ARD in Gaussian Process Regression for Variable Selection
Jia Cai
Automatic relevance determination (ARD) is the standard device for input selection in Gaussian process (GP) regression. By giving the covariance kernel a separate lengthscale for every input and learning those lengthscales by maximizing the marginal likelihood, ARD lets the data decide which coordinates matter: irrelevant inputs receive very large lengthscales and are effectively switched off. We trace this mechanism to the
Bayesian Occam's razor
embodied in the marginal likelihood, derive the gradient through which it prunes inputs, and emphasize that ARD delivers effective rather than exact sparsity. We review the
algorithms used in practice and the rules that turn lengthscales into selections, and we survey the asymptotic theory, distinguishing the fixed-domain identifiability obstruction on the lengthscales from the high-dimensional selection-consistency guarantees recently established for hierarchical GP priors, and noting what remains open for plain ARD. We compare ARD with spike-and-slab priors, sparse axis-aligned and global-local shrinkage priors including the
Bayesian lasso and horseshoe, penalized likelihood kriging, sensitivity and projection criteria, and additive kernels. We argue that ARD endures because of its seamless integration with kernel learning, universal software support, and low cost, and we close with its limitations and remedies.
Read the full abstract ▾
Machine Learning · Cross-listed from stat.ML
Calibrated Derivative-Process Sensitivity for Gaussian-Process Variable Selection
Jia Cai
Automatic relevance determination (ARD), the default tool for variable selection in Gaussian-process (GP) regression, ranks inputs by inverse lengthscales – which measure how fast a function varies, not how much an input contributes to prediction – and offers no calibrated rule for deciding which inputs to keep. The prediction-centred alternative, the derivative sensitivity $\nu_j = \mathbb{E}[(\partial f/\partial x_j)^2]$, is available in closed form from a fitted GP, but turning it into a selection rule is harder than it looks: at a null input the
estimator is a degenerate quadratic form, so Wald and Bernstein-von Mises cutoffs are anti-conservative, and the natural residual bootstrap is mis-scaled. We show that a studentized multiplier bootstrap of the GP derivative process repairs both, prove its validity through an invariance principle for quadratic forms, and obtain asymptotic family-wise and false-discovery-rate control across inputs. Over 100 replications the rule controls FDR wherever inputs are truly null, while uncalibrated derivative rankings breach the target by up to 2x and a Bernstein-von Mises cutoff by 2.2x; at matched FDR it loses no power; it holds under a Matérn kernel and input correlation up to 0.99; on real data with planted and authentic null inputs it admits 5-12x fewer spurious inputs; it costs 5-18% of the GP fit; and a block-averaged variant retains validity at cost linear in $n$.
Read the full abstract ▾
Machine Learning
TerMeZO: Ternary Sparse Zeroth-Order Optimization for Fine-tuning BitNet Models at the Edge
Houssem Sifaou, Prabodh Katti, Bipin Rajendran et al. (+1 more)
Fine-tuning anguage models (LLMs) with first-order optimizers requires a
memory several times larger than that required for inference.
Memory-efficient zeroth-order optimization (MeZO) sidesteps this cost by estimating gradients from forward passes only. However, for BitNet architectures, a family of LLMs with ternary {-1,0,1\} weights and 8-bit activations, fine-tuning requires updating full-precision latent weights, and thus the
memory footprint of MeZO no longer matches that of inference. A promising solution is to finetune only a subset of the latent weights, but existing sparse zeroth-order (ZO) methods either ignore the ternary structure or require first-order gradient information to build a sparse mask, which is at odds with the purpose of ZO fine-tuning. We propose TerMeZO, a sparse MeZO scheme that exploits the geometry of the ternary quantizer itself to identify the latent weights that are more likely to change values during fine-tuning, at no additional data or
memory cost. Our convergence analysis shows that TerMeZO can converge faster than full-parameter MeZO, owing to its optimized reduction of the fine-tuning effective dimension. We run extensive experiments on BitNet models ranging from 1B to 3B parameters, spanning classification, instruction-following, and mathematical
reasoning tasks. TerMeZO matches or exceeds the performance of full-parameter MeZO while substantially reducing the fine-tuning
memory footprint.
Read the full abstract ▾
All authors Houssem Sifaou, Prabodh Katti, Bipin Rajendran, Osvaldo Simeone
Software Engineering
Neuro-Symbolic Indirect-Call Analysis under Opaque Pointers
Kaixuan Li, Bozhi Wu, Jian Zhang et al. (+3 more)
Resolving indirect calls is central to call-graph construction for C. Scalable type-based analyses such as MLTA use type information in LLVM IR to associate indirect calls with functions assigned to the corresponding structure fields. However, a single pointee type often misrepresents the
memory a pointer addresses, and LLVM 17 removed pointee types in favor of opaque pointers. Therefore, field-sensitive analyses lose their matching key. Recovering the erased types restores the matching key but still misses the relation that the type encoded: which functions the program assigns to the field. We present Facet, to our knowledge the first analysis that reconstructs this dispatch relation over opaque IR. Facet identifies the structure field from which an indirect call loads its function pointer. It separately recovers the functions assigned to that field through initializers, stores, and aggregate copies. It then joins the two by field identity, without requiring an end-to-end value-flow path. Facet classifies proposed call-graph changes under distinct evidence rules for edge addition and removal and records the assumption behind each refinement. An LLM decides only the residual cases among symbolically bounded candidates. One analysis yields both a recall-preserving call graph and a refined call graph. On 14 C programs, Facet reduces the mean target-set size from 25.9 to 5.2 and raises observed recall from 0.79 to 0.99. Its recovered field identities agree with typed IR at 98.1% of jointly resolved sites. Applied to bug detection, the refined call graph found 17 deep bugs in C software from nginx to the Linux kernel, three of them latent for over a decade; 12 are confirmed.
Read the full abstract ▾
All authors Kaixuan Li, Bozhi Wu, Jian Zhang, Peixin Wang, Ting Su, Yang Liu
Robotics
Steer2Grasp: Inference-Time Embodiment-Aware Steering for Diverse Physically Feasible Grasp Diffusion
Vignesh Vembar, Ayush Kaura, A Padmaprabhan et al. (+5 more)
Current grasp
diffusion models provide rich priors for generation, yet their object-centric approach can violate the kinematic and collision constraints imposed by the embodiment and the environment. Existing embodiment-aware methods primarily perform local corrections around generated grasps through gradient guidance or optimization, making it difficult to recover from fundamentally infeasible modes. We present Steer2Grasp, a training-free, embodiment-agnostic framework for inference-time grasp steering that adapts a frozen Cartesian grasp
diffusion model using deployment-specific rewards. Through Feynman-Kac (FK) inspired particle reweighting and resampling, the method reallocates population mass from infeasible to high-reward grasp modes, enabling population-level mode transitions without modifying the pretrained
diffusion model or requiring differentiable constraints. The framework enables a unified treatment for single and dual arm grasping through reachability and collision aware rewards, followed by gradient free gripper level local refinement. Across diverse objects,
robot embodiments, and constrained environments, our method substantially improves feasible grasp generation while maintaining proximity to the underlying grasp prior.
Read the full abstract ▾
All authors Vignesh Vembar, Ayush Kaura, A Padmaprabhan, Siddharth Sinha, Kailash Nagarajan, Keshab Patra, Md Faizal Karim, K Madhava Krishna
Distributed Computing
Adaptive Client Clustering and Coordination for Federated Learning Workflow Management in Edge Networks
Jieping Luo, Qiyue Li, Yuxuan Chen et al. (+4 more)
Federated learning (FL) is increasingly deployed as a managed learning service rather than as a set of isolated training jobs. In networked edge environments, dependent FL service flows must coordinate heterogeneous clients, non-IID data, fluctuating communication latency, and precedence-constrained tasks under service-level completion requirements. These coupled factors make participant management central to both time-totarget performance and learning stability. This paper proposes A-CoDa, an adaptive clustered coordination framework for managing dependent FL flows. A-CoDa first uses label-distribution divergence (LDD)-based greedy-balanced clustering to construct statistically coherent and size-aware client groups, which serve as a scalable management abstraction. Building on this structure, we design FedMIX, an uncertainty-aware intra-/inter-cluster participation mechanism that ranks clients by a loss-latency-uncertainty utility and adaptively controls cross-cluster probing according to training progress and latency conditions. A dependency-aware DAG scheduler then orchestrates layer-wise task execution so that parallelism and precedence constraints are jointly respected. We further provide a convergence analysis that frames the result as a sufficient loss-domain design bound, explicitly relating the attainable error floor and sufficient communication rounds to LDDinduced sampling mismatch, residual distribution shift, local-SGD drift,
stochastic variance, and adaptive probing budgets. Experiments on handwriting, wearable-sensing, product-image, and medical-imaging tasks evaluate A-CoDa under dependent FL workflows and demonstrate its effectiveness in reducing end-toend completion time while maintaining competitive accuracy.
Read the full abstract ▾
All authors Jieping Luo, Qiyue Li, Yuxuan Chen, Hang Qi, Jiaying Yin, Jingjin Wu, Qian Wang
Artificial Intelligence
OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning
Miaobo Hu, Shuhao Hu, Xiaobo Guo et al. (+4 more)
Coupled rollouts can reduce the noise of counterfactual action comparisons, but two issues prevent standard common-random-number constructions from serving as a general learning primitive in imperfect-information environments. First, an invalid coupling may expose hidden state, synchronize endogenous policy randomness, or misalign chance events after counterfactual histories diverge. Second, in multi-action policy optimization, lower return-contrast variance is not by itself the relevant objective: the optimizer depends on the return covariance
matrix after projection through the local policy-gradient geometry. We introduce observation-safe counterfactual coupling (OSCC), a framework that defines an admissible class through marginal preservation, information-state safety, branch-local policy randomness, semantic event
alignment, and trace-before-oracle replay. We derive a gradient-aware coupling criterion showing that, for marginal-preserving couplings, policy-gradient noise changes are determined by policy-Jacobian-weighted off-diagonal return covariance. This motivates OSCC-Select, a calibration-only selector that chooses among independent, root-only, continuation-only, and fully coupled rollouts using separate safety and gain certificates. Its gain target combines projected gradient noise with measured physical sampling cost and falls back to independent sampling whenever a simultaneous lower confidence bound does not certify improvement. On 100,000 fixed-root Leduc comparisons, the fully coupled CP-GRPO instantiation reduces return-contrast variance from 41.1158 to 18.1441, a 55.87% reduction, while preserving the declared branch marginals. With three actions, OSCC-Select chooses continuation coupling and attains gradient-noise trace 0.0783 versus 0.0917 for return-variance selection. Increasing calibration from 64 to 2,048 groups raises certification from 0.327 to 0.995.
Read the full abstract ▾
All authors Miaobo Hu, Shuhao Hu, Xiaobo Guo, Xin Wang, Bokun Wang, Rui Chen, Daren Zha
Robotics
A Disk-Shaped Magnetoelastic Torque Sensor for Robotic Joints Using Permanent Magnetization
Bruno Brajon, Enrico Gasparin, Nicole Yazigy et al. (+5 more)
Direct torque sensing is a growing need in the robotics community to enable precise control and interactions where torque estimation from motor current is not sufficient. This paper presents a novel disk-shaped magnetoelastic torque sensor with a compact axial envelope of about 1 cm, suitable for integration in robotic joints. A four-magnetometer architecture is used to measure the field modulated by the stress affecting a narrow magnetized region while rejecting the effects of parasitic cross forces. A custom-designed
magnetic shield enhances the torque sensitivity while reducing the external stray fields by a factor 7x. The device measures the torque with an accuracy of 1.34 %FS relative to the 50 Nm full scale (FS). The paper details the development of the sensor through the mechanical design, the magnetization procedure, and the experimental validation. The results demonstrate the potential of the proposed sensor for robotic applications.
Read the full abstract ▾
All authors Bruno Brajon, Enrico Gasparin, Nicole Yazigy, Lisa Salamin, Florian Copt, Guzmán Borque Gallego, Elias Klauser, Gaël Close
Artificial Intelligence
Talgat Daulbaev, Ilya Glazkov, Maxim Rakhuba et al. (+1 more)
Looped reasoners spend test-time compute by iterating a weight-tied map, but a small residual does not mean the state is a fixed point when that map lives in unconstrained latent space. We propose
Geometric Fixed-Point
Reasoning (GFPR), in which the iterated state is the prediction itself: a field of categorical beliefs on a product of simplices, whose argmax is the answer at every step. Because the state is a belief, task structure can be imposed through compact convex relaxations, either as structured readouts or directly in the recurrent state; in the latter case the update remains a continuous self-map, so a fixed point exists for any parameters. At about 7M parameters, GFPR reaches 95.1% exact match on Sudoku-Extreme, 92.0% on Maze-Hard, and 100% sequence accuracy on S_5 length 128, above the published FPRM numbers at the same scale. The same update also trains a 201M
language model on FineWeb-Edu in which each site is a distribution over the vocabulary; with 24 Picard steps it is above GPT-2 small on four zero-shot multiple-choice tasks and above GPT-2 medium on ARC-Easy.
Read the full abstract ▾
All authors Talgat Daulbaev, Ilya Glazkov, Maxim Rakhuba, Ivan Oseledets
Computation and Language
Jev Matches 7B Language Models for Speech-Neuroprosthesis Rescoring
Gabriele Cinà
A
speech neuroprosthesis decodes attempted
speech from
brain activity and ends by rescoring the decoder's candidate sentences with a
language model of several billion parameters, the only component that needs a GPU. Replacing that model with a cheaper one is hard: general
language models asked to pick one sentence from a list answer from where a label sits in the list rather than from the sentence itself. We pose rescoring as a single typed decision, one call that returns a probability for every candidate, served by Jev, a hosted model trained for calibrated decisions, and combine it with the decoder's own score. On 978 held-out sentences from a participant with ALS, where the published decoder alone reaches 8.1% word error, Jev reaches 7.5% against 7.8% for both OPT-6.7b and Qwen2.5-7B; with the decoder's weight re-tuned, 6.9% against 7.2% and 7.4%. Jev is ahead in all four comparisons and at most 0.2 points behind at the 95% bound. It costs 0.07 USD per thousand sentences and needs no GPU; a dedicated GPU running a 7B model is cheaper per sentence only above 43% utilisation, far beyond what one user generates. End-to-end latency over the internet is 262 ms, of which 62 ms is spent at the provider, the same order as a 7B model on a local GPU (27 ms) but not faster.
Read the full abstract ▾
Machine Learning
Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Serving
Jiantong Jiang, Yue Yang, Peiyu Yang et al. (+1 more)
Large
language model (LLM) serving is increasingly constrained by the GPU
memory consumed by key-value (KV) caches. Existing compression, eviction, and offloading techniques alleviate this pressure, but serving runtimes typically treat only the configured target KV representation as execution-ready. Under
memory pressure, this target-only contract can turn KV shortage into request stalls and preemptions. We present ElasticKV, a mixed-fidelity KV runtime built on the observation that target fidelity need not gate execution. ElasticKV introduces a compact intermediate KV state, making fidelity a runtime-managed execution property. To realize this state in a paged serving runtime, ElasticKV combines (i) a pair-structured layout that turns fidelity reduction into reusable GPU capacity, (ii) a dual-mode
attention backend that directly consumes the compact state while preserving the native target-only path, and (iii) pressure-aware fidelity management that adapts KV fidelity to
memory pressure. Our extensive evaluation across diverse workloads, model families and scales, and GPU platforms demonstrates the effectiveness and generality of ElasticKV. Under high concurrency, ElasticKV achieves 3.8-4.0$\times$ lower time-to-first-
token (TTFT) and 9.1$\times$ lower P90 TTFT than vLLM while preserving generation quality.
Read the full abstract ▾
All authors Jiantong Jiang, Yue Yang, Peiyu Yang, Feng Liu
Game Theory
Nearly Group-Separable Elections
Piotr Faliszewski, Jan Jabrocki, Stanisław Kaźmierowski et al. (+3 more)
We study the problem of computing how close a given election is to being group-separable, measuring proximity by swaps of adjacent candidates in the votes. We also consider several other domains, including caterpillar group-separable, balanced group-separable, single-peaked, and single-crossing ones. Our problem is generally intractable, but we find practical FPT
algorithms parameterized by the number of candidates or swaps. For the latter case, our
algorithm applies to all domains characterized by finite forbidden subelections, resolving a well-established open problem. We supplement our theoretical findings with experimental analysis.
Read the full abstract ▾
All authors Piotr Faliszewski, Jan Jabrocki, Stanisław Kaźmierowski, Kristýna Pekárková, Šimon Schierreich, Ildikó Schlotter
Computation and Language
ManiEdit: Sequential Unstructured Knowledge Editing for Language Models from a Manifold Perspective
Rui Liu, Chenheng Zhang, Haoxuan Li et al. (+1 more)
Large
language models (LLMs) inevitably generate some incorrect or outdated content, necessitating efficient and precise mechanisms for continual knowledge updates. However, existing model editing methods struggle to sequentially edit unstructured long-form knowledge, suffering from severe edit forgetting and degradation of general capabilities. To address these challenges, we reframe knowledge editing from a
manifold perspective, viewing it as a localized displacement of an edit sub-
manifold within the global knowledge
manifold. Under this formulation, the problem can be decomposed into two key questions: (i) how to identify representative edit points that effectively anchor the edit sub-
manifold, and (ii) how to preserve the remaining
manifold structure during the sub-
manifold displacement process. Based on this perspective, we propose ManiEdit, a novel
manifold-aware autoregressive editing framework consisting of two core components. Pivot Localization addresses the mediocre-point dilemma by identifying high-leverage pivots to anchor the edit sub-
manifold.
Manifold-Aware Preservation preserves different knowledge types through an energy-weighted penalty combined with recursive null-space
alignment. Experiments on two base LLMs and four unstructured editing
benchmarks demonstrate that ManiEdit achieves state-of-the-art performance, outperforming the strongest baseline by up to +27.81 BERTScore and +8.50 ROUGE-L, while maintaining near-original general capabilities across six representative downstream tasks. Our code is available at: https://github.com/Areyliu/ManiEdit
Read the full abstract ▾
All authors Rui Liu, Chenheng Zhang, Haoxuan Li, Zhouchen Lin
Machine Learning · Cross-listed from stat.ML
Neural Scaling Laws of Transformer Operator Network
Haoran Yan, Zhongjie Shi, Yuanzhe Xi et al. (+2 more)
Transformers have emerged as powerful architectures for learning solution operators of physical systems. Empirically the prediction error has been observed to decrease when the data size and model size increase, suggesting neural scaling behavior. Yet a theoretical understanding of such scaling laws for
transformer-based operator learning remains limited. In this work, we develop a theoretical framework for characterizing the approximation and generalization errors of
transformer-based operator learning. Our analysis builds on a local-to-global approximation principle that is naturally aligned with the softmax
attention mechanism and yields discretization-invariant output functions. On approximation theory, we derive a universal approximation error of
transformer-based operator learning for Hölder-regular operators. On generalization theory, we establish a power scaling law between the prediction error and the training data size. The rate of convergence represented by the scaling exponent explicitly reflects the dimensions of the input and output domains, the regularity of the underlying functions and operators, and crucially, the intrinsic dimension of the input function class. By exploiting this intrinsic low-dimensional structure, our analysis yields a power-law generalization rate for operator learning, in contrast to the logarithmic-type power-law rates appearing in existing analyses of operator learning with feedforward neural networks. Numerical experiments validate the predicted power-law scaling and confirm that the convergence rate varies systematically with the intrinsic dimension of the input function class.
Read the full abstract ▾
All authors Haoran Yan, Zhongjie Shi, Yuanzhe Xi, Peng Chen, Wenjing Liao
Cryptography and Security
Where the Numbers Come From: Auditing Evaluation in Provenance-Based Intrusion Detection
Jihwan Moon, Gunhee Kim and Myeongjang Pyeon
Reproducing a provenance-based intrusion detector's score does not establish what that score says about its emitted alarms or the information its encoder uses. We audit nine released implementations, execute four detectors using their own code, and isolate three measurement effects. First, a fixed-alert comparison separates label choice from neighbourhood credit: ThreaTrace reports precision 0.938 with neighbourhood credit, although only seven of its 994 alarms carry its own attack label. Second, removing test-label checkpoint selection lowers attack detection precision (ADP) by 0.16 to 0.33 across four forty-member word2vec configurations without changing detector order. Third, a buffer-reuse defect gives a linear encoder unintended degree-dependent inputs. Correcting it lowers type-only ADP in every identical-input initialization pair on two hosts, while historical word2vec effects depend on the host. These findings qualify claims from the inspected implementations about alarm precision, performance magnitude and static-attribute sufficiency. STRICT connects them to six checkable reporting requirements. Because the comparisons condition on
benchmark targets, they neither validate those labels nor establish a universal detector ranking.
Read the full abstract ▾
Computation and Language
E-CONAN (Entailment, CONtradition And Neutral) Diagnostics Dataset Investigating Linguistic Phenomena in Arabic Natural Language Understanding
Khloud AL Jallad, Nada Ghneim and Ghaida Rebdawi
Natural
Language Understanding (NLU) plays a crucial role in various applications, yet its performance suffers from weaknesses in handling the complexities of human
languages, ranging from lexical ambiguity to high-level
reasoning difficulties. Analyzing errors across diverse linguistic phenomena is crucial for NLU improvement, as it will help humans get insights to comprehensively assess models' limitations and capabilities, so optimizing models' generalization. Notably, several
benchmarks contain diagnostics
datasets designed for investigation and fine-grained error analysis. When highlighting the gaps in the state-of-the-art, we noted that there is no naming convention for macro and micro categories or even a standard set of linguistic phenomena that should be covered. To overcome this gap, we propose an initial hierarchy for Cross-Lingual NLU error analysis. Moreover, we propose a methodology to create an NLI hierarchical framework and applied a case study on Arabic NLU. Moreover, this paper introduces E-CONAN diagnostics
dataset, a freely available
dataset manually-annotated with coarse-grained and fine-grained categories based on our proposed Arabic hierarchy. E-CONAN
dataset helps NLU designers better understand their models by doing error analysis and in-depth investigation. We used E-CONAN to investigate the performance of 9 pretrained
language models and 5 LLMs. Results indicate that LLMs outperform pretrained models in world knowledge and commonsense
reasoning macro-category, and underperform pretrained models in syntactic macro-category. Moreover, the hardest phenomena for all models is
Reasoning, and the easiest phenomena for all pretrained models is Syntactic, and the easiest for LLMs is Lexico-Syntactic.
Read the full abstract ▾
Machine Learning
Discovering Symmetries in Neural Network Parameter Spaces
Bo Zhao, Nima Dehmamy, Robin Walters et al. (+1 more)
Parameter space
symmetries are important for understanding neural networks' loss landscape, training dynamics, and generalization. However, systematically identifying these
symmetries remains a challenge. In this paper, we formalize data-dependent parameter
symmetries and characterize loss invariance and the group-action axioms through infinitesimal conditions, which provide objectives for jointly learning group generators and nonlinear action maps. Our framework systematically uncovers parameter
symmetries, including previously unknown ones. To study larger networks, we establish conditions under which subnetwork
symmetries extend to the full model. The same construction gives an explicit family of finite-batch
symmetries, providing both analytical examples and a foundation for discovery through small subnetworks. Using the infinitesimal characterization and subnetwork construction, we implement a framework for automated discovery of parameter
symmetries, and successfully uncovered
symmetries in various architectures, including pretrained
transformer models.
Read the full abstract ▾
All authors Bo Zhao, Nima Dehmamy, Robin Walters, Rose Yu
Computers and Society
Latent Class Analysis of Digital Content Use Contexts in AI-Generated Synthetic Personas
Eunjeong Song and Sehee Hong
AI-generated synthetic personas are increasingly used for content planning and virtual-user simulation, yet the digital content use contexts embedded in their narratives remain underexamined. Using all 1,000,000 records of NVIDIA's Nemotron-Personas-Korea, this study coded mentions of five engagement modes-viewing, discovering and sharing, interacting, reading, and listening-with an openly released Korean coding dictionary and applied latent class analysis with a three-step approach accounting for classification error. Four classes emerged: viewing-centered (20.8%), reading-centered (3.8%), listening-centered (2.1%), and low-mention (73.3%). Age showed the strongest association: the odds of the viewing-centered versus low-mention class were multiplied by 0.15 per 10-year increase, while mentions of any non-viewing mode declined from 49.4% among those under 30 to 2.3% among those aged 70 and over. Sex differences in predicted class probabilities were below 1 percentage point, and provincial differences reached at most 5.0 percentage points. These classes describe narrative configurations, not consumer segments, and provide a basis for assessing diversity in synthetic persona use contexts.
Read the full abstract ▾
Artificial Intelligence
EverMine: Dissecting the Self-Evolution of Research Capabilities in Long-Horizon Alpha Research
Siyuan Li, Jiangfeng Zhang, Rui Yao et al. (+3 more)
Self-evolving
agents aim to turn research feedback into reusable skills, tools, and research rules. Whether these accumulated capabilities continue to improve later research requires controlled evaluation. Long-horizon alpha discovery provides a state-dependent setting: once a new factor enters the portfolio, the predictive information already covered changes, so the value of the same candidate or experience may change over time. We introduce EverMine, an empirical framework for studying self-evolving research capabilities in long-horizon alpha discovery. EverMine decomposes the research state into history (Hist), the current factor portfolio (Frontier), and reusable capabilities (Cap). Under matched resource limits, we compare complete runs with fixed or evolving Cap, and replace Cap while holding Hist and Frontier fixed to estimate the conditional value of accumulated capabilities. We also combine full trajectories with historical-state replay to examine how experience-based decisions affect candidate selection and portfolio outcomes. Across 18 long-horizon trajectories, end-to-end comparisons show no consistent gain from Cap
evolution. Across 48 continuation branches from shared Hist and Frontier states, accumulated Cap also does not consistently outperform the initial Cap. Parameter tuning of existing factor structures can still improve the portfolio. In an exploratory replay of two screening batches from one Evolving trajectory, some screened-out candidates have positive marginal value at the original state, yet submitting all screened-out candidates sequentially slightly lowers final portfolio IC in both batches. These results show that candidate value depends on the evolving portfolio and submission order, and motivate evaluating self-evolving research capabilities through end-to-end outcomes, conditional capability value, and the consequences of experience-based decisions.
Read the full abstract ▾
All authors Siyuan Li, Jiangfeng Zhang, Rui Yao, Weihua Qiu, Mingyang Xu, Zixuan Yuan
Computer Vision
In-Token Learning for High-Fidelity Image Restoration via Diffusion Transformers
Xingfu Yi and Xiaoxue Yu
We present In-
Token Learning, an image restoration framework that adapts a pretrained
diffusion transformer using conditional rectified flow matching. Clean targets paired with degraded inputs supervise transport from Gaussian noise to restored images. Spatially aligned degraded-image
tokens are fused with evolving latent
tokens along the channel dimension, preserving the image-
token count at a given resolution. Direct Low-Quality Guidance (DLG) combines frozen degraded-image
embeddings with a fixed task
prompt through the native conditioning pathway, without a trainable ControlNet-style branch or image captioning. We evaluate super-resolution and denoising on DIV2K, LSDIR, FFHQ, RealLQ250, and RealPhoto60, and automatic colorization on DIV2K and LSDIR. The tasks use separately trained checkpoints under the same framework. Results show competitive fidelity and perceptual quality under the evaluated protocols, with weaker generalization on RealLQ250. We report full-image QHD ($2560{\times}1440$) inference and a tiled $12$K restoration demonstration of Along the River During the Qingming Festival.
Attention cost still increases with resolution. This technical report preserves the early broader study underlying Fill2SR, which subsequently developed the real-world super-resolution direction.
Read the full abstract ▾
Robotics
AI-Driven Collaborative Assembly Line Inspection: System Integration and Deployment Challenges
Asya Ünal, Amr Okasha, Ege Ç{\i}rakman et al. (+1 more)
Manual visual inspection on assembly lines is a persistent manufacturing bottleneck: operator fatigue over extended shifts lowers defect-detection rates. This paper presents the design, integration, and field deployment of an AI-assisted collaborative inspection
cell at the Silverline kitchen-appliance factory, developed within the AI-PRISM project. The
cell couples a Universal
Robots UR 10e cobot carrying a machine-
vision defect-detection pipeline with a Comau Racer-5 cobot for functional tests, coordinated through ROS 2 Humble on an Ubuntu 22.04 LTS server. Multi-modal data (Basler camera imagery, TIA microphone acoustics, and SPS electrical-safety measurements) are logged locally and visualised in real time with Grafana. We report the practical deployment challenges (close-proximity safety, AI robustness under glare and reflections, ROS 2 namespace collisions across two cobots, and operating-system and dependency issues) together with the engineering solutions adopted, and structure the integration through a four-level Human-
Robot Interaction analysis. The deployed
cell cuts per-unit quality-check time from 82 s to 61 s (about 25%), raises final-control resource efficiency from 0.75 to 0.88, reduces operator visual-inspection viewing time by 82%, and significantly lowers operator mental demand (p = 0.005, NASA-TLX).
Read the full abstract ▾
All authors Asya Ünal, Amr Okasha, Ege Ç{\i}rakman, Perin Ünal
Machine Learning
LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation
Jie Wang, Yanbo Sun, Zheng Yan et al. (+2 more)
Trust evaluation plays a critical role in cybersecurity by supporting risk mitigation and decision-making. A variety of trust evaluation methods have been proposed, with learning-based approaches offering high accuracy and automation. However, they often require substantial ground truth, suffer from low training efficiency, lack support for basic trust properties, and provide limited explainability. Large
Language Models (LLMs) offer a compelling alternative due to their strong zero-/few-shot
reasoning abilities and broad knowledge. To this end, we propose LLM4Trust, the first
benchmark framework that systematically explores the capabilities of LLMs for trust evaluation. We first construct diverse trust graphs to model five basic trust properties and design corresponding property understanding tasks. We then assess the ability of eight representative LLMs to understand these properties under nine
prompt methods. Based on this exploration, we identify the most effective LLM-
prompt combinations and apply them to five real-world
datasets for validating LLMs' trust evaluation capability. During this process, we propose two strategies to extract key information from large-scale trust graphs, addressing the context window limitations of LLMs. Extensive experiments show that LLMs can effectively understand basic trust properties and have great potential for real-world trust evaluation, particularly under limited supervision. However, they remain vulnerable to attacks targeting trust graphs and demonstration examples used in few-shot prompting, and incur high inference costs. Accordingly, we propose a defense mechanism and batch inference to improve the robustness and efficiency of LLM-based trust evaluation. The source code of LLM4Trust is available at https://github.com/Jieerbobo/LLM4Trust
Read the full abstract ▾
All authors Jie Wang, Yanbo Sun, Zheng Yan, Jiahe Lan, Elisa Bertino
Computer Vision
Anatomy-Structured Hierarchical MIL for Weakly-Supervised Thoracic Disease Detection in Chest X-rays
Jeongin Kim, Sohyun Ahn, Seo Young Kang et al. (+6 more)
Weakly-supervised thoracic disease detection in chest X-rays (CXR) is challenging due to subtle appearances and complex anatomical overlap, motivating anatomy-aware modeling for improved localization. However, prior anatomy-aware methods typically rely on coarse region proxies or static spatial priors, which may restrict dynamic instance discovery and limit precise localization of small abnormalities. We propose Anatomy-Structured Hierarchical Multiple Instance Learning (ASH-MIL), a framework that introduces parallel anatomy-structured observation branches (cardiac, pulmonary, and agnostic) combined with hierarchical MIL aggregation. Anatomical priors are injected as soft spatial biases into decoder cross-
attention, enabling anatomically grounded evidence maps without disease bounding-box supervision. Instance localization is derived directly from MIL-weighted cross-
attention maps without bounding box supervision. Experiments on CXR8 and cross-domain MIMIC-CXR held-out sets demonstrate consistent improvements over prior weakly-supervised and anatomy-aware approaches, particularly under stricter localization criteria. Our code is available at https://github.com/jn-kim/ash-mil.
Read the full abstract ▾
All authors Jeongin Kim, Sohyun Ahn, Seo Young Kang, Jaeyi Sung, Soomin Kim, Sungho Cho, Rena Lee, Kwanchang Kim, Junhyug Noh
Computer Vision
SceneScaffold: Active Scene-State Construction for Unified 3D Scene Understanding
Xiangqi Li, Libo Huang, Jiarui Zhao et al. (+4 more)
Recent 3D large
multimodal models (3D-LMMs) rely on a visual bottleneck to compress complex 3D scene evidence into a limited number of visual
tokens compatible with large
language models (LLMs). Current visual bottlenecks, however, often passively compress heterogeneous 3D evidence into a homogeneous object-centric
token sequence, leaving the spatial organization of the scene under-represented. This under-representation forces the LLM to recover spatial relations from a flattened
token sequence, leading to unstable
reasoning in relation-intensive and spatially ambiguous scenes. To address this issue, we propose SceneScaffold, an active scene-state construction framework for unified 3D scene understanding. SceneScaffold reformulates the visual bottleneck from a passive feature compressor into an active scene organizer, constructing a role-aware spatial scaffold before
language reasoning. Specifically, SceneScaffold organizes superpoint-level visual evidence into scene-state components with distinct structural roles: entity states preserve core object semantics, scene-frame states maintain spatial references via boundary and region anchors, relation states encode object-environment interaction cues, and a global summary provides compact context. Through this role-aware construction, SceneScaffold provides the LLM with a spatially organized scene representation before
language reasoning. Experiments on unified 3D scene understanding tasks, including 3D visual grounding, question answering, and dense captioning, demonstrate the effectiveness of SceneScaffold, while diagnostic results further show its applicability to relation-intensive and spatially ambiguous cases. Code is available at https://github.com/lixiangqi707/SceneScaffold.
Read the full abstract ▾
All authors Xiangqi Li, Libo Huang, Jiarui Zhao, Weilun Feng, Chuanguang Yang, Zhulin An, Yongjun Xu
Multiagent Systems
TRACE: Governing Memory Validity in Evolving Multi-Agent Systems
Wenjun Xiong, Shengtao Zhang, Shangding Gu et al. (+5 more)
Persistent
memory lets
language-model
agents carry information across long-running collaborations, but leaves a lifecycle question open: what may a returning
agent still act on once the shared state has changed? A
memory can be correctly retrieved, relevant to the current task, and faithful to its source, and nonetheless be inadmissible for action: an itinerary saved before a pause still names the hotel the team has since replaced. We formalize this as temporal
memory admission and present TRACE, a training-free layer that treats re-entry as an eligibility decision rather than a storage or
retrieval operation, reconciling a departure checkpoint against absence-period updates, resolving explicit and implicit invalidation, and releasing a bounded Return View only when it covers the returning role's open obligations. We evaluate TRACE under three actor models on Memora, STALE Type II, and a derived ManBench-Return setting, each recast as return episodes: one
agent departs, four teammates change the shared state, and the
agent rejoins. What separates methods is not overall accuracy but whether one can retain valid
memory and reject stale
memory at once, and no single-policy baseline can: Restore (reinstate the departure checkpoint in full) admits stale state, Reset (start the return from an empty
memory) discards valid state, each bottoming out at 0% on one of the two. TRACE is the only method high on both, reaching 92.6-98.3% valid-information availability with 98.4-99.5% invalid-information rejection on ManBench-Return, within 3.8 points of the best baseline's overall accuracy. On STALE Type II it improves Overall over the strongest comparison policy by 22.3 (Qwen), 18.5 (Gemini), and 27.5 (DeepSeek) points at roughly 2.3 times their
tokens, while a write-time consolidation pipeline is more accurate still at 3.99 times TRACE's.
Read the full abstract ▾
All authors Wenjun Xiong, Shengtao Zhang, Shangding Gu, Bo Tang, Zhiyu Li, Feiyu Xiong, Ying Wen, Muning Wen