Machine Learning · Cross-listed from cs.CL
When Noise Fabricates Bias: The Fragility of LLM-as-a-Judge Bias Measurement under Noisy Text
DongHyun Ryu, Jaehyeok Lee, YeongJun Hwang et al. (+1 more)
Large
language models are increasingly used as judges to measure social bias in text, yet the passages they judge are often noisy, containing typos, informal spelling, and broken punctuation. The consequences of such surface noise for social bias measurement remain unclear. To investigate this question, we apply five realistic noise conditions at multiple intensity levels to 3,822 stereotype-related responses and compare the resulting bias judgments with those on the original text. We find that such surface noise does not degrade bias measurement symmetrically: it is far more likely to turn neutral judgments into biased ones than biased judgments into neutral ones, by up to a 120x margin. We further observe two non-obvious effects across four LLM judges: in the most fragile judge the distortion is at its purest at mild, realistic noise levels, where erasure is scarcest, and as judges grow robust it attenuates toward parity rather than reversing. Bias measured on noisy text is therefore systematically overestimated, most in the categories that matter most for
fairness.
Read the full abstract ▾
All authors DongHyun Ryu, Jaehyeok Lee, YeongJun Hwang, JinYeong Bak
Machine Learning
The information geometry of large language models is shared, learned, and controllable
Dario Picozzi
Large
language models learn similar behaviours, yet it remains unclear what structure they share or how to change one behaviour without disturbing others. The Fisher-Rao geometry of next-
token probabilities connects these questions: behaviour determines this geometry up to output-preserving
symmetries, whereas activation geometry depends on coordinates. Across
transformer, state-space and recurrent models, output geometries agree more strongly than activation geometries, and shared geometry supports semantic-category transfer. Agreement with human word choices increases with predictive accuracy, scale and training, and improves further after model-only calibration.
Token probabilities and read-out geometry jointly predict the spectrum and its effective dimension. Controlled
language assignments show that geometry follows the
language law across architectures. Pretraining corpus statistics predict held-out fact acquisition without recalibration, while randomised experiments show that deeper evidence substantially delays acquisition across every tested architecture and evidence construction. Finally, the geometry prescribes minimum-disturbance local interventions, predicts their relative cost, and supports reusable control: updates learned on donor
prompts transfer to unseen
prompts while better preserving behaviour on reference
prompts than Euclidean control. The same
geometric correction improves steering, editing, attribution, dictionary learning and fine-tuning.
Read the full abstract ▾
Machine Learning
EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression
Motahare Mounesan and Irfan Khan
Recent advances in
multimodal large
language mod- els (MLLMs) have opened new opportunities for edge intelligence by enabling
reasoning across heterogeneous sensor modalities, such as
vision, text, and telemetry data. However, deploying these capabilities on resource-constrained edge platforms remains challenging due to the substantial computational,
memory, and communication demands of modern MLLMs. Rather than transmitting raw sensor observations or partitioning neural networks at intermediate layers, Edge Multi-Modal Intelligence (EMMI) communicates a compact representation between edge devices and server resources, enabling communication-efficient edge MLLM inference. To achieve this, EMMI performs modality-specific encoding, cross-modal representation fusion, and learned compression at the edge, transmitting only a compact latent representation to server-side resources for high-capacity MLLM
reasoning. This representation-centric design reduces communication overhead, preserves local data
privacy, and provides a fixed-size interface between heterogeneous edge devices and server-side MLLMs. Evaluation on a representative
multimodal benchmark demonstrates that EMMI can reduce the communication payload by 32x while maintaining comparable downstream accuracy, resulting in up to a 3.4x reduction in estimated end-to-end inference latency under bandwidth-constrained edge conditions.
Read the full abstract ▾
Machine Learning
T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks
Junyao Yang, Yucheng Shi, Zhongzhi Li et al. (+4 more)
Agent usage is shifting toward long-horizon tasks such as coding and scientific discovery, among which terminal tasks are especially important. We introduce T1, a Mixture-of-Experts model of 122B total trained with reinforcement learning, operating a real shell in a cloud sandbox for up to 300+ tool-call turns per task, rewarded by executing each task's own verifier. We provide a comprehensive recipe: First, an aggressively warm-started to stabilize actor-critic training, with a dense process reward scoring trajectories by the absolute number of passing verifiers. Second, stable optimization through TITO construction, training on the exact sampled
token identifiers with drift repair at turn boundaries, and rollout routing replay, recording the sampler's per-
token expert choices at every MoE layer and replaying them during training. Third, fully out-of-distribution training corpus: isolated seeds and synthesized tasks disjoint from Terminal-Bench 2.1 ensures gains reflect genuine capability transfer over
benchmark overfitting. Together, TITO and R3 cut the training-to-inference log-probability difference from 0.021 to 0.013, with exactly aligned zero
token drift in the loss region. On Terminal-Bench 2.1, our post-train pipeline raises initial base model from 43.8% to T1 with 64.0% resolved. On Long-Horizon Terminal Bench, T1 reaches 27.9% and surpasses GPT-5.4 and GLM-5.1.
Read the full abstract ▾
All authors Junyao Yang, Yucheng Shi, Zhongzhi Li, Ruhan Wang, Zongxia Li, Haitao Mi, Leowei Liang
Machine Learning · Cross-listed from cs.CV
Meta-Learning for Classifier Selection in Image Datasets: A Feature-Driven Framework for Accuracy Prediction
Zahra Nabizadeh_Shahre_Babak, Farzaneh Koohestani, Nader Karimi et al. (+2 more)
No Free Lunch
theorem implies that any performance gains achieved by a classifier on a particular image distribution are necessarily offset by a loss of performance over the set of all possible problems; thus, no single model is universally optimal. Selecting the most suitable classifier for image
datasets is a critical yet challenging task due to the intrinsic complexity and diversity of images. This paper proposes a meta-learning framework that leverages a comprehensive set of meta-features capturing
dataset complexity to predict classifier performance without exhaustive training. By extracting and selecting features using methods such as autoencoders, pre-trained networks, and dimensionality reduction techniques, we train regression models to efficiently estimate classifier accuracies. Additionally, clustering techniques are employed to group classifiers with similar performance patterns, simplifying the recommendation process. The
datasets used span a wide range of concepts, including nature, animals, numbers, motorcycles, medical images, and human bodies, to ensure broad generalization. Evaluated on 56 diverse image
datasets, our approach achieves an average ranking prediction accuracy exceeding 86%, demonstrating its effectiveness in guiding model selection. This scalable and interpretable framework provides a practical solution to improve classification performance while reducing computational costs.
Read the full abstract ▾
All authors Zahra Nabizadeh_Shahre_Babak, Farzaneh Koohestani, Nader Karimi, Shahram Shirani, Shadrokh Samavi
Machine Learning · Cross-listed from cs.CL
Rebalancing Token Importance in Language Models with TF-IDF Weighted Cross-Entropy Loss
Zhijian Li, Stefan Larson and Kevin Leach
Large
language models are typically trained under uniform
token weighting, which allows frequent and low-information
tokens to dominate learning and can increase the tendency to memorize surface-level text spans. To address this, we present an information-weighted cross-
entropy loss that rescales
token-level contributions using TF-IDF statistics, emphasizing semantically informative
tokens while down-weighting ubiquitous ones. Experiments on five decoder-only LLMs ranging from 1.1B to 13B parameters show consistent reductions in memorized substring length while preserving perplexity and downstream task performance. Under LoRA fine-tuning, TF-IDF reduces average substring memorization length by 14% across all five models. Under full-weight fine-tuning on TinyLLaMA 1.1B, the reduction reaches 58%. Our approach is architecture-agnostic and can be incorporated into existing training pipelines with less than 3% computational overhead, offering a lightweight and principled way to mitigate memorization without disrupting standard training dynamics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CV
New Evidence, Same Choice: Testing Physical Experiment Selection in Vision Language Models
Sourajit Saha, Shubhashis Roy Dipta, Nobin Sarwar et al. (+4 more)
A model first sees an image from one physical measurement experiment, such as how far a block coasted, and must answer a question about a new trial, such as whether the block will pass a target after a fixed push. The initial experiment may provide enough information to answer, or the model may need another measurement, such as the object's mass, friction, restitution, or spring stiffness. We study whether
vision language models can decide when to answer immediately and, when more evidence is needed, which experiment to perform. Current physical
reasoning benchmarks usually evaluate only the final answer, so they do not directly measure this decision-making ability. We introduce a controlled evaluation where each problem provides one measurement image and four possible physical worlds created by combining two possible masses and two possible values of another relevant property. The model must either stop and answer or select the cheapest additional experiment that can resolve the question. We construct matched problem pairs where changing either the observed measurement or the question changes the optimal action. Since all possible worlds and experiment costs are known, we can explicitly determine the optimal choice. Across six open models and 144 physical parameter sets, direct responses repeat the same action for 95.1% to 100% of image pairs even when the correct action changes. Brief
reasoning improves action switching, but the best model makes both decisions correctly for only 5.9% of image pairs. Additional analysis reveals failures in measurement interpretation, physical
reasoning, and response formatting. By evaluating evidence selection separately from final answers, our
benchmark reveals limitations in physical
reasoning that conventional answer accuracy can overlook.
Read the full abstract ▾
All authors Sourajit Saha, Shubhashis Roy Dipta, Nobin Sarwar, Shaswati Saha, Yuxuan Jiang, Siyuan Li, Qiheng Wang
Machine Learning
Topological Necessities: Mechanism-Invariant Strategic Subgoals for Cross-Embodiment Goal-Conditioned Control
Hao Shi and Xi Li
Long-horizon goal-conditioned reinforcement learning delegates control to a high-level module that proposes subgoals, but existing subgoals are implicit byproducts of value functions or latent actions, tied to the executor that produced them. We study a different object: a route-conditioned order of unavoidable stages that every successful executor must traverse, recoverable from offline trajectories and belonging to none of them. Its defining properties are topological: an unskippable stage is a separating set that every admissible path must cross, and a loop in free space forces a route choice. We read the two by homology in dimensions 0 and 1 over a transport-weighted carrier built from successful trajectories, yielding an enumerable gate set with shell-level certificates; the certified gates are what we call topological necessities. Certified gates enter the decision loop as a recursive topological gate hierarchy. Under a fixed, isomorphic free space, the object survives executor replacement: gates frozen on PointMaze data transfer without retraining to Ant and
Humanoid, attaining the highest
Humanoid aggregate under a unified interface (96.1), with +36.0 over a map-privileged reference on the multi-route task (p=1.4e-5); the planner saturates PointMaze (100+/-0) and matches or exceeds the strongest baselines on AntMaze (
giant +22.9) and Kitchen (+15.8/+12.6).
Read the full abstract ▾
Machine Learning
Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift
Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi et al. (+3 more)
Unlabeled-unlabeled (UU) learning allows us to learn a binary classifier from two sets of unlabeled data with different class-priors. It is a general framework because it includes a wide variety of supervised learning such as positive-unlabeled (PU) learning, noisy label learning, and similarity-based learning. Existing UU learning assumes that the test and training distributions have the same class-conditional densities. However, this assumption rarely holds in practice due to distribution shifts. This paper proposes a distribution shift adaptation method for UU learning that uses UU data in the training distribution and a few UU data in the test distribution. The proposed method is based on the importance weighting, which minimizes the test risk by using training data with estimated importance weights. Although existing importance weighting methods cannot handle UU data, we show that it can be done in a principled manner. Thanks to the generality of UU learning, our method can handle various learning problems such as PU and noisy label learning under distribution shift within a single framework while existing methods are usually tailored to a specific problem. Moreover, it does not require any assumption of the shift types such as
covariate shift. We experimentally demonstrate the effectiveness of the proposed method with real-world
datasets.
Read the full abstract ▾
All authors Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara
Machine Learning · Cross-listed from math.NA
A variational physics-informed graph neural network for heterogeneous solid mechanics
Aashay Rajan Yadav, Amiya Prakash Das and Ratna Kumar Annabattula
Stress localization in heterogeneous solids is governed by the bimaterial interface, where the displacement field remains $C^0$-continuous, while in-plane stresses jump due to the stiffness mismatch. Coordinate-based physics-informed neural networks (PINNs) represent this jump via a prescribed regularization width or a weighted interface penalty, making their accuracy sensitive to how phase-contrast changes are handled. This work presents a variational, label-free physics-informed graph neural network (PI-GNN) in which the heterogeneity is carried by the discretization rather than by the trial field. The solver operates on a conforming adaptive mesh graph, assigns constitutive behavior per element, and minimizes the discrete total potential energy as a single unweighted objective in which only first derivatives appear. The discrete energy on piecewise-linear elements coincides with the finite element (FE) Ritz functional. Dirichlet conditions are enforced by construction, with no penalty term, no interface weight, and no prescribed transition width. Using one fixed architecture, optimizer, and loss across small-strain elasticity and finite-strain Neo-Hookean hyperelasticity in two and three dimensions, the von Mises error remains below $3.58\%$ across a stiffness-contrast sweep spanning $(E_{\mathrm{inc}}/E_{\mathrm{mat}}\in[10^{-2},10^{2}])$, where a strong-form PINN degrades to $5.58\%$, and its displacement error reaches $7.66\%$ against $0.49\%$ for the PI-GNN. A trained network halves the ($\sigma_{xx}$) error of an energy-based PINN ($5.01\%$ versus $10.94\%$). Training cost exceeds a single FE solve by more than an order of magnitude, so the construction is a variationally consistent, penalty-free interface representation for parametric surrogates and inverse identification rather than a replacement for a one-off FE analysis.
Read the full abstract ▾
Machine Learning
Thompson Sampling for Non-Monotone Convex Ridge Bandits: Monotonicity Is Not Needed for Polynomial Regret
Xuan Li
Bakhtiari, Lattimore and Szepesvári (COLT 2025) proved that Thompson sampling (TS) has
Bayesian regret $\tilde O(d^{5/2}\sqrt n)$ for bandit convex optimisation with convex monotone ridge losses $f(x)=\ell(\ip{x}{\theta})$, and asked whether monotonicity of the link is necessary. We give a qualitative negative answer. For every prior on $[0,1]$-valued, $1$-Lipschitz convex ridge losses with an arbitrary convex, possibly non-monotone, link, and for any fixed measurable selection of minimisers, exact-posterior TS has
Bayesian regret $O\big((d+1)^4\sqrt{dn}\,\log(e+nd\max\{1,\diam K\})\big)=\tilde O(d^{9/2}\sqrt n)$. The monotone proof relies on a single-removal John-ellipsoid dichotomy; we show by an explicit twelve-point configuration that this dichotomy fails for non-monotone links, and replace it by an $O(d^2)$ cardinality bound for “uninformative” configurations. The bound uses a Boolean rounding argument: a $0$-$1$
matrix within $1/(4r)$ in max-norm of a rank-$r$
matrix has rank at most $2r-1$. We construct $d(d+1)$ uninformative losses, showing that the cardinality bound is tight up to constants in the large-diameter-to-gap regime, and give a self-contained information-ratio-to-regret transfer that is uniform over fixed measurable selections. Whether the $d^{5/2}$ dependence of the monotone case can be retained remains open.
Read the full abstract ▾
Machine Learning
EGGROLL, Unrolled: Understanding and Improving Low-Rank Evolution Strategies at Scale
Ege C. Kaya and Abolfazl Hashemi
EGGROLL (Sarkar et al., 2026) makes
evolution strategies (ES) practical for LLMs by replacing dense Gaussian weight perturbations with low-rank Gaussian products, often of rank one. This choice is computationally attractive but geometrically severe: Each rank-one perturbation lies in a zero-volume subset of the ambient
matrix space, despite having identity covariance. We characterize the EGGROLL update mean field at finite rank and nonzero perturbation radius as a resolvent applied to the gradient of the perturbation-smoothed objective. This transformation can make the mean field nonconservative and reverse the local stability of an optimum. EGGROLL nevertheless recovers the gradient exactly on quadratic objectives at every rank and radius. In finite populations, the additional sampling variance of rank-one perturbations relative to dense Gaussian ES decays inversely with
matrix width under a local affine model, and is only $0.098\%$ at width $4096$. Finally, we introduce LOO-ROLL, a leave-one-out
estimator that replaces EGGROLL's two antithetic evaluations per direction by one. At equal evaluation cost, LOO-ROLL halves
estimator MSE in
transformer blocks. Across fourteen post-training settings up to 14B parameters, matched-time comparisons with EGGROLL yield eleven improvements in individual paired tests and no significant loss. At 1.7B, 8B, and 14B parameters, matched-time gains are $2.9$, $14.1$, and $7.9$ percentage points on GSM8K and $12.2$, $8.4$, and $8.1$ points on MATH-500.
Read the full abstract ▾
Machine Learning
Phases in a class of associative memories via hidden neurons
Toshihiro Ota and Masato Taki
Associative
memory in the Hopfield network is attractor dynamics in a disordered many-body system, and higher-order and exponential extensions turn its
retrieval update into softmax
attention. The
polynomial and exponential regimes have been analyzed by different methods, with no common architecture in which to ask what fixes the storage scale. In this paper we study the bipartite architecture of Krotov and Hopfield, which we call the class $H$, whose model is fixed by a Lagrangian for each layer, taking the hidden
neurons as the order parameter of
retrieval. At
polynomial load the replica method yields the replica-symmetric phase diagrams and closed-form capacities, and the crosstalk moment is common to Ising and spherical visible
neurons, so their differences come from the visible
entropy. With a softmax hidden layer the load is exponential, and a copy representation maps the thermodynamics onto random-energy-model counting, with paramagnetic, condensed, and frozen phases. Heating destabilizes
retrieval by quantized reassignments of
attention, and typical Gaussian patterns remain metastable at every load. The regimes differ in their crosstalk statistics, central-limit at
polynomial load and large-deviation at exponential load, and the class $H$ splits
retrieval into two roles, the visible Lagrangian fixing stability and the hidden one the storage scale, two axes that may also guide the design of new Lagrangians.
Read the full abstract ▾
Machine Learning
When More Is Not Better: Component Anti-Synergy in a P300 Speller
Lucas Yang, Rui Liu and Fusheng Wang
P300
brain-computer interface (BCI) spellers can provide hands-free communication for people with severe motor impairments. Modern pipelines combine multiple individually promising components, often assuming that 'more-is-better'. We tested this assumption using a four-component full-factorial experiment varying the inclusion of Euclidean
Alignment (EA), xDAWN spatial filtering, subject calibration, and
language model priors on a public P300
dataset. Performance was evaluated using accuracy, repetitions, and information transfer rate (ITR) with mixed-effects models. Results show that the value of components is conditional rather than additive. Calibration was the strongest singular contributor, while EA compensated for its absence in zero-calibration settings. Adding independently useful components could also reduce performance, revealing component anti-synergy. Contrary to conventional wisdom, LM support was not universally beneficial: its effect depends strongly on the strength of the underlying
EEG pipeline, while results from a larger LM showed a similar pattern. Together, these findings challenge maximal 'all-on' pipeline design and highlight the value of selecting spatial and
language-support components according to the quality of available
EEG evidence.
Read the full abstract ▾
Machine Learning
Measuring the Value of World-Model Updates: A Counterfactual Utility Protocol for Continual Adaptation
Anqi Peter Li and Kaden Kim
Continual world models must decide whether new data justify changing the model. Fixed replay schedules and prediction-error triggers specify when to update, but neither reveals the value of an individual update: one deployment run cannot show how the same model would have performed at that moment had it held its parameters. We introduce the fork ledger, which branches a deployment stream at pre-registered decision points into matched update and hold continuations under common random numbers. It evaluates both continuations on the same episodes and records $\Delta R = R_{\mathrm{update}} - R_{\mathrm{hold}}$. Always applying one fixed update mechanism lowers return on all three simulated control tasks: CartPole ($-144.0$; checkpoint-bootstrap $95\%$ CI $[-185.4,-116.1]$, against a converged return near $650$), Walker ($-82.8$; $[-101.1,-61.7]$) and Cheetah ($-18.6$; $[-29.0,-6.6]$). Divergence is an outcome of applying the update, so the estimand counts every attempted fork; restricted to the $693$ of $720$ that did not collapse, CartPole and Walker are unchanged in sign ($-113.4$ and $-82.1$) and Cheetah becomes unresolved ($-3.9$; $[-17.5,+13.0]$). The task is the unit of inference: each contributes $240$ attempted forks over five pretrained checkpoints crossed with two drift directions. The ledger makes counterfactual utility observable for a fixed mechanism, allowing triggers to be judged by the updates they select rather than by surprise detection alone.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CR
Empirical Evaluation of Data Poisoning Attacks in Supervised Learning
Toshif Khan and Muhammad Abusaqer
Data poisoning corrupts training data to degrade a model or to plant attacker-controlled behavior. This study evaluates two representative training-time attacks, label flipping and
backdoor poisoning, on MNIST and Fashion-MNIST with three baseline classifiers: Logistic Regression, Linear SVM, and Random Forest. Clean training is compared with poisoning rates of 5%, 10%, and 20% using clean-test accuracy, macro-precision, macro-recall, macro-F1, and, for
backdoors, attack success rate. Label flipping caused clear degradation, largest for Logistic Regression and Linear SVM, while Random Forest stayed comparatively stable.
Backdoor poisoning reached attack success rates from 0.9667 to 1.0000 on both
datasets and all three models while often keeping clean-test performance near baseline. The results separate indiscriminate poisoning, which shows up in standard metrics, from targeted
backdoor poisoning, which stays comparatively stealthy while
embedding highly effective malicious behavior, and they support security-oriented evaluation beyond conventional clean-test metrics.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
Testing Between the Test Cases: Proving End-to-End Steering in Conditions You Never Drove
Menuka Ghalan, Charles Rodgers and Zachary D. Asher
AI-based automated vehicle testing is challenging because a model that passes every test condition can still fail in the real world. Formal verification offers a way to directly address this gap. On a simulated highway and an arterial road we trained two small end-to-end steering networks each in CARLA, one on clear conditions alone and one on clear, fog, night and low sun. All four models were driven against a 2.19 ft lane-departure budget. Without driving again, we used bound propagation, a formal method that reads the trained weights, to compute how far steering can drift at every disturbance strength between two captured images. One calculation covers more than a campaign could drive: on the arterial it spans 133 poses, where ten intensities each would be 10^133 combinations, in minutes on one GPU. Not only did formal verification find conditions that broke the clear-trained policy without simulation testing, it provided some preliminary evidence for potential failures between the test cases. Our overall conclusion is that formal verification is a viable complement to simulation, and could be adopted as a part of verification and validation for automated driving.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.CL
Robust Multimodal Sentiment Analysis with Incomplete Modalities via Semantic-aware Completeness based Reconstruction
Han-Jun Choi, Byunggill Joe, Saim Shin et al. (+1 more)
Recent
multimodal sentiment analysis studies increasingly adopt text-centric fusion approaches to exploit the rich sentiment information inherent in the textual modality. However, these approaches often suffer from performance degradation during inference due to partially missing or noisy data in real-world scenarios, especially when sentiment-related cues are missing. To address this issue, we introduce a new completeness estimation approach that quantifies the degree of sentiment-relevant information preserved in incomplete data to guide the reconstruction of missing semantics. Furthermore, we propose a training strategy that stabilizes multi-task learning while jointly optimizing sentiment prediction and completeness estimation. Extensive experiments and in-depth analyses on three
benchmark datasets demonstrate that the proposed approach enables more accurate semantic reconstruction, leading to more precise sentiment prediction.
Read the full abstract ▾
All authors Han-Jun Choi, Byunggill Joe, Saim Shin, Jin Yea Jang
Machine Learning · Cross-listed from q-bio.NC
The Platonic brain bridge hypothesis: human brain networks as an architectural prior for multimodal large language models
Pengfei Zhang, Biao Tian, Xiangang Li et al. (+1 more)
Multimodal large
language models predict
brain activity, but
brain alignment has been a measurement, not a design tool. We propose the Platonic
brain bridge hypothesis: omni models,
multimodal large
language models that process
video, audio and text jointly, converge on
brain-like representations usable in both directions. From model to
brain,
brain-likeness of seven omni models is stable across participants, rises with every input channel in three bases, and our encoders lead the Algonauts 2025 out-of-distribution leaderboard. From
brain to model,
Brain-MoE fixes the expert partition of a frozen base to the seven networks of human cortex, trains experts on network-labelled
Brain-AVQA questions, raises held-out accuracy in all 15 model-
benchmark pairs by 6.42 percentage points on average and exceeds capacity-matched random experts in 14.
Brain-Scope localizes the correspondence to sparse features whose removal weakens
brain prediction. Human
brain organization is therefore a usable architectural prior for
multimodal large
language models.
Read the full abstract ▾
All authors Pengfei Zhang, Biao Tian, Xiangang Li, Li Liu
Machine Learning · Cross-listed from cs.CR
Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2
William Novak and Muhammad Abusaqer
Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model, a
privacy risk that matters in natural
language processing (NLP), where training data can contain sensitive user text. This paper presents a controlled
benchmark of membership inference vulnerability for text classification on the GLUE SST-2 sentiment
dataset. A TF-IDF + Logistic Regression pipeline and a fine-tuned DistilBERT classifier are compared under a loss-threshold MIA, with utility measured by development accuracy and macro F1. DistilBERT reached 0.9466 accuracy and 0.9460 macro F1 against 0.8756 and 0.8727 for Logistic Regression, yet both models leaked membership signal (Attack AUC 0.5615 and 0.5800, respectively). Two mitigations were tested. Stronger regularization reduced leakage for Logistic Regression at a visible utility cost, whereas fine-tuning DistilBERT for 2 epochs instead of 3 reduced leakage with negligible accuracy loss. Lightweight training adjustments can improve the
privacy-utility trade-off without complex defenses.
Read the full abstract ▾
Machine Learning
AUC Maximization from Biased Positive-unlabeled Data with Confidence
Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi et al. (+3 more)
Maximizing the area under the receiver operating characteristic curve (AUC) is a standard approach to imbalanced binary classification. Although positive and negative data are required for maximizing the AUC, negative data are often difficult to collect in some real-world applications due to
privacy concerns or the need for specialized expertise to annotate them. Thus, AUC maximization from positive and unlabeled (PU) data has been attracting
attention. Existing methods assume that labeled positive data are unbiased samples from the true positive distribution. However, this ideal assumption is often violated in practice. In this paper, we propose a method to maximize the AUC from biased PU data. To address the bias, our key idea is to exploit confidence, i.e., the probability that an instance is positive, associated with the small number of labeled positive data. We derive an
estimator of the AUC risk using biased PU data with confidence, enabling AUC maximization under such bias. We further show that the rewritten AUC risk induces a Bayes-optimal AUC ranking even when the available confidence is any strictly increasing transformation of the true posterior probability. We experimentally show the effectiveness of our method on eight real-world
datasets.
Read the full abstract ▾
All authors Atsutoshi Kumagai, Tomoharu Iwata, Hiroshi Takahashi, Taishi Nishiyama, Kazuki Adachi, Yasuhiro Fujiwara
Machine Learning · Cross-listed from cs.CL
Structurally Speaking: Motif-Oriented Graph Captioning through Bidirectional Graph-Text Translation
Hsiao-Ying Lu, Dongyu Liu and Kwan-Liu Ma
Graph captions should help readers understand graph structure, rather than simply translate adjacency
matrices into long textual edge lists. A useful graph caption abstracts connectivity into recognizable motifs, such as hubs, paths, cycles, cliques, and bridges, because these motifs provide compact structural units that are easier to read, compare, and recover. In this paper, we study motif-oriented graph captioning as a bidirectional graph-text translation task, where captions must both preserve enough
topology for graph recovery and express the graph through concise motif-level descriptions. We show that direct prompting of GPT-5.1 often produces graph-recoverable captions by enumerating node-to-node connections, but these captions are verbose and can contain inconsistent motif interpretations. To address this gap, we introduce Structurally Speaking, a lightweight structured prompting protocol that guides translation between explicit connectivity and motif-level abstraction. Experiments on a synthetic motif-based
dataset show that structured prompting produces shorter and more motif-consistent captions while maintaining comparable graph recovery. These results suggest that explicit
topology-to-motif
reasoning guidance can make LLM-generated graph captions more interpretable without model fine-tuning.
Read the full abstract ▾
Machine Learning · Cross-listed from cs.RO
ObstaDiff: Generalizable Diffusion Policy Learning via Obstacle-aware Representations
Jiawen Wang, Kevin Yao and Khalid Jawed
Imitation learning has achieved impressive results in robotic manipulation, yet most existing approaches assume clean backgrounds and lack explicit mechanisms for obstacle-aware motion generation. Extending such policies to cluttered, real-world scenes with unstructured obstacles remains a key generalization challenge. We present ObstaDiff, a decomposed
diffusion-policy framework with a lightweight obstacle-aware visual encoder. ObstaDiff extracts a structured target-obstacle-background representation, enabling the downstream
alignment policy to generate end-effector trajectories toward a target-centered bottleneck pose while
reasoning about surrounding obstacles. We evaluate ObstaDiff on 61 real-
robot greenhouse trials per method (366 executions in total). ObstaDiff achieves 75.41% average task success and 8.20% average obstacle collision rate, outperforming representative imitation-learning baselines and improving generalization in cluttered agricultural scenes.
Read the full abstract ▾