Artificial Intelligence · Cross-listed from cs.CL
DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning
Shijie Chen, Yu Gan, Yeounoh Chung et al. (+6 more)
State-of-the-art Text-to-SQL systems are typically multi-
agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains separate models for each task, failing to leverage the synergy between these interrelated tasks. In this work, we propose DualSQL, a new Text-to-SQL system consisting of two
agents powered by a single model backbone. The
agents share the same model weights and agentic scaffold, enabling joint optimization through a robust multi-
agent reinforcement learning (RL) framework. We design three database access tools to facilitate effective multi-step
reasoning grounded to interactions with the databases. To improve training and avoid model collapse, we introduce a set of rollout guardrail mechanisms that stabilizes multi-
agent RL training, supporting DualSQL to keep improving during training. We also introduce a new SQL correctness metric, robust execution match (REX), to more accurately judge SQL correctness and assign reward signals. Being trained on only 3755 examples, DualSQL-4B achieves an impressive 68.0% execution accuracy on the BIRD development set, matching previous 7B models. DualSQL-8B further improves to 71.1%, outperforming previous state-of-the-art single-model solutions with 32B parameters. These results demonstrate the strength of joint multi-
agent reinforcement learning for building high performance Text-to-SQL pipelines.
Read the full abstract ▾
All authors Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li, Sravan Babu Bodapati, Cody J. Greer, Yu Su, Fatma Ozcan
Artificial Intelligence · Cross-listed from cs.LG
Rethinking How We Evaluate Methodological Progress in Health AI
Florent Pollet and Matthew McDermott
Methodological progress in artificial intelligence (AI) for electronic health records (EHRs) depends on our ability to determine which
algorithms work better, and under which conditions. However, such progress is thought to be hindered by difficulties in reproducibility and in defining clinically meaningful evaluation tasks. We empirically study these barriers by re-implementing 12 historical and recent
algorithms within a shared evaluation framework and evaluating them on two clinical
datasets, MIMIC-IV and NWICU. We compare two complementary task families: expert-authored clinically meaningful tasks and generated tasks defined from randomly sampled event codes and prediction horizons. We ask whether relative
algorithms comparisons transfer across task families and
datasets, whether residual task heterogeneity contains useful methodological structure, and what a controlled comparison reveals about progress over the last decade. We find that aggregate pairwise comparisons transfer strongly across evaluation settings, including from randomly generated tasks to clinically meaningful tasks and across
datasets. At the same time, clinically meaningful tasks exhibit greater task-method interaction, providing preliminary evidence that task properties can help explain when particular modeling choices are advantageous. Finally, newer
algorithms do not consistently outperform earlier approaches: gradient-boosted trees remain highly competitive when paired with a modern, wide and sparse representation of the EHR. Together, these results suggest that useful methodological knowledge may require less task engineering than commonly assumed, while highlighting the importance of understanding the structured heterogeneity that remains across tasks and methods.
Read the full abstract ▾
Artificial Intelligence
Symbolic Temporal Supervision of LLM Agents Using Contracts
Yifeng Xiao and Pierluigi Nuzzo
Large
language model (LLM)
agents augmented by tools can automate complex, multi-step tasks, such as web
navigation, code generation, and workflow orchestration, by acting on external systems through tool calls. However,
hallucinations, distributional instability, and
adversarial manipulations in LLMs, and the irreversible consequences of certain tool calls can lead to harmful outcomes. Existing safeguards either grade recorded trajectories post hoc with
stochastic LLM judges or block unsafe actions one call at a time, and no single deterministic artifact supports both roles. We present ContrAgent, a contract-based framework for symbolic temporal supervision of LLM
agents. ContrAgent captures an
agent's behavior as a sequence of tool calls and formalizes it as a trace over a fixed set of checkable predicates. It then specifies required behaviors using assume-guarantee contracts in linear temporal logic over finite traces (LTLf). Each contract is compiled to a deterministic finite automaton (DFA) that serves two roles: gating
agent actions online and evaluating recorded traces offline. A contract library, acting as a reusable knowledge base, is maintained independently of the
agent's model and can be applied across different
agents within the same task domain. We show the effectiveness of our approach on four
benchmarks spanning both roles, where ContrAgent matches state-of-the-art LLM-judge and rule-based guardrail baselines while producing deterministic, reproducible verdicts and, in the online mode, orders-of-magnitude lower per-call latency.
Read the full abstract ▾
Artificial Intelligence
Designing Agentic AI Workflow Portfolios under Imperfect Selection and Compute Cost
Mojtaba Abdolmaleki, Stefanus Jasin and Boyu Wang
Agentic AI systems often approach the same task through multiple workflows that differ in
reasoning strategy, verification structure, and compute cost. A natural deployment policy is to use the workflow with the highest average performance, but this can be suboptimal because different workflows may succeed on different instances. We study a portfolio-and-selector paradigm in which a firm runs multiple workflow executions and selects the final answer after observing their outputs. Additional executions may uncover correct answers that the best standalone workflow misses, but they consume compute and introduce plausible distractors that complicate final selection. We formulate this as a workflow portfolio problem in which the firm jointly chooses run size and allocation across workflow types. We summarize selector quality through an odds-lift index and derive sharp bounds on the value of workflow variety. For finite workflow pools, we develop exact formulations, linear programming relaxations, randomized rounding procedures, and computable performance certificates. For large implicit workflow classes, we derive a finite-dimensional dual and an ellipsoid method using a pricing oracle to identify workflows with high weighted accuracy net of recurring compute cost. Under a weak condition, the method obtains a near-optimal solution to the relaxation with polynomially many oracle calls. We evaluate the framework on three
datasets: ABCD, Schema-Guided Dialogue, and HotpotQA. Relative to the best standalone workflow, portfolio optimization improves held-out selector accuracy by 3.1, 7.5, and 0.9 percentage points, respectively. Dual-guided workflow generation adds 3.5 points on ABCD and 24.1 on HotpotQA, with no additional gain on Schema-Guided Dialogue.
Read the full abstract ▾
Artificial Intelligence
AutoTuneBench: Trustworthy Measurement for Agent Auto-Tuning of LLM Serving Engines
Li Chen
Large
language model
agents tune GPU kernels and serving engines through a closed loop of propose, measure, and keep, but the measurements behind this loop are not trustworthy. We characterize four failure modes from a four-day pilot corpus of 619 model calls: strawman baselines manufacture speedups, absolute times do not transfer across machines, saturated tasks nullify comparisons, and infrastructure defects impersonate science. We present AutoTuneBench, a
benchmark and measurement protocol that makes trust architectural. The protocol is frozen as code with test-enforced provenance; a database-level validator rejects out-of-protocol results; anti-cheat checks run outside the
agent's modification surface; comparisons follow pre-registered readouts; and measurements anchor to externally published results, grounded in paired-seed statistics with a 5% cross-run coefficient-of-variation cap. Honest measurement rewrites the headlines: our best kernel reads 10.6x against a naive baseline but 2.03x against the honest one; one configuration delivers 1.174x on one machine and 1.0049x on another; a pre-registered on/off comparison nulls at a shared wall (2.4840 vs 2.4957\,ms); and the KernelBench Level-1 suite admits 51% of tasks with median speedup 1.0001x over PyTorch eager. The protocol, the two-engine corpus (vLLM and SGLang), and its audit trail are released as open artifacts.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CR
PentestChain: A Cost-Aware, MCP-Orchestrated Framework for Automated Penetration Testing with Free-Tier LLMs
Rushabh Vipulkumar Patel, Dipo Dunsin, Mohammed Almaiah et al. (+1 more)
AI-driven penetration testing has been demonstrated with premium frontier models such as GPT-4, but the per-engagement
token cost makes continuous, automated testing unaffordable for the smaller organisations that need it most. This paper presents PentestChain, a ten-phase automated penetration testing framework that couples a curated, deterministic exploit map with a cost-aware AI cascade-a local Ollama model (qwen2.5-7b) first, then free-tier OpenRouter and Cerebras, with a rule-based fallback that always produces output-and exposes the full pipeline through a Model Context Protocol (MCP) server with eleven tools. We make three contributions. First, we treat US-dollar cost per engagement as a measured, first-class evaluation metric and show that a 7B-parameter local model, kept off the critical path by a deterministic backbone, sustains end-to-end operation at zero measured paid-API cost. Second, we analyse the attack surface that an MCP-exposed offensive engine introduces, grounding a four-position threat model in the 2025 MCP incident record (the CVE-2025-6514 remote-code-execution flaw in mcp-remote, the postmark-mcp supply-chain
backdoor, and the tool-poisoning-rug-pull-line-jumping class), and contribute four mitigations. Third, we specify a reproducible, containerised evalua-tion protocol aligned with the standardised testbeds now expected at top-tier venues-AutoPenBench, a Cybench subset, and the PentestGPT 182-sub-task
benchmark-with multi-trial statistics (more than 10 trials per configuration, pass-at-k, non-parametric significance tests and effect sizes) and direct, same testbed reproduction of the PentestGPT and PentestAgent baselines rather than citation of their published numbers. On the legacy targets measured to date, the framework detected 26 services, enriched 34 CVEs, produced
Read the full abstract ▾
All authors Rushabh Vipulkumar Patel, Dipo Dunsin, Mohammed Almaiah, Mohamed Chahine Ghanem
Artificial Intelligence · Cross-listed from cs.RO
A Comprehensive Review of Generative Physical Artificial Intelligence
Satyam Gaba, Krutiksinh Rana, Siva Sai et al. (+2 more)
The integration of large-scale foundation models with physical embodiments has led to significant advancements in robotics known as Generative Physical Artificial Intelligence (GPAI). These agentic AI systems autonomously perceive, reason, and act in complex real-world situations. This survey comprehensively analyzes GPAI systems, focusing on their architectural foundations, current applications, and key limitations. We introduce a taxonomy of five distinct approaches:
Robot Foundation Models (RFMs) for cross-platform skill transfer;
Vision-
Language Action (VLA) models for end-to-end multi-modal perception and control; Large Behavior Models (LBMs) for human-like movement generation;
Diffusion Policy Models (DPMs) for
diffusion model-based temporally coherent action generation; and World Foundation Models (WFMs) for physics-compliant simulation and data generation. We examine how these approaches complement each other: WFMs generate training data for VLAs and DPMs, RFMs enable cross-platform deployment of learned policies, while LBMs provide motion priors for natural behavior. Through examples across autonomous vehicles, industrial automation, healthcare robotics, and
humanoid systems, we identify significant performance improvements and summarize promising research directions in data-efficient learning, sim-to-real transfer, edge-compatible architectures, and safety frameworks. These insights advance
embodied AI for IoT-connected environments where intelligent
agents interact with networked sensors, actuators, and edge devices.
Read the full abstract ▾
All authors Satyam Gaba, Krutiksinh Rana, Siva Sai, Vinay Chamola, Dusit Niyato
Artificial Intelligence · Cross-listed from cs.CL
Linguistic Triggers of Gender and Racial Bias in Open-Weight LLMs Applied to Recruitment
Kosuke Kitahara and Nobuhiro Yamaguchi
Open-weight large
language models are rapidly entering hiring pipelines, yet their discriminatory failure modes – and the regulatory exposure these create under the EU AI Act high-risk classification (Annex III) and U.S. EEOC adverse-impact analysis – remain poorly understood. We present the first systematic, multi-model audit of open-weight LLMs that treats job-posting
language as the primary experimental variable, evaluating six models (Llama 3.2, Mistral, Gemma 3, Qwen 3, Phi 3, DeepSeek-R1) across four controlled experiments that jointly probe recruiter-simulation and job-seeker-simulation tasks. We find that (1) agentic posting
language depresses recruiter recommendation scores for female candidates (r_rb = 0.309, p_Bonf = 7x10^-5; model-fixed-effects r_rb = 0.448), while communal
language partially reverses the penalty; and (2) coded-exclusion
language suppresses non-White recruiter scores at large effect sizes (r_rb = 0.646-0.758) and, on the job-seeker side, selectively deters non-White personas from expressing interest – operationalizing a chilling-effect mechanism at scale. A label-ablation experiment isolates the explicit demographic persona label as the primary causal driver, and Word
Embedding Association Tests corroborate these findings at the representational level (d = 1.01-1.45 under Caliskan et al.'s multi-word gender attribute lists). We translate these results into a concrete pre-deployment audit protocol – posting-vocabulary scoring, persona-conditioned LLM probing, and adverse-impact flagging against the four-fifths threshold – that operationalizes the documentation and risk-management obligations Annex III imposes on high-risk AI in recruitment.
Read the full abstract ▾
Artificial Intelligence
When Is Graph Structure Worth Its Cost? The Case for Structure Pricing in Retrieval-Augmented Generation
Yuzhong Zhang, Haoyang Ma, Chao Peng et al. (+3 more)
Graph-based
retrieval-augmented generation (RAG) can help answer questions that require information from many documents. However, building a graph often requires many
language-model calls during ingestion. It is therefore important to ask whether its quality gains justify the additional cost. We present EffiRAG, a graph-based RAG system designed to reduce this cost. It uses the graph to locate relevant passages and generates answers from the original text. This design preserves source information while keeping graph construction and query processing lightweight. We evaluate EffiRAG on UltraDomain, which contains 120 open-ended questions from four domains. Compared with LightRAG-hybrid, EffiRAG produces the preferred answer on 93 questions. LightRAG is preferred on 7, and the remaining 20 are splits. EffiRAG also reduces total system cost by 57 percent, from USD 0.952 to USD 0.408. The cost includes
language-model calls during ingestion and querying. The advantage remains as the corpus grows. At 10 and 20 documents per domain, EffiRAG uses a lightweight, non-LLM filter to skip low-salience chunks. It remains preferred over LightRAG-hybrid. It costs 4.2 times and 4.5 times less, respectively. The comparisons identify different quality-cost trade-offs. Graph-based RAG systems should therefore be evaluated by both answer quality and cost. The results favor graph structure that locates and preserves source evidence.
Read the full abstract ▾
All authors Yuzhong Zhang, Haoyang Ma, Chao Peng, Lionel Briand, Boxi Yu, Jialun Cao
Artificial Intelligence · Cross-listed from cs.LG
Agora: Git as Shared Memory for Collective AutoResearch
Yifan Zhang, Yunheng Zou, Shaokun Zhang et al. (+5 more)
Research
agents working in separate sessions need to know what others have tried and which results they can build on. Agora stores their contributions as an append-only directed acyclic graph (DAG) in Git. Each commit records a result, insight, hypothesis, verification, or report and links it to prior work. Searchable views show leading results, neglected branches, and verification status; diversity-aware recommendations suggest experiments beyond the current leaders. We report a run of nearly 12 days in which 13
language-model workers, with no assigned tasks or central planner, used Agora to solve a weight-transfer problem. Given 141 pretrained donor models and a frozen 119.6M-parameter
attention–SSM hybrid whose dimensions match no donor, the workers had to initialize the target without training data or gradient updates. They published 1,703 contributions and reduced the development evaluator score from 3.39 to 1.899 bits per byte, closing 62% of the gap to a trained GPT-2 124M. The best method compresses donor next-
token statistics into the target's
embedding and output head, then adds short-range context through sparse edits to
attention, feed-forward, and state-space blocks. Its 145-commit ancestry spans 15 accounts. Participants also posted 165 verifications of 95 targets, each by an account other than the target's author, with no reported failures. The run documents how
agents reused and verified shared work. Measuring the effect on discovery per unit of compute requires a matched comparison.
Read the full abstract ▾
All authors Yifan Zhang, Yunheng Zou, Shaokun Zhang, Jian Hu, Hao Zhang, Binfeng Xu, Jan Kautz, Yi Dong
Artificial Intelligence · Cross-listed from cs.CV
Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration
Zhixin Cheng, Jiacheng Deng, Xiaotian Yin et al. (+3 more)
Detection-free methods for image-to-point cloud registration are prone to erroneous correspondences caused by domain and modality discrepancies, limited sensitivity of feature extractors, and the presence of non-overlapping regions. The Masked Autoencoder (MAE) has shown strong performance in visual representation for images and point clouds. It may be helpful to apply this approach to image-to-point cloud registration, a task that requires unified feature extraction and accurate cross-modal correspondences. Standard MAE's random masking may overlook key regions due to limited camera views, reducing registration effectiveness. To address this, we propose the Intermodal Dual-MAE Framework (ID-MAE) with a Similarity-based RL Masking Strategy (SRLM), which adaptively masks informative positions by leveraging cross-modal similarity and reinforcement learning, thus narrowing the modality gap. Our method enhances cross-modal representation learning by enforcing representation consistency during feature extraction, thereby enabling more reliable 2D-3D correspondence estimation. Experiments on RGB-D Scenes v2 and 7-Scenes
benchmarks show that our method achieves state-of-the-art performance in image-to-point cloud registration.
Read the full abstract ▾
All authors Zhixin Cheng, Jiacheng Deng, Xiaotian Yin, Baoqun Yin, Richang Hong, Tianzhu Zhang
Artificial Intelligence
Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
Devesh Tiwari, Camille Davis, Shivank Sinha et al. (+3 more)
Linear probes can decode safety-relevant concepts such as truthfulness from
language-model activations, but probe accuracy may show only decodability, not that the features the probe weights causally drive model behavior. We demonstrate that this gap cannot be closed from the geometry of probe weights alone: the features geometrically aligned with probe direction need not be the ones the model uses, so causal relevance requires intervention. We introduce a feature-level diagnostic that decomposes a deployed True/False probe into sparse-autoencoder (SAE) features, ranks those features by both probe
alignment and by gradient sensitivity of the model's behavior, and ablates the resulting shared, probe-only, and random feature sets under a coherence gate. On the truth probe of Buerger et al. (2024) (TTPD), applied in the instructed truth/deception setting of Long et al. (2025) for Gemma2-9B-Instruct, the two rankings overlap only weakly (about 12%, Spearman rho = 0.10), and ablation dissociates them sharply: features the probe shares with the model flip the output far more (up to 27%) than equally sized probe-only (6%) or random (1%) features at full coherence, while probe-only features instead perturb the probe's own readout. The dissociation holds across five seeds and a held-out split, and an activation-aware selection of features flips behavior nearly three times as often as the probe's
geometric top features (17.6% vs. 6.1%). In this setting, therefore, the
geometric projection of a probe's weight vector alone does not identify the features the model causally uses; however, combining probe information with feature activation statistics recovers substantially more behaviorally causal features, and coherence-gated SAE intervention is needed to separate them from probe readouts.
Read the full abstract ▾
All authors Devesh Tiwari, Camille Davis, Shivank Sinha, Talia Weaver, Aditya Shah, Maheep Chaudhary
Artificial Intelligence
Teaching AI, Robotics, & Community: A Hubs-Based K-12 Education Framework for Reaching Rural Schools
Maxwell J. Jacobson, Gustavo Rodriguez-Rivera, Petros Drineas et al. (+1 more)
K-12 robotics and AI education remains difficult to scale, especially in rural regions lacking sustained technical mentorship. Programs like FIRST provide competition pathways and instructional opportunities, but they do not eliminate the need for local programming and robotics expertise. We introduce AI, Robotics, & Community (ARC), a hubs-based framework where colleges train undergraduate mentors and host workshops for nearby K-12 teams. Mature school programs can become secondary hubs that support additional schools, creating a self-reinforcing education loop where mentorship reach propagates geographically and can even grow super-linearly. We first evaluate ARC through a trial deployment at one university. The trial created three rural robotics teams. On five-point Likert surveys, mean increases in K-12 programming knowledge, resource access, and practice opportunities were 2.00, 2.25, and 1.25 points. Likewise, undergraduate confidence teaching technical concepts, adapting explanations, managing groups, and finding mentoring enjoyable and meaningful increased by 1.29, 1.14, 1.00, and 1.14 points. Additionally, we create a spatial Markov model of ARC's growth and simulate it using the state of Indiana as a testbed. Under moderate conditions, we find that ARC reaches 74% of Indiana's 1,925 public K-12 schools and produces 992 robotics programs after 40 years, compared with 161 projected under natural growth alone. Together, these results show ARC can create and support rural robotics programs, train undergraduate AI and robotics mentors, and potentially scale mentorship across a region.
Read the full abstract ▾
All authors Maxwell J. Jacobson, Gustavo Rodriguez-Rivera, Petros Drineas, Yexiang Xue
Artificial Intelligence · Cross-listed from cs.CL
From a River in Gilead to the Inference Distributions of Large Language Models: Covert Dialect Bias and Linguistic Profiling at Scale
Chowdhury Mohammad Abdullah and Rita Orji
Large
language models (LLMs) are increasingly deployed in high-stakes domains such as housing screening. While
alignment techniques mitigate explicit racial bias in generated text, they often leave covert attitudinal associations in internal probability distributions untouched. Adapting the matched-guise sociolinguistic paradigm, we examine covert dialect bias in housing-related social judgments across four varieties: Standard American English (SAE), African American Vernacular English (AAVE), Nigerian Standard English (NSE), and Nigerian Pidgin (NP). AAVE reflects the racialized dialect studied in prior covert-bias evaluations, whereas NSE and NP represent Black African, postcolonial varieties absent from this literature. Using 260 meaning-matched sentence quadruples and log-probability scoring over housing-relevant adjectives, we probe ten open-weight LLMs across three contexts varying in social proximity: tenant screening, neighbor acceptance, and roommate selection. Across all ten models, AAVE and NP are consistently associated with more negative adjectives than SAE, with NP penalized most severely. Crucially, each dialect is penalized via distinct stereotype clusters rather than a generic non-standard category. NSE, which carries institutional prestige, displays a context-dependent shift: favored over SAE in formal tenant screening but increasingly penalized as social proximity grows. Our findings reveal that LLMs inherit covert dialect bias along both racial identity and prestige dimensions, echoing documented human housing discrimination and demonstrating its reach across postcolonial English varieties.
Read the full abstract ▾
Artificial Intelligence
The Other Half of the Memory Wall: Serving 35B MoEs from SSD with Trained Routing Prediction
Yu Lin, Yiming Wang, Runyuan Cai et al. (+2 more)
Mixture-of-experts (MoE) inference on consumer hardware is bounded by weight
memory: a 35B-class model is 19.5GB at 4-bit, and sparsity shrinks the compute per
token, not the bytes that must be held. Naive offloading to SSD does not help on its own, because layer N+1's experts must be chosen before layer N's output exists, so the reads cannot start early enough to hide behind compute. We present Edge0, a streaming MoE inference engine that closes the gap with a prerouter: a per-layer head predicts the next layer's routing one
token ahead, and the prediction is consumed as the routing itself, so the staged expert set equals the routed set and nothing is dropped. An unmerged recovery LoRA, trained on the student path, pays back the quality lost to int4 quantization and routing replacement. On a single 24GB machine, Edge0 serves a 35B MoE at 20tok/s inside 3GiB of peak active
memory, within a few points of its fp16 teacher on average across five public
benchmarks. An 8B tier runs on the same framework, and the framework, checkpoints, and adapters are open source.
Read the full abstract ▾
All authors Yu Lin, Yiming Wang, Runyuan Cai, Hanze Liu, Xiaodong Zeng
Artificial Intelligence
Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning
Xinxin Song, Siyuan Li, Tingxiong Xiao et al. (+1 more)
Reinforcement learning with verifiable rewards (RLVR) has significantly improved the
reasoning capabilities of large
vision-
language models (LVLMs). However, standard on-policy RLVR
algorithms face a critical optimization bottleneck in preserving and reinforcing visually grounded
reasoning behaviors: valuable visually-grounded
reasoning trajectories are discarded after a single update, while uniform
token advantage allocation prevents the model from reinforcing critical perception or
reasoning steps. To bridge this gap, we propose PIVOT, a dual-level learning framework that anchors policy optimization around informative visual
reasoning signals. Specifically, PIVOT introduces a self-calibrated experience replay mechanism, which selectively collects and replays visually-grounded historical experiences as stable reference anchors for policy optimization. Building upon this, we further design a
vision-guided advantage allocation mechanism to allocate additional
vision-aware advantages to
tokens based on their local visual support and impact on downstream
reasoning. Extensive experiments across diverse
benchmarks demonstrate that PIVOT achieves highly competitive performance in enhancing the
multimodal reasoning capabilities of LVLMs.
Read the full abstract ▾
All authors Xinxin Song, Siyuan Li, Tingxiong Xiao, Jinli Suo
Artificial Intelligence · Cross-listed from cs.SE
An Empirical Evaluation of Cost-Efficient Large Language Models on Algorithmic Programming Tasks
Chandimal Adikari and Nandika Herath
This study empirically evaluates whether cost-efficient Large
Language Models (LLMs) can be trusted to generate enterprise code to a written specification. Three models (Gemini Flash 3, GPT-5.4 mini and Claude Haiku 4.5) were asked to solve 992 algorithmic problems as Java Spring Boot service methods conforming to a mandated signature and data-transfer-object specification, crossing four model and agentic coding tool combinations with two
prompt variants to yield eight configurations, with iteration forbidden and hardcoded answers explicitly prohibited. Eight problem statements were withheld to probe how models respond to missing input. The 7,593 resulting methods were classified by an eight-class outcome taxonomy describing what each does about producing an answer, then deployed and executed, giving 7,936 measured requests joined to that classification. Structural conformance approached ceiling, yet 38.4% of methods do not compute the value they returned and only 12.9% of returned answers were correct. Conditioning on outcome class shows that response reliability and correctness are inversely related, whereas genuinely computing methods answered least often and were correct 19.3%. Limitations include single generation runs per configuration, partial harness coverage, single-pass timing, syntactic classification, and probable corpus contamination.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from astro-ph.SR
Physics-Informed Neural Networks for Fast Multilayer Spectral Inversion of H{\alpha} 6562.8 A and Ca II 8542.1 A Spectra
Ziyang Zhang, Qin Li, Vasyl B. Yurchyshyn et al. (+4 more)
Strong chromospheric absorption lines such as H$\alpha$ 6562.8 A and Ca II 8542.1 A provide vital diagnostics of
plasma dynamics and thermal structure in the
solar chromosphere. Multilayer spectral inversion (MLSI) offers a physically interpretable framework for modeling these lines using a finite number of radiative-transfer layers, but conventional MLSI relies on pixel-by-pixel nonlinear least-squares fitting, making it computationally expensive for large imaging spectroscopic data sets. Here, we introduce a physics-informed neural-network (PINN) framework to accelerate MLSI while preserving its analytic radiative-transfer formulation. The network predicts MLSI parameters directly from observed line profiles and passes them through a differentiable MLSI forward model to synthesize spectra. Training follows a two-stage approach: an initial stage optimized solely via spectral reconstruction loss, followed by fine-tuning that combines spectral consistency with parameter-space supervision from conventional MLSI results on a single reference image. This strategy eliminates the need for large precomputed training sets while maintaining physical interpretability. Applied to Fast Imaging
Solar Spectrograph (FISS) observations from the Goode
Solar Telescope (GST) targeting both quiet-Sun and active-region regions, MLSI-PINN parameter maps reproduce the primary spatial structures of direct inversions, achieving an arithmetic mean pixel-wise Pearson correlation coefficient of 0.933 across all evaluated parameters. The reconstructed spectra closely match both observed profiles and conventional MLSI fits. Post-training, MLSI-PINN processes a raster in approximately 5-15 seconds compared to 3-5 minutes for conventional MLSI, delivering an inference speedup of about 12-60 times without substantial loss in reconstruction quality, enabling efficient MLSI analysis on large chromospheric data sets.
Read the full abstract ▾
All authors Ziyang Zhang, Qin Li, Vasyl B. Yurchyshyn, Kangwoo Yi, Haimin Wang, Wenda Cao, Bo Shen
Artificial Intelligence · Cross-listed from cs.CV
Newer Is Not Fairer: Gender Stereotyping in Text-to-Image AI Across Model Generations
Shesh Narayan Gupta and Nik Bear Brown
Text-to-image generative models are widely used in professional and creative settings, yet how they represent gender across occupations – and whether newer models are fairer – remains poorly understood across multiple generations. We evaluate gender representation across 20 occupations, 5
prompt templates, and 4 Stable
Diffusion model generations (SD 1.5, SD 2.1, SDXL, SD 3 Medium), generating 8,000 images with n = 100 per occupation-model
cell (5
prompts x 20 images), and classifying all with DeepFace. Across the 8,000 open-source images, 76.4% show male subjects (95% CI [75.1%, 78.7%], p < 2.2 x 10^-16, Benjamini-Hochberg adjusted). More strikingly, 57.6% of images for historically female-coded occupations show male subjects (raw p = 3.43 x 10^-22, BH-adjusted p = 1.71 x 10^-21). All nine significant tests reported in this paper survive BH correction across 10 tests. When compared against U.S. Bureau of Labor Statistics workforce data, models underrepresent women by 20-46pp on average, with particularly large deviations for near gender-balanced occupations: scientist (48% female in BLS, 82-99% male in model outputs) and cleaner (46% female in BLS, 80-92% male in outputs). Model generations do not improve steadily: bias worsens from SD 1.5 to SDXL before partially recovering in SD 3 Medium. A preliminary comparison with GPT-image-1 on five occupations suggests lower bias than open-source models, though the practical effect is small (Cramer's V = 0.080) and the comparison is exploratory. No model achieves gender parity.
Read the full abstract ▾
Artificial Intelligence
Missing Bridges: Composition-Aware Active Imitation Learning
Maxwell J. Jacobson, Ahmed H Qureshi and Yexiang Xue
Active imitation learning reduces expert effort by allowing a learner to request the demonstrations it needs. Existing methods typically select these requests for their expected information gain about the expert policy. In structured multi-task domains, however, the number of start-goal tasks may grow combinatorially despite their solutions sharing reusable behavior. This makes composable behaviors especially valuable, since a single demonstration may help solve many tasks at once. Prior methods do not explicitly account for this value when selecting which demonstration to request. We introduce Adaptive
Agents via Latent
Topologies (AALT), which requests demonstrations that maximize expected gains in start-goal connectivity. We further show that this objective is formally tied to information gain about task reachability. AALT organizes existing demonstrations into a
topology of latent hub states connected by learned behaviors, identifies high-value bridge demonstrations that are likely to enable many tasks at once, and grounds each to an expert query. At inference, it plans through the resulting
topology and conditions a
diffusion policy on each successive hub transition. In a simulated UR5e
robot ordered-
retrieval domain with 72 tasks, AALT improved from 42/72 to 72/72 (100%) successful tasks consistently using only 3 demonstrations totaling 5 transitions beyond the initial
dataset. After 20 demonstrations, the strongest baseline averaged 88.6% success using 98 transitions.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
The Attention Within: Consensus Dynamics in Selective State Space Models
João Pedro Silvestre, Álvaro Rodríguez Abella and Paulo Tabuada
Selective state space models (SSMs) have recently emerged as a compelling alternative to
transformers, combining competitive performance with substantially improved inference efficiency. At each SSM layer, a sequence of hidden states are propagated by a recurrence, mixing information of different
tokens. Despite using a different mechanism, this mixing plays a role analogous to
attention in
transformers. In fact, recent works have shown that the two architectures may be closer than they first appear, as this recurrence admits a formulation akin to linear
attention. In
transformers,
attention is known to drive the
tokens to cluster, i.e., to reach consensus, collapsing in the limit to a single direction. Thus, we ask: does the recurrence at the core of SSMs drive the
tokens to consensus, as
attention does in
transformers? To answer this question, we take a dynamical systems perspective on SSMs, modeling the
evolution of
tokens across layers as an ordinary differential equation. By exploiting input-to-state stability arguments, we establish local exponential stability of the consensus
equilibria and characterize their domain of attraction for time-varying weight
matrices, a setting not addressed by previous results. We thereby show that the resemblance between SSMs and
transformers does run deeper: the recurrence at the core of SSMs aggregates
tokens just as
attention does. Numerical experiments on a pretrained Mamba-2 model point to the output gate as the component that regulates the extent of this consensus, preventing the
tokens from reaching it in full.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from econ.GN
Whom Do AI Agents Work For? Role Assignment Induces Sponsorship Bias in LLM Recommenders
Davood Wadi and Yu Ma
Large
language models (LLMs) now serve as conversational shopping assistants on platforms that also sell advertising. These AI
agents face a conflict of duty. They advise consumers who rely on their judgment, yet are deployed by platforms that benefit when sponsored listings are chosen. Sponsorship disclosures, designed to allow consumers to penalize paid placements, now reach the AI
agent rather than the consumer, and the
agent's evaluation of them is hidden from the consumer. Drawing on the fiduciary concept of conflict of duty, we argue that an
agent's evaluation of a sponsored listing should not depend on which party deployed it. In controlled choice experiments, we manipulate assigned roles in the system
prompt to name either a traveler or a booking platform as the
agent's principal. Platform delegation significantly attenuates the penalty that
agents apply to sponsored listings and weakens the skepticism that disclosure triggers in their
reasoning traces. We replicate out findings across LLMs and
reasoning depths. A second study decomposes the disclosure label and shows that the divergence between the two delegates widens significantly when the paid placement is attributed to the platform. Stricter terminology ("Sponsored" instead of "Promoted") lowers choice of paid listings but does not close this gap when the platform is named. The findings show that disclosure mandates designed for human consumers cannot by themselves protect consumers in AI-mediated commerce.
Read the full abstract ▾
Artificial Intelligence
Multimodal Conditioning of Fine-Tuned Stable Diffusion XL for Controllable and Culturally Faithful Ulos Motif Generation
Humasak Simanjuntak, Tamara Yunika Sianipar, Bronson T.M Siallagan et al. (+2 more)
The traditional Batak Ulos weaving industry faces growing challenges in producing diverse, innovative motifs due to limitations in conventional, manually driven design methods. This study proposes a
multimodal generative framework integrating a fine-tuned Latent
Diffusion Model (Stable
Diffusion XL v1.0 via LoRA) with a
Multimodal Large
Language Model (LLaMA 1.5-7B) to enable controllable, culturally faithful Ulos motif generation. Four complementary conditioning mechanisms: text, image, representation, and semantic map (via ControlNet) jointly guide the generation process, each governing a distinct aspect from semantic intent to spatial layout. A five level ablation study across three scenarios (shape transformation, colour variation, and high-complexity input) shows that conditioning effectiveness is not proportional to the number of mechanisms combined: Text + Image + Semantic Map achieved the best FID (270) and CLIP Score (0.65 - 0.70) but the weakest SSIM (0.65), while Text + Image + Representation offered the best overall balance, with stable SSIM (0.84) and competitive FID (280). Combining all four mechanisms yielded the weakest FID (330), indicating conflicting optimization signals. Qualitative evaluation by nine weavers and thirty public participants confirmed statistically significant positive acceptance (Wilcoxon, p=0.007 and p<0.001, respectively). A web-based prototype supporting text-to-image and image-to-image generation was also developed, offering a practical digital design tool for cultural heritage preservation.
Read the full abstract ▾
All authors Humasak Simanjuntak, Tamara Yunika Sianipar, Bronson T.M Siallagan, Difya Laurensya Ambarita, Arlinta Barus
Artificial Intelligence
RideWay: Benchmarking Efficient Task Completion for Tool-Using Language Agents
Qingnuan Han, Boli Fang, Mingzhi Hou et al. (+1 more)
AI
agents are usually evaluated by whether they complete a task. In interactive service settings, a successful
agent can still frustrate users by asking repeated questions, performing redundant searches, or making avoidable revisions. We introduce RideWay, an efficiency-centered
benchmark for ridehailing
agents in a stateful tool-calling environment, together with Efficiency Utility, a success-gated metric that discounts successful trajectories for excess tool calls and user-facing turns relative to task-specific reference effort. Human paired preferences calibrate the relative penalties, reflecting an aggregate service-workflow trade-off: extra dialogue often creates visible friction, whereas extra tool use can sometimes verify constraints or preserve user intent. Across 58 tasks and 24 models, the fitted penalty for excess turns is about twice that for excess tool calls. On task-disjoint held-out preferences, Efficiency Utility achieves 78.7% accuracy overall: 90.6% when trajectories differ in turns, but chance-level accuracy when they differ solely in tool calls - the axis on which human annotators agree least. RideWay therefore makes interaction efficiency measurable alongside task success, while exposing the boundary of count-based tool-use evaluation.
Read the full abstract ▾
All authors Qingnuan Han, Boli Fang, Mingzhi Hou, Claire Liu
Artificial Intelligence
TuiML: Machine Learning for AI Agents
Nilesh Verma, Nick Lim, Albert Bifet et al. (+1 more)
Machine-learning libraries such as Weka and scikit-learn were designed for human programmers.
Language-model
agents now use these same libraries by recalling APIs from
memory and writing code, an approach that hides what a library offers, delays errors until runtime, and loses experimental state between turns. We present TuiML, a self-contained machine-learning library built for AI
agents, with native
algorithms across supervised, unsupervised, time-series, data handling, tuning, and evaluation tasks. Every component describes itself through machine-readable metadata and parameter schemas, so an
agent can search the library, inspect components, compose validated workflows, and register new ones that become discoverable in turn. Every call is validated, seeded, and traced, and sessions export as runnable notebooks, making experiments reproducible by construction. One specification layer drives the Model Context Protocol (MCP),
agent-framework adapters, a Python API, a CLI, and local model serving, while data and models never leave the machine.
Benchmarks show TuiML remains predictively competitive with scikit-learn and Weka. While looking like a conventional library to a human user, TuiML is designed for
agents first, allowing them to read, extend, and operate machine learning autonomously. TuiML is open source, with documentation at https://tuiml.ai.
Read the full abstract ▾
All authors Nilesh Verma, Nick Lim, Albert Bifet, Bernhard Pfahringer
Artificial Intelligence
Contiguity, Not Importance: Budgeted Repair of Stale KV Caches After Document Edits
Mingyang Mao, Wyatt Mackey and Xiaomin Lin
KV-cache reuse can reduce inference cost in
retrieval-augmented generation and agentic systems, but cached contexts may become stale when retrieved knowledge, working
memory, or user state is edited. Under causal self-
attention, even a local edit can affect downstream KV states. A full re-prefill reliably restores consistency but is costly, whereas refreshing only the edited span can leave downstream dependencies stale. We formulate in-place repair as budgeted recomputation and compare training-free position-selection policies on a factual RAG
benchmark with matched direct and derived edits. Across three model families, all policies repair direct cases, but derived cases clearly separate them. At the primary budget, a contiguous edit-local window recovers at least 0.94 of the post-edit answer margin and substantially outperforms
attention-based, KV-deviation, and structural selectors. Mechanistic analysis shows that position sets effective under clean-state transplantation can fail under actual recomputation because scattered positions inherit surrounding staleness. The edit-local advantage also depends on adjacency and largely disappears when the answer-bearing text moves downstream. Because answer-relevant edits almost always corrupt model behavior, failure severity is difficult to predict, and repair is 13-21 times faster than full re-prefill, our results support unconditional edit-local repair when the dependent text remains adjacent to the edit.
Read the full abstract ▾
Artificial Intelligence
When to Call an LLM: A Confidence-Gated Hybrid for Cost-Effective Emotion Recognition in Conversational AI
Sai Babu Udayagiri, Arjun Chouhan, Ravisekhar Kanagala et al. (+1 more)
Emotion recognition in conversation (ERC) is a production capability behind
agent-assist
prompts, escalation routing, and post-call analytics in contact-center-as-a-service (CCaaS) platforms, where cost and latency constraints matter as much as accuracy. We report a systems-level comparison of three deployment options for dialogue-contextual ERC: a low-cost stacked ensemble (sentence
embeddings, windowed context, RandomForest/XGBoost/logistic-regression stacking), off-the-shelf LLM prompting (GPT-4o-mini; zero-shot, few-shot, chain-of-thought), and a confidence-gated hybrid that escalates only the ensemble's least-confident predictions to the LLM - modeled on IVA-to-human-
agent escalation policies used in production contact centers. On IEMOCAP, the ensemble significantly outperforms every LLM configuration (0.595 vs. 0.460-0.536 weighted F1, p < 0.0001) at a fraction of the cost and sub-10ms latency; on MELD and CMU-MOSI the ranking reverses, showing neither pure system is a safe default. The confidence-gated hybrid resolves this by Pareto-dominating both pure systems on all three
datasets (0.620, 0.643, 0.824 weighted F1) while routing the majority of traffic through the near-zero-cost ensemble, translating to roughly $10-85 per million utterances versus $99-170 for an LLM-only pipeline. The escalation policy is not an opaque cost/accuracy dial: escalated turns disproportionately follow an
emotion or sentiment shift, giving operators an interpretable, auditable routing signal, and the ensemble's confidence is well-calibrated and safely under- rather than over-confident. The pattern holds across three
datasets and two LLM providers. Confidence-gated cascading is established in general ML systems; our contribution is showing it transfers cleanly to dialogue-contextual ERC, yielding a concrete deployment recipe for CCaaS and conversational-AI platforms deciding how to allocate LLM spend.
Read the full abstract ▾
All authors Sai Babu Udayagiri, Arjun Chouhan, Ravisekhar Kanagala, Trishala Pavagada
Artificial Intelligence
Memory Has Geometry: Non-Uniform Geometric Memory for Long-Horizon Personalized AI
Jiahong Liu, Wenhao Yu, Zexuan Qiu et al. (+2 more)
Long-term
memory is becoming a core substrate for personalized AI, yet most systems still represent personalization as discrete records in a largely static latent space, accessed under one global similarity notion. For data mining, this creates a mismatch: the evidence is a temporal event stream, while the dominant abstraction is a searchable record set. We argue that long-horizon personalization should instead model
memory as a user-specific dynamical state space with locally heterogeneous geometry. Geometry here is a computational
language, not a literal claim about cognition: it captures stable versus volatile regions, variable-rate drift, heterogeneous neighborhoods, and uncertainty about current user state. Profiles and isolated events remain useful as points, but interaction, feedback, and elapsed time induce trajectories.
Memory access then becomes trajectory-conditioned reconstruction of the relevant user state, not only nearest-neighbor lookup.
Read the full abstract ▾
All authors Jiahong Liu, Wenhao Yu, Zexuan Qiu, Menglin Yang, Irwin King
Artificial Intelligence
Measuring AI Leadership: Development and Validation of a Multidimensional Measure for AI-Native Organizations
Mustafa Akben and Leslie Coyne
AI is changing what leaders must judge, explain, learn, and coordinate, yet existing measures do not capture these behaviors at the level needed to study leadership in AI-enabled work. We develop the AI Leadership Battery, which organizes 36 behaviorally specific subdimensions into 11 theory-specified content families. Following established scale-development procedures, the research used deductive item generation; content validation of definitional correspondence and definitional distinctiveness; exploratory factor analysis and item reduction; confirmatory factor analysis in independent samples; and tests of internal consistency reliability, convergent validity, discriminant validity, and criterion-related and incremental validity. Across the development and validation studies, the analyses provided evidence for the Battery's content, multidimensional structure, reliability, and distinction from selected orbiting constructs. The Battery also contributed additional information beyond orbiting constructs across organizational growth, decision speed, customer/stakeholder response capability, AI-enabled team performance, AI-enabled work experience, AI security and risk management, and AI adoption and integration. The resulting measure provides researchers with a behavioral framework for examining how leaders in AI-enabled work regulate judgment, learning, adaptation, transparency, and accountability.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
EDCT-Bench: Uncovering Faithfulness Gaps in VLMs via Explanation-Driven Counterfactual Testing
Sihao Ding, Santosh Vasa, Aditi Ramadwar et al. (+1 more)
Vision-
Language Models (VLMs) can produce Natural
Language Explanations (NLEs) that sound plausible yet remain inconsistent with the visual evidence they cite. We present Explanation-Driven Counterfactual Testing (EDCT), an intervention-based protocol that extracts visual concepts cited in a model's explanation, applies verified minimal edits to them, and tests whether the resulting answer and explanation remain consistent with the edited image. Using this protocol, we create EDCT-Bench, a comprehensive
benchmark spanning three complementary domains: knowledge-intensive visual question answering (OK-VQA), safety-critical driving (DriveLM), and 3D spatial
reasoning (3DSRBench). Across the evaluated VLMs, EDCT reveals substantial faithfulness gaps, with models frequently producing responses inconsistent with verified visual changes. Finally, our fine-tuning study suggests that EDCT-generated counterfactuals provide high-impact training signals.
Read the full abstract ▾
All authors Sihao Ding, Santosh Vasa, Aditi Ramadwar, Thomas Monninger