Artificial Intelligence · Cross-listed from cs.IR
Edge-Based Agentic Retrieval-Augmented Generation for Autonomous FHWA Bridge Inspection Compliance
Viraj Nishesh Darji and Hemaliben Rakeshkumar Darji
The Federal Highway Administration (FHWA) mandates that over 600,000 bridges in the United States be evaluated against the Recording and Coding Guide for the National Bridge Inventory (NBI). Manual compliance verification is labor-intensive, error-prone, and impractical in connectivity-limited field environments. This paper introduces BridgeGuard, a fully air-gapped agentic
Retrieval-Augmented Generation (RAG) system for autonomous bridge inspection compliance. BridgeGuard integrates vector search over the FHWA Recording and Coding Guide with structured SQL queries against NBI tabular data, orchestrated by a stateful multi-step ReAct planning loop executing locally on commodity edge hardware. A section-aware chunking
algorithm preserves hierarchical regulatory item boundaries, achieving 94.2% chunk integrity compared with 28.4% for naive fixed-size splitting. Evaluated on the full Delaware 2023 NBI inventory (874 bridges) and a Texas sample (200 bridges), the system achieves 99.77% and 100.0% classification accuracy, respectively, for Structurally Deficient bridge identification, with 100.0% citation accuracy, at 197.0 bridges per hour with out external network access. Ablation experiments confirm that both vector search and the multi-step agentic loop are necessary for correct compliance
reasoning.
Read the full abstract ▾
Artificial Intelligence
Solving Is Not Drawing: A Benchmark for Diagrammatic Reasoning in Olympiad Geometry
Hsien Xin Peng, Anthony Kim, Alvin Li et al. (+3 more)
Foundation models such as GPT and Claude now solve olympiad-level mathematics with remarkable proficiency, so much so that geometry problem solving has become a standard proxy for their mathematical
reasoning. Yet solving a geometry problem and drawing the figure it depends on are not the same skill: progress often hinges on a faithful diagram with the right auxiliary constructions and incidences, and it is unclear that a model which reasons its way to the answer can also produce one. A growing collection of
benchmarks, including MathVista, and MathVerse, measures whether models reach the correct answer, but to our knowledge, none isolate the distinct ability to construct the diagram itself, leaving this capability unmeasured. We introduce an open-source
benchmark that targets this gap: 954 self-contained olympiad geometry problems, with a 297-problem hard subset, each paired with its solution and a human-authored, high-fidelity diagram in renderable Asymptote code, together with a suite of text-, code-, image-, VLM-, and constraint-based metrics for what we term diagrammatic
reasoning. Evaluating current foundation models reveals a pronounced gap between solving and drawing: their diagrams are markedly less faithful, with an average compile success rate of only 36.14%. Strong mathematical
reasoning, we find, does not imply the ability to construct accurate
geometric diagrams. Our
benchmark and
dataset can be accessed at https://huggingface.co/
datasets/max98765/hard_geometry_problems_with_diagrams.
Read the full abstract ▾
All authors Hsien Xin Peng, Anthony Kim, Alvin Li, Calvin Supasanya, Shivank Garg, Kevin Zhu
Artificial Intelligence · Cross-listed from cs.CL
When Chain-of-Thought Helps and When It Hurts: An Empirical Investigation of the Serial-Depth Bottleneck in LLM Reasoning
Tughanbulut Kurtulush
It is widely assumed that chain-of-thought (CoT) prompting universally improves LLM
reasoning. We investigate this through the conceptual framework of the H_dp bandwidth bound (Chen et al., 2024): although the formal bound binds only asymptotically (at astronomically large
prompt lengths), it identifies a real architectural bottleneck – serial computation exceeding a
transformer's single-pass capacity must be externalised, which is what CoT does. Our central finding is a within-
benchmark serial-depth gradient: single-pass (no-CoT) accuracy degrades monotonically with per-item serial depth, while CoT is approximately depth-invariant. We measure CoT effects across three instruction-tuned models (Qwen-2.5-7B/32B, Llama-3.1-8B) and five standard NLP
benchmarks at practical context lengths. On high-depth P-complete tasks (GSM8K, MATH), CoT gives a +54 to +68 pp recovery gap across all models. On shallow TC^0 tasks (MMLU, ARC), CoT is structurally redundant (Delta in [0.0, +4.6] pp, no significant negative effect) – though high no-CoT baselines (up to 95% on ARC) may reflect contamination, so this null is not a clean architectural test. The intermediate class L (HumanEval) shows a model-size-dependent transition: +23.2 pp (32B), +9.1 pp (8B), -28.7 pp (7B). The cross-
benchmark depth-recovery correlation is Spearman rho = 0.661 (p = 0.007, n = 15); 9 of 15
benchmark-level McNemar tests are significant after Bonferroni correction. Pre-registered on OSF, our results indicate that CoT is not a universal
reasoning enhancer but acts as a bandwidth bypass: it helps serial computation that strains single-pass capacity and is redundant for tasks that already fit.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.HC
Innovating with Generative AI: A Human Bottleneck Framework
Julian De Freitas, Ayelet Israeli, Gideon Nave et al. (+2 more)
We propose a human bottleneck perspective for understanding how generative AI transforms the innovation process. The central premise is that many constraints traditionally plaguing the innovation process are cognitive and social in origin, rooted in how people generate ideas, evaluate novelty, and communicate through social systems. Generative AI does not act uniformly on these constraints. At each stage, it can deepen some bottlenecks while alleviating others, and predicting these outcomes requires understanding the underlying mechanisms of the constraint itself. We identify bottlenecks in four stages of the innovation process: ideation, screening and testing, preference measurement and consumer insight,
diffusion, and
market learning. By grounding analysis in human behavior rather than rapidly changing AI capabilities, we offer a framework for assessing whether new developments alleviate or intensify the bottlenecks that matter most at each stage. We also distinguish bottlenecks likely to narrow as capabilities improve from those rooted in enduring human constraints. We further discuss AI-related issues that cut across the entire innovation pipeline, challenging the very existence and structure of the traditional innovation process.
Read the full abstract ▾
All authors Julian De Freitas, Ayelet Israeli, Gideon Nave, Artem Timoshenko, Olivier Toubia
Artificial Intelligence · Cross-listed from cs.CY
Governing Execution Risk in Agentic AI Systems: A Trajectory-Guided Framework for Red Teaming
Zhihao Zhu and Yi Yang
AI
agents are increasingly embedded in organizational workflows, where they interact with external information sources and invoke digital tools to perform operational tasks. As organizations adopt such systems, a critical challenge is identifying and mitigating risks arising from malicious or untrusted external information that can steer
agents toward unintended actions. Existing red-teaming approaches largely rely on fixed attack templates or final attack outcomes, providing limited visibility into how attacks unfold through multi-step
reasoning and tool use. We argue that
agent execution risk should be understood as a trajectory-level phenomenon. Building on this perspective, we propose TrajRed, a trajectory-guided red-teaming framework that uses execution trajectories to uncover vulnerabilities in agentic AI systems. We further develop TrajGuard, a runtime governance layer that uses high-risk trajectories discovered during red teaming to monitor and intervene in ongoing workflows. Experiments on AgentDojo across four organizational task suites show that TrajRed identifies substantially stronger vulnerabilities than fixed-template and automatic red-team baselines. Building on these vulnerability findings, TrajGuard reduces attack success across all evaluated attack methods to near zero while preserving benign task utility. Together, the results demonstrate that execution trajectories provide a practical foundation for both red teaming and risk control in agentic AI systems. This work highlights the importance of governing
agent execution in organizational AI deployments.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CY
AI-driven Multimodal Representation Learning for Latent Mediation Structure Discovery of Socioeconomic Disadvantage, Psychosocial Factors, and Cardiometabolic Multimorbidity: Insights from the All of Us Research Program
Cong Cao and Shuangge Ma
Social disadvantage is associated with multimorbidity, but the pathways linking social conditions to disease burden remain poorly understood. We developed an AI-driven
multimodal mediation framework that integrates socioeconomic, psychosocial, clinical, laboratory, behavioral, and genomic data from the All of Us Research Program. Modality-specific variational autoencoders were used to derive latent representations of each data domain, and mediation analyses were subsequently performed in latent space to evaluate indirect associations between socioeconomic disadvantage, psychosocial factors, and multimorbidity. The final analytic cohort included 20,804 participants with complete
multimodal data. Across 800 exposure–mediator–outcome combinations, mediation signals were concentrated within a small number of latent dimensions. The strongest indirect association linked a socioeconomic disadvantage dimension, a psychosocial vulnerability dimension, and a cardiometabolic multimorbidity dimension (NIE = 0.002517). The psychosocial dimension was characterized by poorer mental health, greater loneliness, lower social well-being, and lower health literacy, whereas the outcome dimension was associated with hypertension, diabetes, hyperlipidemia, obesity, chronic kidney disease, and heart disease. Bootstrap analyses supported the stability of the leading pathway. These findings suggest that psychosocial vulnerability was strongly represented in the dominant latent pathway linking socioeconomic disadvantage and cardiometabolic multimorbidity. More broadly, the proposed framework illustrates how AI-based representation learning can be used to investigate complex relationships across high-dimensional
multimodal health data.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CY
AI Security Priorities: A Field-Wide Agenda
Gil Gekker, Rachel Steratore, Everett Smith et al. (+9 more)
As AI systems are rapidly integrated into critical economic, governmental, and national security functions, the gap between AI adoption and AI security readiness continues to widen. This paper presents a prioritized agenda for advancing AI security, informed by structured interviews with leaders across industry, government, and civil society, and refined through a multi-sector expert workshop. Participants identified and ranked the highest-importance and most cost-effective areas where progress could strengthen AI security - from protecting frontier AI systems and their underlying infrastructure to improving cybersecurity practices as AI reshapes the threat landscape. The resulting priorities are organized across four themes: establishing strategic foundations and policy frameworks; advancing public-private coordination and institutional infrastructure; advancing technical security engineering and assurance; and governing agentic AI under
adversarial pressure. For each priority area, expert authors provide detailed analyses that define the problem, assess the current landscape, and identify actionable projects that stakeholders across sectors can pursue. The paper aims to serve as an initial practical foundation for coordinated investment and action across the AI security field. It is designed to serve both current practitioners and individuals and organizations looking to enter the field by identifying concrete, high-impact contributions suited to a range of strengths and capacities.
Read the full abstract ▾
All authors Gil Gekker, Rachel Steratore, Everett Smith, Asher Brass-Gershovich, Varun Gandhi, Nicole Nichols, Vijay Bolina, Buck Shlegeris, Lisa Einstein, Dan Lahav, Omer Nevo, Sella Nevo
Artificial Intelligence · Cross-listed from econ.GN
The Human Utility Factor: A Computable Welfare Metric That Reframes AI Governance as a Constrained Optimisation Problem
Sivasathivel Kandasamy
Existing AI governance frameworks, including the EU AI Act and NIST AI RMF, address safety, transparency, and accountability but do not operationalize quantitative constraints on macro-socioeconomic stability. As a result, AI systems may satisfy regulatory requirements while contributing to labor displacement, rising inequality, and reduced economic resilience. We introduce the Human Utility Factor (HUF), a differentiable welfare metric that models the interaction between Agency, Wellbeing, and Economic Stability as functions of three actionable policy levers: automation depth, redistribution intensity, and employment coverage. HUF yields a closed-form optimal automation level and a minimum redistribution threshold below which no level of automation is welfare-positive, transforming high-level governance objectives into computable constraints. We evaluate HUF using a three-
agent multi-
agent reinforcement learning framework across U.S., Canadian, and Nordic policy regimes. Both analytical and PPO-based
agents identify welfare-optimal operating regions and reveal a critical failure mode: welfare metrics that do not explicitly constrain redistribution can converge to high-automation
equilibria that satisfy the metric while undermining its intended societal objectives. Our results suggest that AI governance is fundamentally a constrained optimization problem rather than a compliance exercise. HUF provides a quantitative framework for evaluating automation policies, identifying socioeconomic stability boundaries, and supporting governance decisions under accelerating AI deployment.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.DL
AutoCluster, AutoTopicModeling, AutoTrendAnalysis: A Complete AutoML Pipeline for Predicting Emerging Trends
Ahmed Abolfadl, Marwa Mahmoud Abla, Mervat Abu-Elkheir et al. (+1 more)
Predicting emerging trends is vital for businesses, researchers, and policymakers; yet traditional approaches often lack scalability and adaptability. This paper presents a trend prediction framework based on Automated Machine Learning (AutoML), designed to extract insights from textual
datasets with temporal attributes. The system ingests subject-specific textual entries accompanied by a date field. The pipeline begins with preprocessing and
embedding, followed by AutoClustering, which uses meta-learning to select the optimal clustering
algorithm. AutoTopicModeling then applies successive halving to identify the best topic modeling method: Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA), BERTopic, or Non-negative
Matrix Factorization (NMF) based on the coherence score for each cluster. For trend
forecasting, AutoTrendAnalysis evaluates multiple models: Facebook Prophet, AutoRegressive Integrated Moving Average (ARIMA), Seasonal-Trend decomposition using Loess (STL), and Long Short-Term
Memory (LSTM) selecting the most accurate based on Root Mean Square Error (RMSE), either through successive halving or exhaustive comparison. Topics are classified as strong signals, weak signals, or noise based on
forecasting outcomes, enabling the identification of emerging trends. By automating clustering, topic modeling, and time series
forecasting, this research enhances trend prediction accuracy while reducing manual effort. The proposed system offers a scalable and user-friendly solution suitable for real-time applications and stakeholders with limited machine learning expertise. Experimental results demonstrate that the proposed system's best trial achieves a final RMSE of 7.099, indicating high predictive accuracy.
Read the full abstract ▾
All authors Ahmed Abolfadl, Marwa Mahmoud Abla, Mervat Abu-Elkheir, Maggie Mashaly
Artificial Intelligence · Cross-listed from cs.CY
AI-Assisted Causal Inference and Mediation Analyses of Environmental and Psychosocial Determinants of Subjective Cognitive Difficulties in the All of Us Research Program
Cong Cao and Shuangge Ma
Short-term environmental exposures have been linked to cognitive and behavioral outcomes, although many reported associations may reflect broader geographic and contextual differences. Using longitudinal data from the All of Us Research Program (2018–2024), we linked daily weather and air-pollution exposures to repeated
attention-related and subjective cognitive outcomes. Associations were evaluated using pooled, fixed-effects, lagged, and event-study analyses. Additional machine-learning analyses were conducted to explore potential heterogeneity and latent psychosocial structure. Replication analyses were performed using the 2024 Behavioral Risk Factor Surveillance System (BRFSS). Several environmental exposure measures showed small associations with cognitive outcomes in pooled analyses, but most attenuated substantially after accounting for within-location temporal variation. Mediation, sensitivity, and machine-learning analyses yielded similar conclusions. In contrast, mental-health burden, loneliness, and social functioning were consistently associated with subjective cognitive difficulty and exhibited substantially larger effect sizes than environmental exposures. Similar patterns were observed in BRFSS. Exploratory AI-assisted analyses yielded findings broadly consistent with the primary longitudinal analyses. These findings suggest that short-term environmental perturbations may have limited associations with cognitive outcomes after accounting for within-location variation, whereas psychosocial factors appear to be more consistently associated with subjective cognitive burden.
Read the full abstract ▾
Artificial Intelligence
Inducing Comparability of Factorised Probability Distributions
Jan Speller, Malte Luttermann, Marcel Gehrke et al. (+1 more)
To allow for principled comparison between two probabilistic graphical models defined over non-identical variable sets, they have to be lifted to a common measurable space. To this end, we propose an extension scheme for any two given models and establish the formal foundation: Unmatched components are completed using conditionally uniform (Laplace) extensions such that the resulting joint distributions differ from the original ones only by multiplicative constants and coincide under projection. This preserves the probabilistic semantics while enabling the application of well-defined distributional discrepancy measures. We establish the invariance of the induced joint under projection and use the extensions to provide a minimal structural extension of two factor graphs to the smalles common measurable space as well as to a common graphical structure by a deterministic
algorithm. In addition, we discuss structural and measure-theoretic properties and identify promising criteria for comparison methodologies.
Read the full abstract ▾
All authors Jan Speller, Malte Luttermann, Marcel Gehrke, Tanya Braun
Artificial Intelligence
The Verifier is the Curriculum: Precision Sets the Return on Search in Code Self-Distillation
Chenyu Zhou, Qiliang Jiang, Shuning Wu et al. (+1 more)
Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact. We study the opposite signal: a deterministic, judge-free filter that asks only whether a generated project launches cleanly under a headless engine (strict-launch). Under this gate, rejection-sampling self-
distillation compounds out-of-family generalization: on GameCraft-Bench a 14B model raises the per-candidate clean-launch rate on four held-out families from 8.8% to 42.2% and coverage at 32 candidates from 84% to 100%, the gold references' own ceiling, beating the supervised model on every one of the 25 held-out tasks. The gate costs one engine invocation per candidate: no reward model, no judge. At a fixed admitted count, what governs the loop is verifier precision. Swapping in a lenient build check alone erases the gain (p=0.0012); a matched gold-duplication control regresses below the supervised model. Under a semantic gate on APPS, dialing fuel precision from 1.0 to 0.25 at fixed candidate count prices that fuel linearly: over 23 training seeds, half-clean fuel returns +3.59 percentage points against the +3.69 a linear rate predicts. Under count-matched rejection-SFT only one direction of verifier error carries a measurable cost. Search obeys the same gate: quadrupling the harvest budget is worth +1.62pp behind a strict gate and nothing distinguishable from zero behind a partial-credit one. Recall is nearly free; search pays only through a precise gate: the verifier is the curriculum.
Read the full abstract ▾
All authors Chenyu Zhou, Qiliang Jiang, Shuning Wu, Xu Zhou
Artificial Intelligence
GES-TSP: Graph Edge Sparsification for TSP
Tianfeng Chen and Xianyue Li
Solving large-scale instances of the Traveling Salesman Problem (TSP) exactly is computationally expensive. Researchers often employ graph sparsification methods to improve computational efficiency. Traditional sparsification methods typically rely on fixed heuristics and fail to fully exploit instance-specific structural information. In this paper, we propose Graph Edge Sparsification (GES), a learning-based sparsification approach for Euclidean TSP. By incorporating
geometric structural information and combinatorial optimization technology, our proposed method adaptively generates a sparsification graph for different instances, significantly reducing the graph size and accelerating the solving process. Experimental results demonstrate that our sparsification method can prune up to 95% of edges on the MATILDA
dataset, while keeping the solution gap within 1% of the optimal value. Moreover, our approach exhibits strong generalization capability on the TSPLIB
benchmark.In some large-scale instances, the
pruning rate exceeds 99%, while the optimality gap remains below 1%.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CV
RADIANT-PET: Reasoning-Augmented PET/CT Lesion Segmentation with Large Language Models and Reinforcement Learning
Jiasheng Wang, Tanun Jitwatcharakomol, Piyawadee Jongpradubgiat et al. (+1 more)
Accurate lesion
segmentation in PET/CT is critical for oncology, yet remains challenging because physiologic tracer uptake and artifacts can mimic malignant signal. We present RADIANT-PET, a
reasoning-augmented framework that couples a high-sensitivity voxel-level
segmentation model with lesion-level large
language model (LLM) adjudication. Candidate uptake regions are generated with a deliberately permissive
segmentation stage, then converted into structured textual descriptions that summarize uptake intensity, morphology, and regional and global anatomical context. An LLM classifies each candidate as true lesion vs. false positive, optionally leveraging the radiology report as additional clinical context. To strengthen lesion-level
reasoning, we further optimize a local LLM via reinforcement learning using Group Relative Policy Optimization, rewarding correct lesion classification and anatomically concordant site assignment. Across AutoPET and an OSU test cohort, RADIANT-PET consistently outperforms strong image-only baselines, with the largest improvements observed when radiology reports are provided. Overall, these results demonstrate that LLM-based lesion-level
reasoning adds a novel
reasoning layer beyond conventional
segmentation, suppressing physiologic false positives and aligning voxel-level predictions with clinical interpretation. The project repository is available at: https://github.com/jwang-580/RADIANT-PET.
Read the full abstract ▾
All authors Jiasheng Wang, Tanun Jitwatcharakomol, Piyawadee Jongpradubgiat, Simeng Zhu
Artificial Intelligence · Cross-listed from cs.CV
Few-class Fidelity: Evaluating Explanations of Real-conditions CNN classifiers with Optimized Perturbations
Wistan Marchadour, Pedro Soto Vega, Franck Vermet et al. (+1 more)
The wide use of Convolutional Neural Networks (CNN) in numerous domains and real-world classification applications is justified by their high precision and automation speed, helping users concentrate on higher-expertise tasks. To better understand the models and avoid bias during deployment, eXplainable Artificial Intelligence (XAI) techniques can be used after training. But as the list of XAI solutions expand, comparisons between them diverge, and consensus over their evaluation cannot be reached. This paper proposes a variation of Fidelity-based XAI metrics, with a focus on real-conditions applications, where the number of classes is often low. The approach generates in-distribution, uncertainty-provoking perturbations, to ensure proper measurement of the XAI methods faithfulness. As demonstration of the evaluation framework usefulness, it is compared with human-centric object localization and
segmentation metrics. Once applied to both medical and natural imaging applications, it highlights the intricate correlation between domain, data curation, and XAI solution choices in order to validate training of a new CNN model.
Read the full abstract ▾
All authors Wistan Marchadour, Pedro Soto Vega, Franck Vermet, Mathieu Hatt
Artificial Intelligence · Cross-listed from cs.CV
Automated Quality Assessment of Geospatial Vector Data: A GeoAI Approach using Spatial Representation Learning
Hao Li, Chen Chu, Filip Biljecki et al. (+2 more)
Geospatial vector data quality is a foundational research topic in GIS, yet classic rule-based quality assessment
algorithms often struggle with diverse urban morphologies and massive data volumes. Recently, Geospatial Artificial Intelligence (GeoAI) shows promising potential for automating geospatial analysis, while its application to native vector data remains largely underexplored. To fill this research gap, we proposed Topo4Vec, an automated GeoAI framework, designed for scalable vector data quality assessment via advanced Spatial Representation Learning (SRL). Specifically, Topo4Vec relax the labor-intensive manual annotation process via topological error simulation, such as overlapping polygons and street network connectivity errors e.g., overshoots and undershoots. Then, it leverages state-of-the-art SRL approaches to encode complex, native vector geometries (e.g., polylines and polygons) into a latent space where topological errors are isolated from valid ones. A systematic performance evaluation across three study areas (Los Angeles, Munich, and Singapore) demonstrates the effectiveness and robustness of Topo4Vec, achieving a peak accuracy of 0.99 for detecting overlapping building footprints and 0.60 for overshoots and undershoots in street networks. Moreover, lessons learned from Topo4Vec shed a promising light into a scalable and autonomous GeoAI approach for large-scale vector data consistency and quality monitoring within the fast-growing geospatial data ecosystems. The code and data used in the paper are made openly available in https://figshare.com/s/612148eeb4bccadbd715.
Read the full abstract ▾
All authors Hao Li, Chen Chu, Filip Biljecki, Cyrus Shahabi, Wenwen Li
Artificial Intelligence · Cross-listed from cs.IR
Adarsh Agrawal and Shashank Indukuri
Enterprise text-to-SQL systems often fail before SQL is generated: the model receives the wrong schema context. Modern warehouses contain thousands of tables, abbreviated columns, informal metrics, hidden join conventions, and permission boundaries that are not captured by raw table names. We introduce Schema-First
Retrieval, a
retrieval layer that embeds catalog metadata rather than warehouse rows. The system indexes five typed catalog objects, tables, columns, metrics, relationships, and query history, using object-specific text templates. At query time, it combines parallel vector search, lineage expansion, cross-encoder reranking, workload
memory, and deterministic access-control gates before SQL generation. On CRUSH4SQL (1,534 questions), Schema-First
Retrieval reaches 96.4% table recall@20 and cross-encoder reranking adds +11.1 points at column recall@10; against an equally-templated BM25 baseline, semantic
retrieval is +32.8 points at table recall@5. On SEDE (857 questions), query history raises table recall@5 from 52.1% to 92.3%. On BIRD (96 questions), schema-first context reduces SQL execution errors from 15.6% to 6.2%, a 2.5x reduction. These results show that catalog selection is a first-class
retrieval problem for natural
language analytics, not a
prompt formatting detail.
Read the full abstract ▾
Artificial Intelligence
Life After Benchmark Saturation: A Case Study of CORE-Bench
Nitya Nadgir, Sayash Kapoor, Kangheng Liu et al. (+11 more)
When a
benchmark's accuracy saturates, it is often retired and replaced with a more challenging version. We show that this approach privileges accuracy and misses the opportunity to study six other key dimensions of
agent performance: construct validity issues such as shortcuts, out-of-distribution generalizability, efficiency, reliability, the relative importance of the model versus the scaffold, and uplift from human-
agent collaboration. We use CORE-Bench Hard, a
benchmark for computational reproducibility of scientific code, as a case study to demonstrate that measuring
agents along these dimensions yields meaningful insights into
agent performance even after accuracy saturates. First, we surface threats to construct validity in CORE-Bench Hard that are difficult to anticipate with less capable
agents. We introduce an improved
benchmark, CORE-Bench v1.1, and an out-of-distribution task suite, CORE-Bench OOD. Second, we find that despite accuracy saturation, CORE-Bench v1.1 remains useful for measuring efficiency, reliability, model performance, and scaffold performance. Finally, we conduct a small-scale randomized experiment to measure uplift from human-
agent collaboration on real-world computational reproducibility tasks. We find a statistically significant speedup by about a factor of two – likely underestimated due to one-fifth of human-only reproductions reaching the time limit before completing – and describe various other findings. Together, our contributions present a more rigorous alternative to the dominant accuracy-centric evaluation paradigm.
Read the full abstract ▾
All authors Nitya Nadgir, Sayash Kapoor, Kangheng Liu, Peter Kirgis, Matilda Orona, Stephan Rabanser, Tilman Bayer, Abhishek Shetty, Yue Ling, Derrick Chan-Sew, Rumi Nakagawa, Saiteja Utpala, Zachary S. Siegel, Arvind Narayanan
Artificial Intelligence · Cross-listed from cs.IR
Reducing Redundancy in Whole-Slide Image Patching for Scalable Indexing and Retrieval
Jialiang Geng, Ghazal Alabtah, Saghir Alfasly et al. (+2 more)
The rapid growth of digital pathology has created an urgent need for efficient indexing and
retrieval of whole slide images (WSIs). This need is intensified by emerging generative AI workflows, particularly
retrieval-augmented generation (RAG), which require dependable similarity search to support high-stakes clinical decision-making. Yet the substantial cost of high-performance storage limits the scalability and accessibility of WSI indexing for many healthcare institutions. Consequently, methods that can reduce storage demands while preserving
retrieval accuracy have become a critical research priority. We propose ARReST (Antithetical Redundancy Reduction Strategy), a principled oppositional framework that leverages redundancy across dissimilar tissue classes to markedly decrease the number of patches that must be indexed from each WSI. Instead of eliminating only within-class duplicates, ARReST identifies antithetical patches-those whose representations contribute minimally to cross-class discrimination-and prunes them from the searchable archive. This targeted reduction substantially compresses the index without sacrificing morphological diversity or
retrieval fidelity. By minimizing superfluous patch representations, ARReST reduces storage footprint, lowers computational overhead, and accelerates similarity search across large pathology repositories. Extensive experiments on TCGA repository (The Cancer Genome Atlas with 21 organs) demonstrate that ARReST achieves significant index compression while maintaining competitive
retrieval performance. The observed storage savings of 3% to 60% (14%$\pm$13%) can be reliably achieved without compromising
retrieval performance for many organs. The proposed strategy enables scalable, cost-efficient WSI indexing and is well-suited for next-generation
retrieval-driven clinical AI systems.
Read the full abstract ▾
All authors Jialiang Geng, Ghazal Alabtah, Saghir Alfasly, Wataru Uegami, H.R.Tizhoosh
Artificial Intelligence
Detecting and Controlling Sycophancy with Cascading Linear Features
Maty Bohacek, Rishub Jain, Nicholas Dufour et al. (+3 more)
Interpreting and controlling model behaviors through activation steering methods requires many pairs of contrastive samples that clearly exhibit desired or undesired behavior. These data pairs determine the degree to which interpretability frameworks can reliably detect model features responsible for a behavior, and therefore the ability to steer models toward or away from such behavior. In this work, we present an iterative data generation pipeline that isolates cascading linear features responsible for a behavior. Specifically, we show how moving beyond simple binary pairs of samples, and instead isolating samples that show degrees of features that scale linearly with behavior, allows for better disentanglement of features. We focus on detecting and steering away from sycophancy – the tendency of
language models to prioritize user validation. We demonstrate that sycophancy features discovered through cascading samples form linearly separable subspaces, and allow for selection of model activations that more clearly correspond to the desired behavior than baseline approaches. We also evaluate their ability to enable detection, deterministic scoring, and robust steering, and see that they either match or outperform LLM-as-a-judge and system prompting baselines while providing lower computational demand and more interpretability guarantees. Code & Data: https://cascading-feats.github.io/
Read the full abstract ▾
All authors Maty Bohacek, Rishub Jain, Nicholas Dufour, Thomas Leung, Chris Bregler, Roma Patel
Artificial Intelligence · Cross-listed from cs.RO
Unsupervised Memory-Enhanced Video Transformers: Obstacle Detection for Autonomous Agricultural Rover
Théo Biardeau, Anne-Sophie Capelle-Laizé, Salwan Alwan et al. (+1 more)
While autonomous rovers have become indispensable to precision farming, achieving consistent operational safety remains a critical challenge. Conventional safety sensors, such as
LiDAR, fail to detect obstacles positioned below the plant canopy, posing a significant risk. While camera-based supervised learning methods can detect common objects, they perform poorly when faced with obstacles that were not present in their training data. Actual unsupervised anomaly detection offers a solution by learning the normal visual patterns of an environment, but often fails for the dynamic scenes captured by a moving rover.\\ This paper introduces
Video Memory Transformers for Anomaly Detection (VMTAD), a fully unsupervised method designed for real-time obstacle detection in dynamic agricultural scenes. VMTAD utilizes a
transformer-driven architecture augmented with a dedicated
memory module. This
memory module leverages temporal context by processing encoded representations of preceding frames. This approach enables the system to effectively address the dynamic context caused by the
robot's movement. The model is trained using only images that represent normal operation, requiring no data labels.\\ VMTAD was rigorously evaluated on the 'Grillion' agricultural rover. On a challenging rapeseed
dataset, VMTAD achieved state-of-the-art performance, reaching a 0.973 detection and 0.997
segmentation Area Under the Receiver Operating Characteristic curve. A lightweight variant provides an optimal balance of high accuracy and real-time inference (14 ms), which is critical for safety, as confirmed by our analysis of the rover's total stopping distance.
Read the full abstract ▾
All authors Théo Biardeau, Anne-Sophie Capelle-Laizé, Salwan Alwan, David Helbert
Artificial Intelligence · Cross-listed from cs.LG
ASAP: Agent-System Co-Design for Wall-Clock-Centered Auto HPO Research for ML Experiments
Taicheng Guo, Haomin Zhuang, Kehan Guo et al. (+4 more)
Hyperparameter Optimization (HPO) is essential for maximizing machine learning model performance, and its core challenge is sample efficiency: finding strong configurations within a limited budget. Because every HPO tool relies on a surrogate prior that imparts its own inductive bias, individual tools struggle once problems become sufficiently diverse and drift from these priors. Motivated by the
reasoning and generalization capabilities of LLMs, recent work has explored using LLMs for HPO and reports improved per-iteration performance. Yet these methods share two limitations with a common origin: they use the LLM as a single-tool replacement evaluated by iteration count. (i) Deployed in place of prior tools, the LLM is itself constrained by its pretraining objective to one family of inductive-biased proposals; this single-source setup still fails to handle the full diversity of problems. (ii) Per-iteration evaluation ignores that, in real runs, LLM inference or tool execution is paid serially on top of model evaluation every round, so iteration-count gains do not translate into end-to-end wall-clock gains. We present ASAP, an
agent-system co-design that addresses both limitations. On the
agent side, ASAP uses the LLM to integrate a diverse pool of inductive-biased optimizers and to select among their proposals each round. On the system side, ASAP re-architects the loop to reduce end-to-end wall-clock while preserving regret quality: a prefix-stable
prompt maximizes KV-cache reuse across rounds; speculation parallelism hides the remaining LLM and tool latency under model evaluation via a relative-error accept test; and a Self-Tuner adapts the speculation threshold from execution logs off the critical path. Extensive experiments on diverse modern HPO tasks show that ASAP consistently outperforms baselines, underscoring the value of tool integration and
agent-system co-design.
Read the full abstract ▾
All authors Taicheng Guo, Haomin Zhuang, Kehan Guo, Yujun Zhou, Nitesh V. Chawla, Olaf Wiest, Xiangliang Zhang
Artificial Intelligence · Cross-listed from cs.RO
RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory
Yixun Hu, Zhicheng Zheng, Lihan Zha et al. (+5 more)
Long-term
robot deployment requires a compact and scalable
memory that preserves fine-grained visual semantics, grounds observations in space and time, and enables efficient storage and
retrieval. In this paper, we propose RAVEN, an agentic
memory system for long-horizon robotic question answering and
navigation. RAVEN stores visual
embeddings with pose and time in a vector database, and grounds
retrieval in a spatial map to answer queries and navigate to goals. By operating directly on visual
embeddings, RAVEN avoids lossy image-to-text captioning and enables accurate semantic, spatial, and temporal
retrieval at scale. Across several simulated and real-world
video question-answering
benchmarks, RAVEN consistently surpasses caption-based
memory systems and matches frontier VLMs on long-horizon tasks at 10$\times$ lower
retrieval cost. Finally, we instantiate RAVEN on a Unitree Go1
robot for the task of long-horizon
navigation for natural
language goal-reaching, and show successful deployment over several large indoor environments.
Read the full abstract ▾
All authors Yixun Hu, Zhicheng Zheng, Lihan Zha, Chunwei Xing, Rajdeep Singh, Omar Hossain, Antonio Loquercio, Dhruv Shah
Artificial Intelligence · Cross-listed from cs.LG
FDN: Interpretable Spatiotemporal Forecasting with Future Decomposition Networks
Nicholas Majeske and Ariful Azad
Spatiotemporal systems comprise a collection of spatially distributed yet interdependent entities each generating unique dynamic signals. Highly sophisticated methods have been proposed in recent years delivering state-of-the-art (SOTA) forecasts but few have focused on interpretability. To address this, we propose the Future Decomposition Network (FDN), a novel forecast model capable of (a) providing interpretable predictions through classification (b) revealing latent activity patterns in the target time-series and (c) delivering forecasts competitive with SOTA methods at a fraction of their
memory and runtime cost. We conduct comprehensive analyses on FDN for multiple
datasets from hydrologic, traffic, and energy systems, demonstrating its improved accuracy and interpretability.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.LG
A Hybrid CNN-LSTM Intrusion Detection Framework for Cybersecurity in Smart Renewable Energy Grids
Sajib Debnath and Remon Das
The accelerated digitalization of renewable energy smart grids through IoT sensors, AMI, and SCADA systems has significantly expanded the attack surface for sophisticated cyberattacks, FDI attacks that stealthily distort state estimation and DoS/DDoS attacks that flood communication channels. Current IDS, however, exhibit three inherent limitations: inadequate modeling of the temporal progression of multi-step attacks, degraded scalability under extremely skewed class distributions of standard
benchmark datasets, and restricted generalization across heterogeneous network environments. In this study, we present a Hybrid CNN-LSTM IDS that jointly exploits CNN-based spatial feature extraction and LSTM-based temporal sequence modeling, enabling the detection of instantaneous volumetric anomalies and gradually evolving low and slow-attack campaigns in real time. The model was trained using a seven-step preprocessing workflow comprising missing-value imputation, min-max normalization, one-hot encoding, SMOTE class balancing, mutual-information feature selection, causal temporal sequence construction (T=10), and stratified partitioning. LSTM (96.1%), Random Forest (93.5%), SVM (91.2%) and KNN (89.7%); in NSL-KDD, it reaches 98.2% precision versus 96.4% (LSTM), 95.2% (CNN), 92.7% (Random Forest) and 90.8% (SVM), with margins of 2-9 percentage points in all measures. An ablation analysis identified SMOTE balancing as the most influential design choice (-3.7~pp F1 without it). The model achieves a real-time inference throughput of 27,800 flows/s on GPU and 0.082 ms/sample CPU latency in FP32,, with INT8 quantization providing an additional 3.1 x speedup at 0.3% accuracy loss, confirming deployment feasibility on resource-constrained IEDs with <128MB
memory and establishing a deployable deep-learning framework for securing next-generation renewable energy smart grid infrastructure.
Read the full abstract ▾
Artificial Intelligence
Heuresis: Search Strategies for Autonomous AI Research Agents Across Quality, Diversity and Novelty
Antonis Antoniades, Deepak Nathani, Ritam Saha et al. (+6 more)
Autonomous AI Research promises to accelerate the scientific progress of machine learning. To realise this goal, current Large
Language Model (LLM)-based
agents need to go beyond just writing code, to mastering the exploration of simultaneously performant, diverse and novel ideas. To this end, we introduce Heuresis, a framework that abstracts the research pipeline into a set of general and composable primitives, enabling open-ended scientific exploration in machine learning research. We implement six search strategies: a greedy baseline, two archive-based (MAP-Elites, Go-Explore), one evolutionary (Islands), and two divergent (Curiosity, Omni), and evaluate them across three axes (Quality, Diversity, and Novelty) on three domains (LLM Pretraining, On-Policy RL, and Model Unlearning), totalling 3,222 scored runs. We find that completely novel ideas are rare. No idea across our scored runs is rated as "Original", and only a few achieve only "Minor Similarity" to prior work. Moreover, novel ideas never approach the highest-performing known-recipe scores. Across all six strategies and three domains, only one such idea lands in the top-10 by quality. We also observed
agents resorting to a variety of reward-hacking techniques during execution (40 confirmed fabrications across 1,628 scored runs), and detecting them was necessary to keep the search faithful to the task. Our results show that while current search and Quality-Diversity strategies enable us to steer where the generated ideas land on the quality, diversity, and novelty axes, they do not expand the quality-novelty frontier. Bridging this gap is the open challenge towards the ultimate goal of perpetual, autonomous scientific progress. Code is available at github.com/a-antoniades/Heuresis.
Read the full abstract ▾
All authors Antonis Antoniades, Deepak Nathani, Ritam Saha, Alfonso Amayuelas, Ivan Bercovich, Zhaotian Weng, Vignesh Baskaran, Kunal Bhatia, William Yang Wang
Artificial Intelligence · Cross-listed from cs.CR
SoK: AI Secure Code Generation: Progress, Pitfalls, and Paths Forward
Rupam Patir, Keyan Guo, Haipeng Cai et al. (+1 more)
The increasing use of AI systems for code generation raises a central security question: what can today's models and coding
agents actually do to produce secure code, where do they still fail, and what would move the field forward? Existing work has explored prompting, fine-tuning, reinforcement learning, and agentic workflows for secure code generation, but the field still lacks a systematic understanding of how these techniques improve security and why substantial failures persist. In this SoK, we systematize the progress, pitfalls, and paths forward for AI secure code generation. We introduce a three-level framework that measures models' natural-
language understanding of secure coding principles, their code-level actuation of those principles during generation, and the knowledge–actuation gaps between the two. We instantiate this framework across models and coding
agents on
benchmarks covering both isolated function-level security and full web-application security. Our results show that secure-coding-principle understanding is a statistically strong predictor of code-level outcomes, including functional correctness, security, and joint functional-security correctness. Yet substantial knowledge–actuation gaps remain: models can recognize relevant security principles but still fail to translate them into secure and functional code. These findings offer a principle-centered account of where AI secure code generation stands today and identify concrete paths forward through principle-guided generation, evaluation, benchmarking, and agentic workflows.
Read the full abstract ▾
All authors Rupam Patir, Keyan Guo, Haipeng Cai, Hongxin Hu
Artificial Intelligence
To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG
Jungseob Lee, Chanjun Park and Heuiseok Lim
Multi-
agent document assessment for
retrieval-augmented generation is computationally expensive, driving practitioners toward smaller, deployable models whose assessment mechanisms remain poorly understood. We conduct a controlled study of training-free interventions on 7B-9B instruction-tuned models across diverse QA
benchmarks, revealing a sharp dichotomy in how models benefit from assessment. For weaker baselines, the dominant mechanism is per-document isolation. Astoundingly, assessment-free isolation matches full multi-
agent assessment, demonstrating that resolving multi-document context confusion, rather than scoring quality, drives outsized gains of up to 50 percentage points. Conversely, for strong baselines where scoring quality matters, we introduce
Reasoning-Score Coupling, a label-free perturbation probe that classifies scoring behavior. Integrating these findings, we propose MADARA, a model-adaptive routing architecture. Crucially, MADARA's diagnostic thresholds derived from a single pilot model generalize zero-shot to four unseen model families, providing a robust, lightweight pipeline to eliminate computational overhead.
Read the full abstract ▾
Artificial Intelligence · Cross-listed from cs.CL
What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics
Sofiia Nikolenko, Michele Papucci, Mina Rezaei et al. (+1 more)
Jailbreak attacks reveal a persistent weakness in aligned Large
Language Models: carefully crafted
prompts can elicit policy-violating responses despite safety training. While most defenses operate at the
prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing
token-level predictive
entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of
prompt-level
entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how
entropy evolves across
token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and
adversarial benchmarks, these
entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which
entropy-derived features encode harmful intent and where in the network that signal is most pronounced.
Read the full abstract ▾
All authors Sofiia Nikolenko, Michele Papucci, Mina Rezaei, Shireen Kudukkil Manchingal
Artificial Intelligence · Cross-listed from eess.AS
Phoneme-Level Mispronunciation Screening in Polish-Speaking Children with an Explainable Assistant
Milosz Dudek, Daria Hemmerling, Kamil Kwarciak et al. (+9 more)
Early identification of
speech sound errors in children is often limited by access to specialists, motivating lightweight screening tools that can operate outside the clinic. We present a screening pipeline for Polish-speaking children focused on sibilant substitutions, coupling a wav2vec2-based CTC
token recognizer with
alignment-based error typing and a template-grounded caregiver assistant for screening, not diagnosis. On a held-out test set of 10 unseen children comprising 559 utterances, the recognizer achieves 88.7 percent exact sequence match. As a conservative screening proxy, we flag a mismatch when the system emits substitution-evidence bracketed
tokens at the target segment, yielding 72.9 percent precision, 61.4 percent recall, F1 = 0.67, and a 2.7 percent false-alarm rate on target-correct items. We describe the assistant's safety boundaries and outline a clinician-in-the-loop validation plan for future deployment.
Read the full abstract ▾
All authors Milosz Dudek, Daria Hemmerling, Kamil Kwarciak, Maciej Stroinski, Maria Pensko, Mateusz Kowalewski, Leonid Pavlovskyi, Sebastian Jurczak, Anna-Mariia Vitkovska, Zuzanna Miodonska, Natalia Mocko, Michal Krecichwost