Computers and Society · Cross-listed from cs.CR
AspisAI: A Canonical, Machine-Interpretable Governance Framework for Automated Multi-Standard Compliance Monitoring
Tsafac Nkombong Regine Cyrille, Hasan Dag, Reiner Creutzburg et al. (+1 more)
Organisations operating in regulated and critical-infrastructure sectors must satisfy multiple, heterogeneous cybersecurity and
privacy instruments simultaneously, including but not limited to ISO/IEC~27001, the NIST Cybersecurity Framework~2.0, Cyber Essentials, and the GDPR. In practice, these obligations are managed through manual mappings, spreadsheet-based tracking, and periodic audits that are costly to maintain, inconsistent across standards, and weak in traceability. This paper presents AspisAI, a bounded, standard-agnostic governance framework that translates selected requirements from several frameworks into a canonical, machine-interpretable control model, and evaluates submitted evidence against condition-based decision rules to produce explainable, traceable compliance determinations. Within a bounded scope of 26 representative requirements, the framework is evaluated in a controlled simulation against five governance-oriented criteria and, critically, against two external reference points that mitigate the circularity of single-author evaluation: its cross-standard mappings are validated against NIST's own published informative references, with 57\,% exact agreement and divergences confined to same-family controls, and the framework is applied to real third-party evidence from the OpenSSF Scorecard, surfacing genuine governance gaps in a live open-source project. The controlled results, comprising full requirement encoding, 88.5\,% mapping coverage, complete traceability, and correct detection of all introduced gaps, establish functional correctness, while the external validation provides evidence of applicability beyond the simulation. The contribution is therefore a demonstration that a canonical, provenance-preserving governance model can render multi-standard compliance both automatable and auditable.
Read the full abstract ▾
All authors Tsafac Nkombong Regine Cyrille, Hasan Dag, Reiner Creutzburg, Knut Haufe
Computers and Society
Following the Preference, Missing the Optimum: Compliance Without Optimization in AI Housing Recommendation
Hsuan Lo
Large
language models are becoming the first point of contact for consumer search in domains where the stakes are material and the law is explicit. Existing audits show that models steer housing seekers by perceived identity, but none can say what a user loses when a recommender overlooks a suitable option, for want of an enumerated inventory to score omissions against. We audit AI housing recommendation against a verifiable ground truth. For each of 150 synthetic renter scenarios in New York City we build a pool of 120 real listings with known rent, bedrooms and GTFS-computed transit commute, compute the exact set satisfying the renter's stated constraints, and derive its Pareto frontier. The primary outcome assumes no utility function: a recommendation is strictly dominated if the same pool holds a listing cheaper, faster to commute from and no smaller in bedrooms. Across 9,945 calls to three models from two vendors, compliance is near-perfect (1.8% violation against a 66.6% random floor), yet 39.0% of recommendations are strictly dominated, and the dominating listing is a median 900 USD/month cheaper and 3.5 minutes closer. A within-scenario manipulation separates two capabilities usually conflated: changing one sentence moves median recommended rent by 646 USD/month in the correct direction, so preferences are honored, yet recommendations still sit 606 USD/month above the five cheapest qualifying listings on the same screen, and an unambiguous lexicographic instruction gives no improvement under equivalence testing against a pre-specified 50 USD/month bound. The gap widens with candidate-set size and replicates across OpenAI and Anthropic models to within 3 USD. We characterize the failure as compliance without optimization, propose dominance-rate instrumentation as a deployable diagnostic, and release all code,
prompts and per-call results.
Read the full abstract ▾
Computers and Society
Alternative AI Philosophy: Daoism as Method for AI in Education
Qin Xie
As artificial intelligence (AI) rapidly iterates and transforms teaching, learning, and knowledge production, philosophical reflection has become increasingly indispensable to educational debates that remain predominantly shaped by Western intellectual traditions. This article proposes Daoism as an alternative philosophical framework for reimagining AI in education. Through philosophical analysis and textual interpretation of classical Daoist sources, brought into dialogue with contemporary scholarship on AI in education, it examines how the Daoist concepts of "Dao nature," "self-cultivation," and the "Zhenren" address fundamental questions concerning reality, the epistemic aims of education, and ethical action in the AI-mediated era. In doing so, the article diversifies the philosophical voices shaping inquiry into AI and education, enriching the field's conceptual resources for grappling with the philosophical questions AI raises for education and offering a genuinely pluralistic foundation for comparative philosophy of education in the AI era.
Read the full abstract ▾
Computers and Society
Designing Technology for Social Wellbeing in Built Environments: A Conceptual Framework
Gul Sher Ali, Michail Giannakos, Monica Lillefjell et al. (+1 more)
Digital technologies are deployed in urban built environments with the aim of supporting social dimensions. However, the research offers no unified guidance for such digital technology design. Additionally, evidence shows that week social wellbeing contributes to mental and physical health outcomes which suggests that the technology designed to strengthen social wellbeing could also function as a form of health promoting and preventive intervention. This chapter addresses this research gap by developing a conceptual framework that supports the design of digital technologies for social wellbeing in built environments. We propose a conceptual framework composed of three core components which are drawn on the synthesis of selected empirical studies on technologies embedded in built environments for social wellbeing. First, a social wellbeing dimensions model that identifies what digital technology could address. Second, a digital technology contribution
matrix that distinguishes the types of contributions a digital technology could make. Third, levels that maps the scope at which technology could support social wellbeing. This conceptual framework could help researchers, practitioners and policymakers to design and guide digital technology interventions that target social wellbeing in the built environment.
Read the full abstract ▾
All authors Gul Sher Ali, Michail Giannakos, Monica Lillefjell, Sobah Abbas Petersen
Computers and Society
Governing AI Research Through Peer Review: A Mixed-Methods Study of the Longitudinal Effects of Ethics Flags Across Resubmissions
Kento Nishi, Alec Laprevotte, Isaiah Bullock et al. (+1 more)
Selective AI conferences have recently begun enforcing ethics flags and related review requirements, with the goal being to steer research towards safer and more responsible practices before publication. But do these requirements actually steer research as intended? In this paper, we show that authors more often revise how projects are presented following ethics flags than redirect their underlying research agendas. We first study the longitudinal effects of ethics flags by following rejected and withdrawn ICLR submissions with ethics flags into later public resubmissions, tracking manuscript changes after the ICLR review ends, when the original reviewers no longer oversee the project. We qualitatively code these resubmissions into five categories based on what changed after review and find that in 83% of 446 cases, authors leave the flagged concern unaddressed or revise the paper without changing the implicated methods or procedures. Then, we ask: if authors rarely change the research in response to ethics flags, what do they change instead? To answer this, we manually read reviews and rebuttals from 25 cases and directly interview authors about their rebuttal processes and resubmission decisions. We find that authors often concede concerns during rebuttal when reviewers can update their assessments, but drop those concessions after rejection when they do not regard the criticism as a sound reason to change the research. Interview participants describe publication changes as separate from changes to research direction, calling review an "editorial process" that shapes "what stories get seen" and, in another case, saying peer reviews are "mostly to filter out papers." Authors more readily change what they publish than what they study or build; we therefore recommend policy changes, especially disclosure of prior ethics flags upon resubmission so accountability carries over.
Read the full abstract ▾
All authors Kento Nishi, Alec Laprevotte, Isaiah Bullock, Mfoniso Andrew
Computers and Society · Cross-listed from cs.HC
What Makes Creation Human? Authorship, Reasons, and Meaningful Human Control in Generative AI
Yuxi Cao
Generative artificial intelligence (GenAI) significantly expands creators' productive capacity, but this does not necessarily entail a corresponding increase in creative agency or authorship. This paper distinguishes creativity at the level of the work from creative agency at the level of the creator, and argues that human authorship cannot be determined solely by manual intervention, degree of automation, the origin of an initial idea, or final selection authority. Rather, authorship depends on whether human judgment and reasons genuinely shape the development of the work. To articulate this requirement, the paper introduces Meaningful Human Control (MHC) into generative creation and identifies a limitation of its classical tracking condition. Creative reasons are not always fully specified prior to interaction with AI; they may emerge, change, or be abandoned as the creative process unfolds. The paper therefore proposes dynamic-reflexive tracking (DRT), which requires that a creator's evolving reasons undergo reflective uptake, exert genuine influence on the subsequent trajectory of creation, and remain capable of rejecting and redirecting the system's default direction. DRT consists of four conditions: diachronic reason formation, reflective uptake, trajectory efficacy, and contestability and redirection, together with a minimal tracing requirement. The paper argues that human authorship under generative AI depends not on how many steps a person personally performs, but on whether that person's reasons continuously, reflectively, and effectively shape what the work becomes.
Read the full abstract ▾
Computers and Society
Emergency Department Revisit Quality Review Screening: Exploring Human Decision-Making and Artificial Intelligence Support
Jonathan A. Handler, Marlene I. Robles-Granda, Jacob E. Mefford et al. (+5 more)
Background: Emergency Department (ED) return visits are commonly reviewed for quality assurance, but are often limited (e.g., to revisits within 48-72 hours) to increase actionable finding yield while minimizing chart review burden. Those limitations may lead to missed quality improvement opportunities. Methods: We conducted an exploratory, retrospective study of randomly selected ED visits to a multihospital health system having an ED revisit within 1-14 days to the same health system. Given only each visit's primary diagnosis, raters (2-3 clinicians and GPT-4 large
language model [LLM]) assessed characteristics of the diagnosis pairs, including the "target": whether a pair warranted further assessment. Informed by rater response analyses, an
algorithm leveraging an LLM-populated knowledge graph ("KGA") was created to automatically screen for potentially concerning pairs, then preliminarily assessed. Results: 99 diagnosis pairs were included. GPT-4 responses poorly correlated to clinician raters, rating nearly all (94%) pairs as warranting follow-up (4.4-13.3 times more than clinicians). However,
prompt engineering was minimal. Among clinician raters, revisit medical
gravity was consistently significantly associated with the target, while a differential diagnosis/complication composite was significantly associated on unadjusted, but not adjusted (though less powered) analysis. The KGA achieved 83-100% positive predictive value for at least one clinician rater determining further assessment was warranted based on the diagnosis pair. Conclusion: These results can inform next steps for improving screening with LLMs like ChatGPT. Further research is warranted to validate this preliminary work's finding that the KGA may enable enhancing the scope and yield of screening without substantially increasing reviewer workload.
Read the full abstract ▾
All authors Jonathan A. Handler, Marlene I. Robles-Granda, Jacob E. Mefford, Jeremy S. McGarvey, Gregory S. Podolej, Colleen J. Klein, Matthew D. Dalstrom, William F. Bond
Computers and Society · Cross-listed from cs.CL
Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization
Ayan Majumdar, Shounak Paul, Pushpdeep Singh et al. (+6 more)
The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models possess the basic capabilities needed to confront this challenge, whether they can reliably moderate online content remains an unanswered question. In this paper, we systematically compare two competing paradigms for
Vision-
Language Model (VLM) guidance: an instruction-driven approach where models reason from policy precepts, and an example-driven approach where they generalize from prior precedents. We ground this investigation in ModerationBench, a new
benchmark of 4,000 manually annotated, in-the-wild posts from the Bluesky platform. Our experiments reveal that foundation models can substantially outperform Bluesky's deployed moderation system, nearly tripling its $F_1$ score (0.60 vs. 0.22) on Random Posts in the
benchmark, with both instruction- and example-driven paradigms achieving comparable peak effectiveness. Our findings thus chart a path toward reliable and adaptable policy operationalization at scale.
Read the full abstract ▾
All authors Ayan Majumdar, Shounak Paul, Pushpdeep Singh, Ines Abdelaziz, Sayeh Jarollahi, Seungeon Lee, Krishna P. Gummadi, Ingmar Weber, Abhisek Dash
Computers and Society · Cross-listed from cs.AI
Cyber-Financial Contagion: Modeling the Propagation of an AI Vendor Compromise Through the Banking System
Alex Leytes
The banking system now depends on a small set of shared artificial intelligence vendors for fraud screening, credit decisioning, anti-money-laundering triage, customer analytics, and internal decision support. This paper studies how a compromise inside one of those vendors can propagate along a chain of operational, informational, and financial linkages until it triggers losses that look, from the outside, like a classical banking crisis. We build a four-layer heterogeneous network that couples AI vendors, financial institutions, interbank exposures, and customer accounts, and we propose CFC-Prop, a
stochastic epidemic-and-clearing model that runs on that network. On a synthetic
dataset with 60 vendors, 220 banks, roughly 2,500 vendor-bank service edges, and 1,400 interbank exposures, CFC-Prop reproduces the heavy-tailed loss distributions and the sharp dependence on patch latency that are consistent with prior cyber-financial evidence. We also train an early-warning model, CFC-GNN, that uses vendor-side incident telemetry and graph structure to flag high-cascade-risk vendors before impact. Across four baselines the proposed model reaches AUROC 0.82 and AUPRC 0.60 while keeping calibration errors bounded. We release the full code, synthetic data, and reproducible scripts. The results argue that cyber concentration among AI vendors is a first-order financial-stability problem and give supervisors a concrete quantitative tool for
reasoning about it.
Read the full abstract ▾
Computers and Society
Total Simulated Survey Error: Designing and Diagnosing Survey Responses from Large Language Models
Indira Sen, Georg Ahnert, Leah von der Heyde et al. (+3 more)
Large
Language models (LLMs), having been trained on vast amounts of human-generated data, may encode the attitudes and behaviors of these humans. As such, LLMs show promise in mimicking human-like patterns that facilitate their use in simulating people in a wide variety of contexts. One such context is using LLMs as 'silicon samples', i.e., proxies of people in answering survey questions to establish public opinion, design policies, or use as (social) scientific data. However, several critical questions of social biases, generalization, and technical limitations remain, further complicated by a vast design space open to simulation designers. Multiverse analyses might help us make sense of the impact of different design choices, however, we lack a systematic understanding of the design space of LLM-generated surveys as well as how these decisions interplay with inherent LLM limitations. Therefore, how do we systematically identify, trace, and document limitations in LLM-generated survey responses? Building on traditions in the quantitative social sciences, specifically survey methodology and measurement theory, we investigate threats to the validity of LLM-generated survey responses. To do so, we design a framework that enumerates conceptual errors and systematic biases that can occur at different stages of the survey simulation lifecycle. Our framework, called the Total Simulated Survey Error (TS2E) Framework, provides a unified and end-to-end perspective on LLM-generated survey data. The framework, illustrated through a theoretical and empirical case study, enables survey simulation designers to systematically identify and reflect on errors in LLM-generated surveys.
Read the full abstract ▾
All authors Indira Sen, Georg Ahnert, Leah von der Heyde, Jana Lasser, Bernd Weiß, Markus Strohmaier
Computers and Society
Senseful Consense: Towards Simplified Cookie Banners using Plain Language
Minela Bećirović, Ha Dao, Mannat Kaur et al. (+2 more)
While the GDPR and ePrivacy Directive mandate that consent information must be clear and accessible, most modern cookie banners remain obscured by technical jargon, vague phrasing, and frequent content overload or underload. This feasibility study investigates the impact of applying plain
language (Einfache Sprache) to cookie banners within the IAB Transparency & Consent Framework (TCF). In our study, we analysed cookie banner texts from 200 websites, using AI-based mapping to categorise extracted content into standardised processing purposes. By substituting complex legal terms with simplified descriptions, we successfully demonstrated that the comprehension barrier can be lowered from a college-graduate level to a 7th-grade level. However, the effectiveness of plain
language is inherently constrained by the informativeness of the original content; it cannot compensate for banners that omit legally required details. We conclude that while plain
language is a vital tool for digital accessibility, it must be paired with standardised implementation guidelines to ensure that cookie banners are both readable and informative.
Read the full abstract ▾
All authors Minela Bećirović, Ha Dao, Mannat Kaur, Martin Johns, Alexandra Dirksen
Computers and Society
Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governance
Samar Ansari
Compute governance today is a governance of training: the thresholds, reporting requirements, and frontier-AI regimes now in force attach to training compute and treat the trained model as the regulatory unit. That picture is incomplete: capability increasingly migrates to the deployment stage through inference-time scaling, agentic scaffolding, and compression onto consumer hardware. This paper asks which mechanisms are available once the regulatory object shifts from the training run to the inference call. We develop a feasibility taxonomy of twenty inference-time mechanisms across monitoring, verification, and enforcement, each rated on a four-point readiness scale against a documented four-vendor evidence base. We then stress the taxonomy against a two-dimensional adversary model (three capability tiers crossed with four adversary roles) and map each mechanism to four governance scenarios (domestic regulation, bilateral or multilateral coordination, industry self-regulation, and compute-marketplace governance). Fifteen of the twenty mechanisms have commercial technical substrates in production today, although governance-grade assurance and
adversarial robustness vary substantially. The adversary analysis shows that this readiness holds only against a cooperative deployer and a low-to-medium-capability user: no mechanism rates adequate against a high-capability state-level deployer, and fine-tuning removes the model-internal components of the enforcement cluster, although platform-external controls can persist. A substitution analysis connects the taxonomy to a companion hardware paper as a conditional substitution principle describing when inference-stage and hardware-stage mechanisms provide comparable regulatory coverage under stated conditions. A second-rater reliability check on a random subset of the readiness ratings returned a quadratic-weighted Cohen's kappa of 0.74.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.LG
Strangers to Themselves: What Language Models Say About Themselves Is Generic
Phil Blandfort and Urja Pawar
Language models can fluently describe how they would behave: whether they would cave to pushback, misuse a tool, or lie under pressure. Is that description actually about the model speaking? We turn self-knowledge into a prediction test. Across nine behavioral evaluations, we measure how a model behaves under different conditions, ask it to predict those rates, and compare its predictions with controls that remove the self from the question. We find that: (i) Direct self-report is weak (r = +0.04), and even showing the model the exact items only raises prediction to +0.24. Crucially, the same item-informed question about "capable AI
agents in general" does just as well (+0.28), while other models' answers about themselves predict the target model at least as well as its own. (ii) Frontier scale does not detectably change this pattern: any gains in prediction are not self-specific, and are consistent with a better theory of how AI assistants behave rather than better self-knowledge. (iii) First-person framing does have one robust effect: it shifts reports in the flattering direction, understating harmful behavior relative to the same question about a generic
agent. (iv) Finetuning on a model's own behavioral record can teach narrow self-predictions, but it also changes the behavior being predicted and the gains do not transfer broadly. The practical implication is simple: asking a model what it would do mostly reveals a theory of AI assistants in general, plus a favorable bias, rather than privileged knowledge of that model.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
When Does Defendant Statement Matter? A Study of Bias and Persuasion in LLM-Simulated Jurors
Cho-Ying Wu
LLMs have been used to simulate human decision-making in professional settings, yet their behaviors in common-law jury trials remain unexplored. We study when and how a defendant's courtroom statement affects LLM-simulated jurors, focusing on persuasion, ideological bias, and background-based affinity. To support the analysis, we introduce JuryBench, a
benchmark containing controversial criminal cases in U.S. criminal law. In each case, a defendant can claim various plausible justifications to support acquittal or reduced liability. We fix the base case and design defendants of different backgrounds, who give courtroom statements with varying emotional appeal or rebuttal. Jurors with diverse ideological profiles across the spectrum are simulated. We examine 20 frontier LLMs, resulting in a total of 432K decisions and rationales, and quantify changes in verdict severity. Our findings show that LLM-jury simulation echoes many human-jury findings. First, emotional persuasion can be detrimental, since jurors may perceive it as evidence of guilt or inconsistency. Next, we show that background fit between jurors and defendants is a stronger and significant factor than other isolated factors, and that jurors are in general harsher toward opposite-background defendants and lenient toward same-background ones. Finally, we find that juror ideology also strongly shapes severity judgments. These findings highlight both the promise and risks of using LLMs to model jury
reasoning and call for careful evaluation. The data and code are available at https://github.com/choyingw/JuryBench
Read the full abstract ▾
Computers and Society
With a Thermomix You Lose the Ability to Cook: A Kitchen Machine Analogy for Applications of Generative AI in Education
Nikol Rummel, Valentina Nachtigall and Ernesto Panadero
The rapid adoption of generative AI tools such as ChatGPT has sparked intense debate about their risks and opportunities for education, as well as the ways researchers should investigate them. In this paper, we approach these discussions through an analogy with the Thermomix, a smart kitchen appliance that has similarly provoked both enthusiasm and critique. By mapping Thermomix use cases onto examples of learning with generative AI, and situating them within the ICAP and SAMR frameworks, we show how different modes of tool use can either support or undermine meaningful engagement and learning. The Thermomix metaphor underscores that the central question is not whether learners employ AI, but how such use shapes their learning processes. In doing so, we provide a conceptual lens for researchers and practitioners to critically examine - and more effectively guide - the integration of generative AI into educational practice.
Read the full abstract ▾
Computers and Society · Cross-listed from stat.ML
A Unifying Perspective on Probabilities as Model Predictions
Benedikt Höltgen
Although probabilistic statements are ubiquitous, foundational disagreements persist about their understanding, as exemplified by debates between Bayesians and frequentists; moreover, it is unclear when and why acting on them actually leads to desirable outcomes. Here, we argue that every probability is the output of a prediction method, that is, it depends on both a particular way of constructing abstractions and a way of transforming them into predictions. Through this, we provide a unifying perspective on supposedly different kinds of probabilities and show that even supposedly objective ones are model-dependent. We demonstrate that when a finite calibration criterion is met, one can anticipate the distribution of utilities for a given policy and inform successful decision-making on finite sets of events. Based on the notion of prediction methods, inductive arguments, and the probability calculus, we explain the feasibility of the calibration criterion in many settings. Overall, we develop a coherent perspective on probabilities and their use, connecting key intuitions behind other interpretations along the way.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.SI
Chance, Persistent Advantage, and the Generative-AI Era in Open-Source Package Careers
Hazem Ibrahim and Yasir Zaki
Studies of careers in science, film, music, and books report a common pattern. When a person's most successful work arrives is close to a random draw over the works they produce. How large their successes tend to be, in contrast, follows a stable, person-specific factor. We test whether this pattern holds for open-source software careers and whether it changed when generative AI coding tools arrived. From the complete public record of GitHub push events (2015-2025), we reconstruct 102.2M career works by 6.15M contributors, and for the 908k contributors whose repositories publish packages, we measure each work's impact by how many downstream packages come to depend on it. First, we find that the timing of a career's biggest hit is close to a lottery over their works, as in science and the arts, with a small, replicable lean toward early career that grows as careers get longer. Second, some coders reliably produce higher-impact work than others, but this lasting personal factor accounts for only part of why impact persists (about a fifth in our primary specification); the rest behaves like momentum, success feeding on itself for a period of time. Third, within the same contributors, this structure did not change after ChatGPT's release. The stable factor's weight grew by about as much as it grew for an earlier cohort that simply aged, and subtracting the effect of aging from the effect of generative AI puts the shift at +0.03 (95% CI [-0.22, +0.23]), indistinguishable from zero. The success pattern documented in science and the arts therefore describes open-source careers too, and it shows no detectable break across the arrival of generative AI. These results have implications for how track records on open platforms should be read and on what to expect from generative AI for the careers built on them.
Read the full abstract ▾
Computers and Society
What Personal Information Improves LLM-Based Next-Location Prediction?
Xin Wang, Paraic Carroll, Kerry Nice et al. (+2 more)
Large
language models (LLMs) are increasingly used for individual next-location prediction, with personal information easily added to
prompts alongside mobility history. Yet the incremental predictive value of such information remains unclear. Using linked sociodemographic records and mobility traces from 5,000 Shenzhen residents, this study separates model responsiveness from predictive value. GPT-5 is the primary model, with GPT-5.5 and Claude Opus 4.6 used for replication. In 1,000 paired prediction instances, models rank 100 candidate destinations with and without age, gender, occupation and income while all other inputs are held fixed. Behavioural history raises top-1 accuracy from 5.6% to 18.5% as prior history increases from zero to six days. By contrast, sociodemographic attributes produce no detectable overall gain, although replacing correct attributes with those of another person reduces accuracy by 5.4 percentage points. Candidate construction also matters, removing distance raises accuracy by 7.7 points under proximity sampling but lowers it by 22.3 points under popularity sampling, with the reversal reproduced across all three LLMs. These findings identify behavioural history as the clearest source of incremental value and show that personal information should be evaluated under matched, explicitly specified conditions before its
privacy and governance costs are justified.
Read the full abstract ▾
All authors Xin Wang, Paraic Carroll, Kerry Nice, Sachith Seneviratne, Li Zhang
Computers and Society
Watermarks Without Verification: AI Text Watermarking After the EU AI Act
Alexander Nemecek, Vipin Chaudhary and Erman Ayday
On August 2, 2026, the obligations of Article 50 of the EU AI Act took effect, requiring generative AI providers to mark the content their systems produce and ensure it can be detected as AI-generated. Days later, Anthropic disclosed that every Claude model released after that date embeds a watermark based on SynthID-Text in all generated text, enabled by default with no user opt-out; Google has deployed SynthID-Text in Gemini since 2024. Users objected that the watermark degrades quality, particularly for code, that it secretly encodes identifying information, and, in mutual contradiction, that it is easily removable and inescapable; the vendor answered with assurances of unchanged quality, no identifying information, and robustness to light editing. In this work, we argue that neither the objections nor the assurances can currently be verified and that this unverifiability, rather than watermarking itself, is the substantive governance failure. We sort the contested assertions by what it would take to settle each and evaluate the open-source SynthID-Text implementation on two open-weight models, because no public tool can test the deployed systems. On prose, the measured effect of the watermark does not exceed that of changing the sampling seed. On code, the cost is three points of correctness on one model and below measurement on the other, while detection remains near chance, a limitation of detectability rather than quality. The remaining gaps trace to withheld access or missing institutions and we map each to a requirement: release of matched outputs, configuration disclosure, accredited audits, a shared evaluation protocol, and interoperable detection.
Read the full abstract ▾