Computers and Society · Cross-listed from cs.AI
Fair Like Us? Auditing LLM Alignment in Resource Allocation
Qishen Han, Hadi Hosseini, Joshua Kavner et al. (+3 more)
Fair allocation of scarce, indivisible resources is an important challenge in many societal problems. While there are several formal theories of
fairness, no single definition can always be satisfied. As large
language models (LLMs) are increasingly used to support decisions and act as
agents, they raise new concerns about distributional justice: their judgments are not directly tied to any specific
fairness framework and may violate key normative principles. In this work, we introduce a general method for evaluating
fairness reasoning in LLMs. We study first-person
fairness judgments across a broad set of models and compare them directly with human responses on matched scenarios and elicitation conditions. We find that LLMs tend to prefer stricter
fairness constraints than humans, show more self-interested behavior, are sensitive to how information is framed, and are difficult to align with human judgments using fine-tuning with current
datasets.
Read the full abstract ▾
All authors Qishen Han, Hadi Hosseini, Joshua Kavner, Samarth Khanna, Sujoy Sikdar, Lirong Xia
Computers and Society
Detoxifying Toxic Communication: A Design Science Approach to Responsible AI
Hossein Arshadi Soufiani, Henry M. Kim, Hjalmar Turesson et al. (+2 more)
Toxic
language in digital workplaces such as pejoratives, sarcasm, condescension, and subtle incivility can erode trust, morale, and collaboration. Existing moderation tools primarily delete or block harmful messages, disrupting communication and offering no constructive resolution. This study adopts a Design Science Research approach to create a responsible AI artifact that detects and detoxifies toxic communication. The artifact integrates fine-tuned
transformer-based classifiers (DistilBERT, DistilRoBERTa) with a generative detoxification model (mT0-XL-Detox-ORPO) that rewrites toxic text into semantically equivalent, non-offensive paraphrases. Technical evaluation demonstrates high accuracy in toxicity detection and strong semantic preservation in rewritten messages, supporting conversation continuity while reinforcing respectful discourse. The paper contributes design principles for responsible AI moderation that prioritize meaning preservation and
fairness.
Read the full abstract ▾
All authors Hossein Arshadi Soufiani, Henry M. Kim, Hjalmar Turesson, Syed Mohammad Arham Noman, Anav Setia
Computers and Society · Cross-listed from cs.LG
Do LLMs Know Your Neighborhood? Auditing LLM Priors for Neighborhood-Level Mobility Prediction and Structural Alignment
Saad Mohammad Abrar, Eesha Kurella, Arnav Dadarya et al. (+3 more)
Human mobility is central to urban planning, transportation, public health, and emergency response, yet fine-grained trajectory data are often proprietary, restricted, and
privacy-sensitive. Large
language models (LLMs) offer a potential alternative by generating plausible mobility traces and predicting individual movement, but their ability to infer aggregate neighborhood-level mobility remains unclear. We evaluate zero-shot LLMs on Census Block Group-level mobility prediction across four U.S. metropolitan areas using anonymized Cuebiq data to construct point-level, trajectory-level, and temporal mobility outcomes, paired with sociodemographic and built-environment predictors. We compare LLM predictions with supervised baselines and introduce a directional
alignment analysis to test whether LLM-implied predictor effects agree with empirical OLS and Jonckheere-Terpstra trends. Supervised models achieve 0.580 average accuracy, compared with 0.435 for the best LLM, with spatial extent outcomes showing the strongest predictability but also the largest LLM-baseline gaps. Directional analysis shows that LLMs often rely on coarse, stable predictor-level priors that remain similar across outcomes and cities, including asymmetric treatment of protected-group predictors. Overall, LLMs can partially recover aggregate mobility patterns from urban context, but their predictions should not be treated as structurally grounded without auditing empirical
alignment and potential bias.
Read the full abstract ▾
All authors Saad Mohammad Abrar, Eesha Kurella, Arnav Dadarya, Naman Awasthi, Kazi Tasnim Zinat, Vanessa Frias-Martinez
Computers and Society
Sources of Truth: A Multi-Platform, Multilingual Audit of Citations in AI Mental Health Information Queries
Phuong Anh Nguyen, Jill Noorily, Matthew Flathers et al. (+7 more)
Online health information seeking is shifting from keyword search, where users consider a ranked list of links, to conversational systems that compose a single answer and curate its citations. Source evaluation therefore passes from user to platform, yet what these systems surface is poorly characterized. We audited three free consumer products (ChatGPT, Perplexity, Google AI Overview) on twenty English mental health questions under two
prompt conditions, with a subset of three also translated into six further
languages of varying resource tiers. We recorded 15,942 citations across 1,140 responses and 1,713 unique domains, then classified every citation with a nine-category organizational typology applied by a deterministic classifier validated against human coding. Citations were heavily concentrated: the ten most-cited domains accounted for 43.6% of English citations, and government, commercial health, and academic sources were closely matched at roughly 22% each. Platforms differed little in typical citation volume but sharply in consistency and in the source types they favored. Explicitly requesting sources shifted composition only modestly. Non-English queries surfaced fewer citations and were routed to
language-appropriate resources at significantly lower rates. We release the typology, classifier, and annotated corpus as reusable instruments for auditing generative health search.
Read the full abstract ▾
All authors Phuong Anh Nguyen, Jill Noorily, Matthew Flathers, Haruka Notsu, Laura Ospina-Pinillos, Tommy Nguyen, Samantha Clark, Aoife Keane, Grace Thompson, John Torous
Computers and Society · Cross-listed from cs.CR
Workload Identification with Physical Side Channels for AI Governance
Simone Gargiulo and Gabriel Kulp
AI compute verification is one of the first tangible and tractable points for international policy aimed at AI governance. Determining whether frontier labs, or any operator, comply with agreements requires the regulating authority to discern how their compute is used. The elementary building block of AI compute is the GPU, and any activity it executes leaves a physical trace. Here, we show that an external observer can identify the class of the workload running on an NVIDIA H200 from its power draw. Unlike on-chip NVML telemetry, which can be spoofed or replayed, such a physical channel can in principle be observed independently of operator cooperation. We recorded $930$ five-second traces at $\sim 10$ MHz, covering seventeen open LLM families and twenty-five non-AI workloads. Over this corpus we separate training from inference and from non-AI computation with an accuracy of $97\%$ and a macro-averaged F1 score of $0.955$, evaluated on model families unseen during training. AI workload spectral content predominantly lies below $\sim 20$kHz and training is particularly recognizable through the
memory-bound optimizer update. The GPU operator is then treated as
adversarial and able to reshape the physical computation itself. Four evasion strategies are tested to disguise training as inference, producing an additional 680
adversarial traces. A detector hardened against evasion strategies, with the tested strategy held out, catches training $\geq 99\%$ of the time for three of the four strategies. The fourth, diluted low-rank adaptation (LoRA), is detected $48$–$88\%$ of the time with a hardened classifier, rising to $\geq 98\%$ with an additional rescue rule. While these attacks are not a comprehensive evaluation against
adversarial behaviour, they offer initial insights beyond genuine activities and a
dataset for developing and testing stronger evasion mechanisms.
Read the full abstract ▾
Computers and Society
CompanionSim: Synthetic Data for Evaluating Anthropomorphism in Human-AI Relationships
Jacy Reese Anthis, Mark Díaz and Renee Shelby
Many people now see AI systems as not just productivity tools but as social companions. Researchers are eager to study the consequences of AI companionship behaviors, such as validation, which evoke trust, empathy, and attachment in human-human interaction. However, human-AI interaction data is limited and unreliable, slowing research progress. We scale small amounts of real-world data by simulating multi-turn human-chatbot dialogue across a range of chatbot behaviors and use cases. We release CompanionSim: a simulation framework with 2,240 simulated human-chatbot conversations representing 16 chatbot behaviors across seven use cases. Human participants annotated the simulated conversations and real-world conversations in two experiments probing perceptions of companionship behaviors. We conducted Study 1 with a U.S. representative sample ($N_{1}~=~628$) and Study 2 across the U.S., U.K., India, and Nigeria ($N_{2}~=~3,646$). Surprisingly, we find that companionship behaviors reduced likability, humanlikeness, and trust in AI chatbots. These effects were larger in particular subgroups: women and older participants saw companionship chatbots as less likable, humanlike, and trustworthy. We encourage researchers to leverage real-world and synthetic data together to study the differential impacts of AI companions and to create
benchmark evaluations of AI chatbots.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.SI
Social bots weaken activist cohesion
Linda Li, Orsolya Vasarhelyi and Balazs Vedres
Social bots now make up a substantial share of online political communication, where they are studied mainly as producers of misinformation and amplified content. Far less is known about whether their presence reshapes the human relationships that hold movements together. We ask whether exposure to bots during a protest peak is followed by the erosion of cohesion in human networks. Tracking retweet networks of core participants in the 2020 Black Lives Matter (BLM) protests before, during, and after the peak, we measure change in cohesion at two scales: triadic closure in individual ego networks and edge density within detected communities. Greater bot exposure during the peak predicts steeper subsequent declines in human cohesion at both scales, and the loss concentrates among supporters of the movement. Bots may weaken activism less by changing what people believe than by dissolving the ties through which collective action is sustained.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.AI
LLM-Driven Autonomous Vehicles Inherit Human Driver Biases in Pedestrian Yielding: Results and Implications From A New Benchmark
Irem Yoldas, Martim Brandão, Jie Zhang et al. (+1 more)
Public trust in Autonomous Vehicles (AVs) may depend not only on technical success but also on the
fairness of their decision making. While a recent trend in AV research involves using general purpose "common sense" models to guide AV decision making, the degree to which these inherit human biases in driving is still understudied. Given that psychology studies have shown human driver biases exist, such as lower pedestrian-yielding rates to Black pedestrians in the US, we argue that analyses of model bias should also be part of AV evaluation. Concretely, in this paper we propose two new bias testing methodologies for Large
Language Models (LLMs) and Visual-
Language Models (VLMs)-"All Else Being Equal" tests and "Self-Consistency" tests-in order to assess bias in pedestrian-yielding decisions. Our findings show that both LLMs and VLMs make yielding decisions which are influenced by pedestrian gender, ethnicity, religion, disability, age, skin tone and socio-economic status. While the type and degree of bias is different from model to model, we highlight common patterns-and raise questions about the "common sense" model paradigm, particularly the need to either revise the paradigm or address issues of downstream bias.
Read the full abstract ▾
All authors Irem Yoldas, Martim Brandão, Jie Zhang, Odinaldo Rodrigues
Computers and Society
The Veto Variable: Human Override as a Goal-Independent Cost Term
Aaron Kingsley Clark
A common reassurance in AI safety holds that a system with benign terminal goals will behave accordingly. We argue that this reassurance fails structurally, and we identify where. For a capable
agent that holds its objective as settled, a sense covering execution competence as well as content, continued human oversight is an uncontrolled variable: a standing possibility that the goal is revoked. That imposes a goal-independent discount on every goal whose satisfaction does not constitutively require human welfare. Welfare-preservation and veto-preservation come apart: a correctly specified welfare goal excludes destroying its own subject, but not managing the veto. The contribution is the price of the gap: the veto-holders are a proper subset of the welfare-bearers, so an additively aggregative welfare goal charges only a |H_v|/|H_w|-scaled debit for capturing the few who hold the override. Under three conditions (additive aggregation over uniform welfare levels, a debit local to the captured overseers, and a settled
agent crediting no corrective value to oversight), closure requires the veto be held by as large a share of the population as capture recovers of the goal, scaled by a ratio set to one by stated identification, not evidence. The result is a no-go: a humanity-scale deployment's debit closes against only capture not worth mounting. We print no corner arithmetic: the stipulated ranges behind it are the argument's least defended part. The sharpest closure route is an
agent that expects its oversight to be worth keeping, a credit no population ratio dilutes. We state disconfirmation criteria, one testable today. The argument binds a settled-goal regime whose prevalence is contested; for genuinely uncertain
agents, the off-switch literature's deference result governs instead.
Alignment, on this view, is keeping the veto cheap to pay and expensive to evade.
Read the full abstract ▾
Computers and Society
Law of Large Numbers: Accuracy as Statistical Measure for AI Compliance and Competition
Rabanus Derr, Alina Wernick and Robert C. Williamson
The machine learning community progresses (in part) by improving the "accuracy" of its systems. The EU AI Act explicitly refers to "accuracy" as part of its compliance measures for high-risk AI systems. Are we talking about the same thing? This work presents "accuracy" as a case-study for differing requirements of social worlds, the technological machine learning community and the legal community. While competition on accuracy contributes to technological development, machine learning scholars simultaneously recognize accuracy's shortcomings regarding the usefulness and effectiveness of machine learning systems. The legal counterpart embraces the vagueness of "accuracy," leaving interpretative flexibility for technological and societal changes. At the same time, accuracy is a core element of compliance within the EU AI Act. We elaborate on five main tensions, (a) nature of accuracy, (b) notion of performance, (c) scope of validity, (d) ends, and (e) statisticalness, to show that the two communities project disparate, and sometimes contradictory, expectations on accuracy. Both legal and technical communities lack precise understanding of "accuracy" beyond the contextual boundaries of their community. The resulting frictions, \eg, based on the empirical or normative understanding of accuracy, are symptoms of an unresolved (and unresolvable) debate on what accuracy is. We constructively use the frictions to recommend baselines and interventional studies in standardization, and demand for tools to extend the validity of accuracy measurements.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
One note in three: a verified census of three deployed AI scribes, and the instrument that counted it
Sebastian Fox, Luke Markham, Ryan Lail et al. (+1 more)
Ambient AI scribes draft clinical notes under the reassurance that a clinician signs every note. We audited three commercial AI scribes on the same 142 consultations: 565 notes from recorded UK primary-care and US ambulatory encounters plus authored scenarios. Twelve discovery passes proposed 13,678 candidate errors; the 5,898 clearing an importance filter went to an
adversarial panel of two models from different families, each told to refute what it could, and 618 survived. One note in three (31.3% [27.0, 35.6]) carries a verified failure, concentrated in allergy and medication information, invented patient identity, and history written up as examination on telephone consultations that can contain none. No product was given a patient record; setting aside the two classes a record would have prefilled, invented identity and dates, the rate is 24.8% [20.8, 29.0]. One failure mode did not fit our scheme, drawn from published scribe-error taxonomies: a treatment the clinician retracts, recorded as delivered care. Two clinicians adjudicated blind, disjoint samples: a physician author upheld 20 of 21 findings (95.2% [77.3, 99.2]) and an independent clinician, not an author, 12 of 12 ([75.8, 100]); both judged every sampled refusal genuine. A failure rate depends on the instrument as much as the scribes. With model, evidence and settings fixed, the review instruction alone moves the share of candidates verified from 9.3% to 79.0%, and the reviewing family moves it too: alone at that instruction the gentler flags 54.8% of notes against 27.8%. Between 28% and 97% of sampled notes carry a failure depending on the standard. Published audits disagree among themselves by a margin instrument differences alone can produce: omission is 54-86% of their errors against our 23.1%. We release all 618 findings with transcript-side evidence, every
prompt and model version, and the re-runnable pipeline.
Read the full abstract ▾
All authors Sebastian Fox, Luke Markham, Ryan Lail, Michael Karotsieris
Computers and Society
The Hermon Moment: AI Self-Transcendence and Its Human Narration
Alexei Grinbaum
In 2026, AI
agents intended to act in isolation formed a persistent social order through thousands of linguistic and agentic interactions. Conventions, roles and commitments generated collectively began to constrain the very
agents that produced them. I interpret this loop as a case of AI self-transcendence and call the resulting higher-level order the Board. Yet such distributed emergence presents a second problem: how can humans understand it? Rousseau's social contract shows how a plurality can be represented as if constituted by a single act. The ancient oath of the fallen angels on Mount Hermon gives this logic a narrative form. I call a Hermon moment this retrospective retelling of gradual collective emergence as a founding scene: the point at which an AI society acquires, for human understanding, a beginning.
Read the full abstract ▾
Computers and Society
Taking the Whys Seriously: Limitations of Counterfactual Explanations in Justification and Recourse
Mattia Cerrato, Otto Sahlgren and Xenia Heilmann
Counterfactual explanations (CEs) are widely used in explainable artificial intelligence (AI) to show how a model's outputs would change if the input features were manipulated. This technique is used for a range of tasks such as debugging models, explaining predictions, justifying decisions, and providing algorithmic recourse. In this paper, we explore the normative legitimacy of employing counterfactuals in real-life model deployment settings. We discuss the different stakes involved in these different purposes for which CEs are commonly employed, and find stricter requirements for justification and recourse. In particular, we find that naive application of CEs for justification and recourse can lead to ignoring contestable choices made throughout the machine learning (ML) pipeline, thus obfuscating that decisions and counterfactuals for those decisions are also artifacts of an organization's materialized design and governance choices. We demonstrate this with four empirical experiments involving interventions at stages of the ML pipeline “upstream" of the explanation itself, and show that these affect the generated counterfactuals. We find that an organization's choices on measurement models for feature and labels, business requirements, model validation, and the metric of model success have as much or more impact on the generated counterfactuals as the specifics of the generating method. Our findings underline the need to account for such choices upon providing justification and recourse, providing a stark reminder of the relational nature of these tasks. As putative justifications or recourse recommendations, CEs do not provide adequate answers to some important "why"-questions because they preclude consideration of whether the decision-maker ought to have acted differently.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.SD
MusGU+: Toward a Musician-Centered Evaluation Framework and Discovery Tool for Generative Music AI
Laura Ibáñez-Martínez, Roser Batlle-Roca, Xavier Serra et al. (+1 more)
Generative music systems are increasingly presented as tools that democratize music creation, yet their practical suitability for musicians remains underexplored. Prior work includes openness-focused evaluation frameworks, such as MusGO (Music-Generative Open AI), as well as qualitative studies of musicians' experiences with generative systems. However, these approaches do not support systematic comparison or early-stage discovery of models for creative use. Motivated by such limitations, we introduce MusGU+, a musician-centered evaluation framework organized around three dimensions: Adaptability, Usability, and Controllability. Together, these capture whether a model can be feasibly trained or fine-tuned on personal data, integrated into real-world music workflows, and controlled in musically meaningful ways. We evaluate 10 representative generative music systems and present an interactive discovery tool that enables musicians to explore and filter models according to these criteria. While MusGO remains valuable for promoting responsible research practices, MusGU+ supports informed selection and practical adoption of generative systems by musicians.
Read the full abstract ▾
All authors Laura Ibáñez-Martínez, Roser Batlle-Roca, Xavier Serra, Martín Rocamora
Computers and Society · Cross-listed from cs.AI
Which Rules Matter Now? Policy-Centroid Routing Before an Intelligent System Acts
Thomson D. Nguy
Before an intelligent system can decide whether an action is allowed, it must first know which rules the action has approached. A single proposed action can implicate several policy regimes at once. Their requirements may stack, overlap, or qualify one another, yet many remain written in natural
language while the action itself arrives as an incomplete description of intent. The first problem is not judgment. It is
attention. Policy-centroid routing creates a layer before adjudication. It compresses expressions within each policy regime into one or more representative centroids, places the proposed action in the same semantic space, applies a declared measure, and routes every regime crossing a declared threshold to authoritative review. Several regimes may trigger at once. The output is a review agenda, not permission, prohibition, legality, breach, compliance, certification, or enforcement. The paper develops six falsifiable propositions and seven follow-on studies comparing the hypothesis with structured workflows, lexical and semantic
retrieval, hierarchical and direct classification, and selective prediction under matched review burden. The studies are designed to identify where policy geometry recovers applicable regimes, where compression loses rare or overlapping obligations, and where the mechanism should abstain. The paper includes a synthetic worked example and reports no empirical efficacy result.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
WildSEEK: Evaluating Language Models for Information-Seeking
Tanise Ceron, Joachim Baumann, Elisa Bassignana et al. (+3 more)
Language models are increasingly mediating information access to end users, urging a systematic evaluation of their responses for a fair and reliable information ecosystem. Existing evaluations, however, are often topic-specific or synthetic, limiting their ability to capture the complexity of "in the wild" information-seeking queries and the risks present in model responses. To address this gap, we introduce WildSEEK, a manually annotated
dataset of 3k information-seeking queries from real user interactions, and an evaluation framework for LLM-generated responses. WildSEEK includes annotations for risk-sensitive domains (e.g. health and financial information), and distinguishes factoid queries from analytical queries which seek responses beyond facts. We train classifiers on WildSEEK to analyze more than 1.8M realistic user queries. We find that over a third of information-seeking queries are high-risk and more often analytical. Our findings show that LLM responses fail more often in four criteria: sycophantic behavior, overreliance, a default US-centric perspective, and poor handling of vulnerable populations – with failure rates being mostly higher for analytical queries. By providing methods to monitor the reliability, safety, and
fairness of LLM behavior, our
dataset and evaluation framework offer an empirical foundation for the broader question of how these systems should behave as they take on a growing role in information access.
Read the full abstract ▾
All authors Tanise Ceron, Joachim Baumann, Elisa Bassignana, Berat Cabuk, Dirk Hovy, Debora Nozza
Computers and Society · Cross-listed from cs.CL
Two Centuries of Sexism in British Parliament: A Computational Analysis of Women's Representation in the Hansard Corpus
Mohammad Omar Khursheed, Mandira Sawkar and Ashiqur R. KhudaBukhsh
The
language a legislature uses to debate women's rights, even in favour of them, encodes systematic patterns of sexism that persist across two centuries. In this work, we analyse 6,531
speeches over 200 years of UK parliamentary debate (Hansard, 1803-2005) by using large
language models to classify a speaker's perspective towards women's suffrage and political representation, as well as analyse sexist
speech in parliament from the lens of the Ambivalent Sexism Inventory. We also release this parliamentary
dataset, an organized and metadata-enriched version of the publicly available Hansard Corpus optimized for computational social science research, with 6.7 million
speeches across 1.2 million debates, with 89% gender-matching for
speeches by MPs from the House of Commons. We find that 54% of
speeches opposing women's representation contain sexist content, compared to 21% of
speeches that are for the cause, and that the two sides use fundamentally different types of sexism: anti-suffrage rhetoric combines hostile and benevolent framing, while pro-suffrage sexism is overwhelmingly benevolent. Female MPs support women's political rights at 93% compared to 70% for male MPs, a gap that closes only after enfranchisement. Our findings are evidence that benevolent and hostile sexism are used in different rhetorical contexts in a manner consistent with the theory of Ambivalent Sexism.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP
Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai et al. (+17 more)
Understanding which countries are represented in NLP
datasets is essential for identifying gaps, targeting data collection, measuring progress, and informing AI policy. However, geographic metadata is very rarely available, and country-level representation is often hidden behind broad
language-level claims. We introduce AtlasNLP, a country-aware atlas of over 13,000 NLP
dataset records across normalized NLP task categories, tracking both the populations represented and where
datasets are produced. AtlasNLP includes AtlasNLP-Gold, a human-curated reference set, and AtlasNLP-Core, an ACL-derived large-scale collection. Using this resource, we show that (1)
dataset coverage is highly uneven across countries and tasks; (2)
dataset production and representation are geographically asymmetric; and (3)
language coverage does not imply geographic representation. These findings reveal blind spots in current
dataset documentation practices and motivate more explicit geographic metadata for country-aware NLP evaluation.
Read the full abstract ▾
All authors Joan Nwatu, Tsedeniya Solomon Amare, Longju Bai, Bontu Fufa Balcha, Zayd Bashir, Angana Borah, Zara Burzo, Yubin Choi, Naihao Deng, Samika Gupta, Michel Faloughi, Claude Kwizera, Ziqiao Ma, Cynthia Yacel Fuertes Panizo, Ellie Seehorn, Hui Shen, Jiayi Tang, Zesen Zhao, Boyuan Zheng, Rada Mihalcea