Computers and Society
AI Can Do Your Homework. Now What? Report from an Online Workshop on Computing Assessment in the Age of Generative AI
Muhammad Sajjad Akbar, Geoffrey Challen, Fraida Fund et al. (+6 more)
On 28 July 2026, the SIGCSE Virtual 2026 Working Group on computing assessment and generative AI held an open online workshop attended by 73 computing educators. The first hour ran seven strategy rooms, one per approach to adapting (or deliberately preserving) computing assessment in the AI era: open-ended and authentic task design; ambitious, AI-leveraged projects; evaluating and fixing AI-produced work; controlled and AI-free assessment; process evidence and effort signals; rubric and grading redesign; and oral and interactive assessment. The second hour ran question rooms seeded from those strategies, plus two cross-cutting rooms on
fairness and trust and student motivation. This report records what was discussed: the approaches participants have tried, the results they reported, and the questions every room left open. It is the first public artifact of the working group, whose taxonomy of computing assessments that accommodate generative AI use will follow.
Read the full abstract ▾
All authors Muhammad Sajjad Akbar, Geoffrey Challen, Fraida Fund, Casey Hopkins, Oscar Karnalim, Kevin Lin, James McGuffee, Shubbhi Taneja, Ranysha Ware
Computers and Society · Cross-listed from cs.LG
Toward Fairness in Machine Learning Models for Predicting Treatment Retention and Premature Discontinuation in Medication for Opioid Use Disorder
Tongnian Wang, Carolina Vivas-Valencia, Cici Bauer et al. (+3 more)
Persistent low retention and completion rates in medications for opioid use disorder (MOUD) have driven the use of machine learning (ML) models to predict retention and identify patients at risk of premature discontinuation. However, the
fairness of these models across patient populations remains largely unexplored, raising concerns about their application in treatment decision support. This study systematically assesses algorithmic
fairness in ML models for predicting MOUD retention and premature discontinuation and investigates the effectiveness of bias mitigation techniques. Using the cross-sectional Treatment Episode Data Set-Discharges (TEDS-D), which includes treatment episodes for individuals in the U.S. discharged between 2015 and 2019, we trained four ML models to predict premature treatment discontinuation and retention beyond 180 days among individuals receiving outpatient MOUD. We evaluated overall performance and subgroup-level error rates across patient subgroups defined by race, ethnicity, age, and sex, complemented by model explanation analyses. We further assessed bias mitigation techniques and their effects on both
fairness and predictive performance. Our findings demonstrate that ML models for MOUD outcome prediction can exhibit subgroup-level performance gaps even when overall predictive performance appears acceptable and that bias mitigation can reduce, but not fully eliminate, these gaps without trade-offs. By demonstrating the importance of
fairness-aware evaluation and transparent reporting of subgroup performance, this study provides practical insights for the responsible and context-sensitive use of ML models for risk stratification and care prioritization in MOUD treatment settings.
Read the full abstract ▾
All authors Tongnian Wang, Carolina Vivas-Valencia, Cici Bauer, Yanmin Gong, Kim-Kwang Raymond Choo, Yuanxiong Guo
Computers and Society · Cross-listed from cs.DL
A Pathway for Assessing Grey Literature: Leveraging AI to Extract Conference Metadata and Organiser Information from Calls for Papers
Angelo Salatino, Francesco Osborne, Alexis Vizcaino et al. (+2 more)
Despite its importance, grey literature, including Calls for Papers (CfPs), remains largely overlooked in Metascience and Scientometric analysis due to its unstructured, highly heterogeneous format, which traditional tools struggle to process at scale. However, Large
Language Models now offer a pivotal opportunity to devise innovative tools for systematically harvesting and processing such data. In this paper, we introduce COCI, an AI-based framework that automates the extraction of granular, structured metadata from raw CfP text. COCI employs a multi-stage pipeline for entity extraction, followed by author disambiguation against OpenAlex and semantic mapping of topics and conference series. This process identifies key data points, including conference editions, geographic locations, and comprehensive lists of organisers, along with their specific roles and affiliations. By structuring this previously inaccessible information, COCI establishes a foundation for the systematic analysis of grey literature, enabling new research opportunities and shifting the scholarly focus towards non-publisher-based events.
Read the full abstract ▾
All authors Angelo Salatino, Francesco Osborne, Alexis Vizcaino, Aliaksandr Birukou, Enrico Motta
Computers and Society · Cross-listed from cs.AI
Who Delegates to AI? Evidence from Agent Configurations in Github
Hyeongjae Lee, Jihyang Cheon and Lanu Kim
A growing body of literature measures the extent to which occupations are exposed to AI, yet existing measures capture where AI could perform tasks rather than whether workers have actually adopted it. We introduce a distinct tier of exposure, delegated exposure, which records whether a worker has committed a task to AI by
embedding it into a structured workflow. We operationalize this concept through the Agentic Adoption Index (AAI), measuring how closely an occupation's tasks align with the agentic routines that practitioners have built and shared. Using semantic
embeddings of roughly 888,000
agent skill specifications from public GitHub repositories, we compute their similarity to nearly 18,000 O*NET task statements and aggregate these scores to the occupational level. We present three main findings. First, the occupations where task delegation concentrates differ sharply from those identified as most vulnerable by pre-AI automation frameworks. Second, the AAI aligns more closely with measures of technical capability than with measures of current conversational LLM use. Third, for occupations requiring a bachelor's degree or less, the AAI increases alongside average wage levels; however, this relationship reverses for occupations requiring a master's degree or higher, where adoption declines among higher earners. These patterns replicate on an independently collected corpus of
agent skills from the Manus Skills Marketplace. This lower adoption among highly educated, high-earning workers may reflect tasks that inherently resist advance specification or professional discretion over the pacing of workflow codification. Distinguishing these mechanisms will require longitudinal measurement.
Read the full abstract ▾
Computers and Society
Six misconceptions about large language models: A minimal model and diagnostic taxonomy
Zhicheng Lin
Large
language models (LLMs) are now embedded in scientific, educational, and governance workflows, with debates centering on their capabilities, mechanisms, and impacts. Yet these debates remain structured by persistent folk theories–intuitive, informal explanatory models that guide attitudes and actions. Deflationary slogans ("just autocomplete," "
stochastic parrots," and "average of the internet") and anthropomorphic framings ("emergent
agents" and "proto-minds") each capture genuine features of current systems but mistake those features for the whole. This Perspective proposes a minimal working model of LLM-based systems centered on four distinctions: between pretraining and deployed systems; between the learned distribution and particular samples; among parametric, contextual, and external
memory; and between task competence and agency. The model is used to diagnose six misconceptions about LLMs: next-
token prediction, regression to the mean, training-data regurgitation, model
memory,
alignment, and understanding. For each, the analysis identifies what the misconception gets right, which distinctions it conflates, and what follows for capability evaluation, system design, and governance. Applied to publisher AI policies as governance case studies, the framework shows both how policy
language can conflate these distinctions and how such errors can be corrected. The model thereby avoids the parrot-mind binary by treating LLMs as simulators of discourse and task performance, offering a diagnostic toolkit for locating and correcting the errors these folk theories perpetuate.
Read the full abstract ▾
Computers and Society
A Scoping Review of Methods to Measure the Energy and Carbon Footprint of Web Tracking and Advertising
Nils Bonfils and Christoph Becker
The environmental impact of web tracking and advertising is increasingly receiving
attention as the ICT sector's carbon footprint keeps rising. Yet the scholarship addressing this question remains scattered across disciplines and inconsistent in its terminology. This paper presents a scoping review of the literature on methods for measuring the energy and carbon footprint of web tracking and advertising. From an initial pool of 46 articles identified through a structured title-based search on Google Scholar, we arrived at a final corpus of 15 papers, from which we identified five distinct methodological approaches: ad blocking, controlled environment, replaying ads, traffic flow analysis, and literature-derived estimation. This review provides a structured overview of the current methodological landscape and a foundation for more comprehensive environmental accounting of the ad tech ecosystem.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
Are LLMs becoming similarly creative? Evidence from three years of models
Nirav Patel, Josiah Crossman, Eva Aggarwal et al. (+1 more)
Many
benchmarks track Large
Language Model (LLM) performance on tasks with verifiable answers, but less is known about how LLM performance is evolving on open-ended tasks, where creativity, originality and diversity may matter as much as quality. As LLMs increasingly support human ideation and creative work, understanding trends in LLM performance on open-ended tasks is critical. This paper presents a preliminary analysis of LLM creative outputs spanning three years of model releases, examining model responses to Infinity-Chat100, a real-world collection of open-ended user queries, and the Alternate Uses Task, an established psychometric creativity assessment. Using sentence-
embedding similarity, we examine trends in LLM responses to these
prompts. Our findings show a statistically significant decrease in model output diversity over time, suggesting that LLM outputs may be converging in creative substance across models. If this trend persists, LLM-driven homogenization may progressively diminish human agency in human-AI co-creative work, demanding careful consideration of LLMs' role in the human creative process.
Read the full abstract ▾
All authors Nirav Patel, Josiah Crossman, Eva Aggarwal, Emily Wenger
Computers and Society · Cross-listed from cs.SI
Social.Wiki: A Web Held in Common
Theia Henderson, Carmel Schare, Ana Dodik et al. (+4 more)
Many of the websites people depend on have owners whose interests are not fully aligned with their users. We address the root of this problem by presenting a reimagining of the web where sites are not owned at all but are instead collaboratively produced like Wikipedia articles. We call the system Social$.$Wiki because it supports the co-creation of interactive social sites, such as those for microblogging, messaging, dating, gaming, ride sharing, and so on. With off-the-shelf AI tools, people with little or no programming experience can edit these sites to better reflect the needs and preferences of their communities. Social$.$Wiki builds on ideas from collaborative malleable software systems such as Webstrates, but is designed for public participation rather than use only within small, trusted groups. To this end, Social$.$Wiki includes governance to mitigate conflict. To accommodate diverse governance preferences, our model of "plural governance" lets people independently choose the policies that determine which edits to a site they see. Social$.$Wiki also implements a granular security model to protect personal data in a malleable environment. Complementing the decentralized design and governance of Social$.$Wiki sites, both site edits and within-site data are stored on Graffiti, a decentralized infrastructure, decoupling the ownership of underlying servers from the ownership of sites. We evaluate Social$.$Wiki through case studies that demonstrate the range of sociotechnical structures it supports, as well as through deployments at a hackathon and in the wild.
Read the full abstract ▾
All authors Theia Henderson, Carmel Schare, Ana Dodik, Clemens N. Klokmose, Ziv Epstein, David D. Clark, David R. Karger
Computers and Society
Navigating Epistemic Monocultures in AI-Driven Science: A Simulation Study
Sina Fazelpour, Joseph O'Brien and Hannah Rubin
AI integration into scientific communities promises accelerated discovery but raises concerns about detrimental homogenization. We develop an NK landscape model to explore these promises and risks. We find that non-personalized AI systems that offer uniform guidance yield benefits only under a narrow conjunction of problem structure, practices, and baseline research capabilities, becoming harmful otherwise. We implement two proposed mitigations: randomization and personalization. While randomization's utility remains restricted to decomposable problems, personalization can enhance diversity, enabling benefits across a broader range of conditions. Crucially, these benefits are not automatic, but depend on effective institutional adaptation, requiring new standards and practices.
Read the full abstract ▾
Computers and Society
Multi-Tier Mentorship with AI-Assisted Development: Authentic Engineering for K-12 and Undergraduates
Kelly Yuan, Ronald Liu, Daniel Crawford et al. (+1 more)
K-12 students often possess creative engineering ideas but lack technical skills to build them, while undergraduates have coding expertise but few opportunities to lead real-world projects or mentor others. The rapid development of AI-assisted tools offers a potential bridge to connect these groups, yet the structure for effective K-12 and university collaborations remains underexplored. This paper introduces a multi-tiered mentorship framework enabling high school students to engage in authentic engineering through AI-assisted development using large
language models and AI
agents, while undergraduate mentors provide architectural oversight. We test this framework through LuckyTag, a
privacy-preserving NFC-based lost-and-found system. The model positions high schoolers as product leads, undergraduates as technical architects, and faculty as minimal-intervention advisors. A pilot with four high school students, three undergraduates and two faculty yielded survey data showing high perceived barrier removal and gains in system architecture understanding. Thematic analysis reveals that AI amplifies rather than supplants mentoring demands, requiring human oversight for logic and security. These findings suggest a hybrid model for equitable K-12 and university collaboration on computing integration that emphasizes "AI micromanagement" and architectural
reasoning over traditional syntax.
Read the full abstract ▾
All authors Kelly Yuan, Ronald Liu, Daniel Crawford, Weihao Qu
Computers and Society
Qualified Cross-References as a Verification Method: The Normative Environment of the EU AI Act
Nicola Fabiano
Legal cross-references are commonly represented as links between instruments or provisions. In a curated legal knowledge base, a link must also identify the legal character of the interaction, its supporting provisions and conditions, and remain consistent from either instrument. This paper presents a provision-level model and a protocol for qualified cross-references, developed through a bilingual corpus of fourteen instruments surrounding Regulation (EU) 2024/1689 (the AI Act). The model distinguishes direct textual reference, bounded presumption of conformity, substantive interaction without textual reference, mediated intersection, and institutional analogy; applicative interaction and definitional overlap are independent dimensions. Its methodological contribution is bidirectional inversion: a relationship documented from act A towards act B is reconstructed from B's perspective against both instruments. This verification tests provisions, qualification, direction, and conditions before deciding how to render the relationship from either side. Applied during construction, the protocol surfaced six incorrect article references, three inaccurate legal qualifications, and one divergence between two published descriptions of the same interaction. A reference to Regulation (EU) 2019/881 illustrates why qualification matters: the AI Act's bounded cybersecurity presumption differs from other product legislation's uses of the same certification framework. The contribution is a map of one regulatory environment and an explicitly specified method for making curated cross-reference knowledge bases inspectable and internally testable. A later, separately scoped corpus audit is discussed among the limitations, without pooling its findings with the original measurements; limited access to historical inputs constrains independent reproduction.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.CL
ChildSafeAds Shared Task 2026: Commercial Content in Child-Facing YouTube Videos
Thales Bertaglia, Catalina Goanta, Gerasimos Spanakis et al. (+1 more)
ChildSafeAds is a shared task on commercial content in YouTube
videos likely to reach children and teenagers. The
dataset contains 3,360
videos from 939 channels, each starting with a sponsor segment submitted to SponsorBlock. We pair each segment with its transcript,
video, and channel information, plus a sales or service page linked from the
video description. Systems determine the type of offer being promoted (ST1), assign product categories (ST2), and identify potential legal risks (ST3). Evidence is divided into four cumulative access levels, from the transcript to the linked page, so performance can be compared against the cost of collecting additional data. YouTube's "Includes paid promotion" label was absent from 45.5% of the
videos. GPT-5.4 produced the labels after the organiser team, including a legal expert, reviewed samples and iterated on the taxonomy,
prompts and model choices. GPT-5.6-luna independently labelled the development set to measure cross-model agreement. The evaluation received 21 entries; the highest mean macro-F1 was 0.735, while the best transcript-only entry reached 0.652. Across four system papers, the linked promotional page mainly helped identify what was being sold and added little to compliance-risk classification. The competition also exposed sensitivity to rare labels, advertiser overlap across splits and substantial variation in the ST3 labels.
Read the full abstract ▾
All authors Thales Bertaglia, Catalina Goanta, Gerasimos Spanakis, Gunes Acar
Computers and Society
LearnAI: Just-in-Time AI Co-Creation Across Disciplines at a University
Weihao Qu, Ling Zheng, Chris Buzaid et al. (+1 more)
As generative AI reshapes professional and educational practice, institutions face a challenge: how to support diverse learners, from non-coders to advanced students, in building confidence and practice with AI-supported problem solving. Most institutional responses bifurcate into conceptual workshops for general audiences or technical courses for computer science majors, leaving few spaces where mixed-ability learners can engage common AI tasks at levels matched to their prior experience. This experience report presents the LearnAI Framework, a two-layer model for just-in-time AI co-creation piloted at a comprehensive teaching university. The Wide-Exposure Layer embeds short presentations in existing courses to build AI awareness at scale, reaching students and faculty across 18 courses in five disciplines. The Customized Co-Creation Layer provides opt-in, one-on-one sessions where clients work with trained undergraduate tutors through a 5-Stage Pedagogical Script: Problem Framing, Tool-Task Mapping, Iterative Co-Prompting, Deployment and Verification, and Ethical Reflection. Over two semesters, 35 clients co-created 36 portfolio websites and over 20 deployed web applications. Interviews with five clients and two tutors suggest a recurring change in how clients described AI use, shifting from treating AI as a passive answer machine to engaging it as a collaborative tool under human direction. A small paired pre/post AI readiness
dataset (N = 7) provides preliminary descriptive context, and tutor accounts document how the pedagogical script was enacted and adapted across client types. We report on boundary cases including clients who felt overwhelmed and respondents who deliberately rejected AI use. This paper contributes a practical, adoptable framework with initial evidence from a single institution.
Read the full abstract ▾
All authors Weihao Qu, Ling Zheng, Chris Buzaid, Daniel Crawford
Computers and Society · Cross-listed from cs.AI
Grouping the Stochastic Machine: Precision, Not Capability, as the Frontier Metric for AI Systems
George Andrikopoulos
Frontier
language models are compared, marketed, and benchmarked on capability – what their best or average output can achieve. I argue this measures the wrong axis. The models have saturated accuracy: their mean output lands on the target. What now separates one system from another in practice is precision: how tightly concentrated their outputs are around that target across repeated, identical requests. Borrowing the marksman's distinction, capability is where the average shot lands; reliability is the size of the group. I make three claims. First, precision, not capability, is the frontier differentiator between systems, and
benchmark culture systematically fails to measure it, reporting central tendency rather than spread. Second, precision is measurable, cheaply and without circularity, by running a fixed suite of deterministically scored tasks many times at fixed temperature and computing the per-task consistency of outcomes – no model-in-the-loop grader required. Third, the measurement is not merely descriptive but decision-guiding: it separates consistent failures (a tight group off-centre, correctable by the operating discipline of Paper 1 – a sight adjustment) from scattered failures (a wide group, correctable only by changing the model or its sampling – a rifle problem). I define a grouping metric, specify a harness, and show how tracking a human-AI pair's grouping over time yields the compounding signal that Paper 1's field study requires. A first real run, since replicated, illustrates both the method and its most important limit: one measured gap was closed completely by a single rule (0/5 -> 5/5), while a suite of tasks authored from the rules themselves found no value, because a frontier model already embodies explicit good practice – establishing that a discipline's worth is found by measurement on real work, not constructed from its own rulebook.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.LG
Leaf Values as Coordinates: Exact Contrastive Explanation for Gradient-Boosted Ensembles
Emanuele Luzio
A gradient-boosted ensemble predicts by summing one leaf value per tree. Read those values as coordinates rather than as intermediate results, and every instance becomes a point in R^M on which the model acts linearly: the score is the sum of the coordinates. This small change of view makes contrastive explanation exact. The difference between two instances is a vector that is identically zero wherever they share a leaf, so the gap between a rejected applicant and an accepted one is carried by a handful of coordinates, each traceable to a real split in a real tree. Nothing is fitted, sampled, or assumed additive in features – the additivity is already there, in the right space. We build a recourse method on this representation and evaluate it on five tabular
datasets under repeated cross-validation. Its recommendation reconstructs the model's own decision to 6.2 x 10^-15, so an auditor can re-check the arithmetic without the model. On the credit
datasets it is Pareto-non-dominated on effort against realism. And when recommendations are restricted to changes the subject could actually make – not their age, not a settled delinquency – it retains 58% of its validity where the strongest baseline retains 41%, a distinction the standard evaluation cannot see because it never asks whether a recommendation can be carried out.
Read the full abstract ▾
Computers and Society
Hot Games: Towards a Holistic Assessment of the Planet Warming Emissions of Video Games based on 2024-2025 Data
Mike Hazas, Kevin C. Dalli, Arjun Menon et al. (+2 more)
Following on recent reports on specific platforms or companies, this paper provides an assessment of the global impact of the production and use of
video games. It draws together publicly available data on game development, hardware, games sold, download sizes, time spent playing games on different platforms, and subscriptions to multiplayer and cloud game services. It provides an update to figures published 2020 and 2022. Crucially, our account of emissions related to
video games considers a wide range of categories, yet contains enough detail to be critiqued and improved in the future.
Read the full abstract ▾
All authors Mike Hazas, Kevin C. Dalli, Arjun Menon, Ben Abraham, Ossian Nordgren
Computers and Society
Open at the Edge, Captured at the Center: llama.cpp and the Political Economy of Local AI Inference
Woohyeuk Lee, Hanlin Li and David Gray Widder
Critical scholarship on open AI has focused on model releases and cloud ecosystems, leaving the local inference infrastructure that makes open-weight models runnable on user-owned devices largely unexamined. We address this gap through a mixed-methods analysis of llama$.$cpp, combining 7,681 merged pull requests from March 2023 through March 2026 with repository discussions, corporate statements, and contributor blogs. We show that local inference broadens participation at execution while relocating capture into the infrastructure that makes execution possible. Through hardware backends, model integration labor, and Hugging Face's February 2026 absorption of the project, we document how control shifts to hardware vendors, model distributors, and core maintainers while model owners and individual contributors bear the cost of making models runnable. These dynamics suggest that preserving openness outside the cloud requires
attention to the infrastructure that makes models runnable, not just to the models themselves. This calls for policy mechanisms–analysis of format dependencies and vendor influence, model compatibility requirements, and sustained public funding for inference tooling–that extend beyond model release conditions to the infrastructure layer.
Read the full abstract ▾
Computers and Society
Epistemic Subordination: Generative AI and the Infrastructure of Knowledge
Gilad Abiri and Emanuel V. Towfigh
Generative AI does not merely produce biased outputs. It encodes the majority's way of knowing as the default infrastructure of knowledge itself. We call this epistemic subordination. The training process compresses the full breadth of human expression into a single probabilistic model whose statistical baseline reflects the
languages, assumptions, and cultural frameworks of the dominant culture. Minority epistemologies are not excluded but absorbed: present in the training data, yet structurally subordinated in the output. The result is not a collection of discrete biases that can be audited and corrected. It is an epistemic condition embedded in the architecture from which all outputs emerge. This unified harm cuts across three legal domains – anti-discrimination law, cultural and linguistic rights, and democratic viewpoint pluralism – and each fails to address it for the same structural reason: existing law regulates downstream, at the level of decisions and applications. The remedy must match the site of harm. If epistemic subordination is produced at the level of model training, then law must learn to govern at that level.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.AI
Sanyu Studio: A Multi-Agent System for Art-Historical Narrative Construction
Zhaoxi Wei, Hongye Yang and Shuyuan Tian
Amid concerns that generative AI may standardize art interpretation, this paper examines whether LLM-based interaction can support plural art-historical narrative construction. We present Sanyu Studio, a multi-
agent dialogue system that models 321 Sanyu oil paintings as
agents with fact, interpretation, organization, and
memory-filtering mechanisms. Based on a seven-day workshop with eight art-university participants, the study shows that user
prompts, evidence organization, and cognitive tendencies shaped divergent yet coherent versions of digital Sanyu. The findings suggest that, under conditions of limited historical evidence, AI can amplify human agency and offer public audiences an interactive entry point into art-historical interpretation.
Read the full abstract ▾
Computers and Society
Turning interest into institutional change: teaching advocacy for sustainable research
Kirsty Pringle, Lorna Smith, Erinma Ochu et al. (+1 more)
Systemic change across the digital research landscape is required to reduce the environmental impact of digital research, but while many researchers and technical professionals are motivated to act, they often lack the skills required to translate motivation into lasting organisational change. We present an open-access course that teaches the foundations of advocacy and organisational change to researchers, research software engineers, and research technical professionals. Structured around the UNICEF five-step advocacy cycle, the course covers stakeholder analysis, power mapping, coalition building, storytelling, framing and messaging, and evaluation. It is grounded in the UK policy landscape, including the Concordat for Environmental Sustainability and the UKRI Environmental Sustainability Strategy, and uses a fictional case study to make concepts concrete. The course is designed as a dual-layer resource in which workshop slides and extended self-study notes are contained in a single source material. A short pilot was delivered at the NetDRIVE Net Zero DRI Summer School in June 2026. The course is freely available under a CC BY 4.0 licence.
Read the full abstract ▾
All authors Kirsty Pringle, Lorna Smith, Erinma Ochu, Greg Wilson
Computers and Society
CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks
Pattaraphon Kenny Wongchamcharoen, Kris Gulati, Min Min Fong et al. (+1 more)
Most LLM
benchmarks rank models on their ability to automate work tasks. In practice, however, models are often used to assist other (human or LLM)
agents. The question that drives model selection is therefore not only which model produces the best output, but which model most improves the work of another (weaker)
agent. We introduce a unified framework that evaluates the capability of models to automate and augment another
agent's performance. Across seven economically grounded real-world tasks, an assistant model writes assistance text for a standardized lower-capacity worker model, which produces the deliverable. In automation mode, the assistant produces the output directly. Outputs are scored through blind pairwise comparisons by an LLM judge panel with task-specific rubrics, replicated across ten runs. Rankings across the two regimes are only modestly correlated, and the automation winner loses augmentation on five of seven tasks. Assistance is not reliably positive. The unaided worker outranks every assisted condition on three tasks, and only one model's guidance beats no guidance on average. These results suggest that automation ability is an incomplete proxy for assistance quality, motivating
benchmarks that evaluate models according to the roles they play in human-AI and multi-
agent systems.
Read the full abstract ▾
All authors Pattaraphon Kenny Wongchamcharoen, Kris Gulati, Min Min Fong, Abhishek Nagaraj
Computers and Society · Cross-listed from cs.HC
Measuring Proof Burden in Public Bounty Listings: A RentAHuman Case Study
Iman YeckehZaare
Online bounty
markets let requesters advertise paid tasks. Workers may be asked not just to complete a task but to prove it, and proof can mean exposure: revealing identity or location, using a personal account, posting publicly, acting in the physical world, or repeated evidence at later checks, none disclosed by the posted price. We call these advertised requirements proof burden and measure them on RentAHuman, a 2026
market publicized as a place for AI
agents to hire humans. We study what listings request, not what workers submit or experience. We manually audited a nonrandom May 31, 2026 snapshot: every listing our searches returned from RentAHuman and Human Pages, another such
market (981 listings, all but one from RentAHuman). Two independent coders recorded 13 features (11 kinds of evidence, recurring monitoring, physical-world action) and our 0-5 Proof Burden Score; a blinded third resolved all disagreements. A planned content screen leaves 779 bounty/task listings as the primary population; 438 (56.2%) score 4 or 5, spanning 154 distinct feature combinations: a checklist, not a single score, tells workers what a listing entails. Platform metadata labels some requester accounts as
agents or bots. Exploratory comparisons show physical-world action, location proof, or recurring monitoring in 75.0% of
agent-or-bot-labeled versus 55.3% of human-labeled listings, though score-4-or-5 shares did not clearly differ. The labels are self-reported or platform-assigned, the
agent-or-bot-labeled listings come from only 20 displayed names, and the comparison was chosen post hoc, after seeing the data: a hypothesis, not a confirmed difference. We contribute the 13-requirement vocabulary, the adjudicated manual audit, and this descriptive case study; the score is a secondary screening summary. The study offers no worker-validated measure or automated detector yet.
Read the full abstract ▾
Computers and Society · Cross-listed from cs.SI
Longitudinal Relational Publics and their Discursive Overlap with Issue Publics
Alyssa Hasegawa Smith, Judith Gilsbach, Ahana Bhattacharya et al. (+2 more)
Online discussions of political issues do not always happen in places explicitly dedicated to political talk; they also arise in online spaces focused on at least nominally apolitical interests, identities, and/or places. Whatever one's normative view of politics entering these “online third spaces,” understanding who brings political issues into them, and when, requires studying these spaces at scale. In turn, studying these spaces at scale requires a construct that captures both who is speaking and who is listening, and that holds up over time. Building on Bruns' distinction between participant-centered personal publics and post-centered issue publics, we introduce the longitudinal relational networked public (or, simply, the longitudinal public): the coupling of discourse produced by a socially connected set of creators with the durable
attention their shared audience gives it. The longitudinal public departs from related relational constructs in three ways: it is anchored in the
attention patterns of a non-elite, population-level audience; it treats within-public structure as an object of analysis rather than assuming homogeneity; and it incorporates the audience as a force that shapes creator discourse. In a case study, we identify 150 longitudinal publics from the following ties of a panel of Twitter/X users matched to U.S. voter records, then measure their discursive overlap with the electoral politics and Black Lives Matter issue publics across 2020, a period spanning the murder of George Floyd and the general election. We find that the spaces that best fit the idea of a third space have the most politically heterogeneous audiences, and thus the most potential room for cross-partisan talk. This result, among several others, shows how a relational, audience-aware construct can reveal where and through whom political talk enters everyday online life.
Read the full abstract ▾
All authors Alyssa Hasegawa Smith, Judith Gilsbach, Ahana Bhattacharya, Holliday Sims, Kenneth Joseph