Search arXiv⌕ Search
Home/Archive/arXiv

arXiv · 1901.00912

Arming the public with artificial intelligence to counter social bots

Kai-Cheng Yang·Onur Varol·Clayton A. Davis·Emilio Ferrara·Alessandro Flammini·Filippo Menczer
View original source ↗Read source document ↗Export metadata ↓

Abstract

The increased relevance of social media in our daily life has been accompanied by efforts to manipulate online conversations and opinions. Deceptive social bots -- automated or semi-automated accounts designed to impersonate humans -- have been successfully exploited for these kinds of abuse. Researchers have responded by developing AI tools to arm the public in the fight against social bots. Here we review the literature on different types of bots, their impact, and detection methods. We use the case study of Botometer, a popular bot detection tool developed at Indiana University, to illustrate how people interact with AI countermeasures. A user experience survey suggests that bot detection has become an integral part of the social media experience for many users. However, barriers in interpreting the output of AI tools can lead to fundamental misunderstandings. The arms race between machine learning methods to develop sophisticated bots and effective countermeasures makes it necessary to update the training data and features of detection tools. We again use the Botometer case to illustrate both algorithmic and interpretability improvements of bot scores, designed to meet user expectations. We conclude by discussing how future AI developments may affect the fight between malicious bots and the public.

Explore related subjects

cs.CY

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Kai-Cheng Yang, Onur Varol, Clayton A. Davis, Emilio Ferrara, Alessandro Flammini, Filippo Menczer. 2019-02-06. Arming the public with artificial intelligence to counter social bots. https://doi.org/10.1002/hbe2.115

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Political Persuasion and Endorsement in Large Language Models

Large Language Models (LLMs) are increasingly employed as proxies for human behavior in computational social science. However, their tendency to internalize biases from training data raises concerns about their reliability in politically sensitive domains, specifically regarding their endorsement of persuasive language. In this work, we examine whether LLMs endorse persuasion-infused messages and whether partisan persona prompting modulates such endorsement. We evaluate six LLMs from different geographic regions on content annotated with persuasion techniques drawn from real-world media sources, measuring the likelihood of endorsement using a five-point Likert scale. The models are prompted as either a neutral social media user or as a user with left- or right-leaning political views. Results show that, without political conditioning, LLMs generally do not endorse messages containing persuasion techniques, although model-level differences emerge, and that partisan persona prompting increases polarization of endorsement, particularly for persuasion-infused content. Endorsement further varies across persuasion techniques and topics. These findings reveal how political conditioning influences LLMs' endorsement behavior toward persuasion-infused content, raising concerns about their deployment as agents in politically sensitive environments and complicating their use as reliable simulators of human political cognition.

cs.CY↗
arXiv2026

When Technically Plausible Advice Is Unsafe: A Cross-Ecosystem Measurement of Online Support for Technology-Facilitated Abuse

Technology-facilitated abuse (TFA) creates an adversarial setting where sound cybersecurity advice can be unsafe: changing credentials or resetting devices may alert an abuser, destroy evidence, or increase escalation risk. Victims seek guidance from search engines, peer forums, and conversational AI, often evaluated for relevance and correctness rather than contextual safety. We measure whether these sources meet victims' needs. From a decade of r/Stalking narratives, we construct 2,797 victim-derived queries spanning 11 misuse categories. We analyze 27,162 Google webpages, 2,476 Reddit query--thread responses, and 250 responses from three general-purpose LLMs and two survivor-support chatbots. Our framework measures technical quality and damaging guidance, plus secondary-link integrity on webpages, toxicity on Reddit, and trauma-informed support in conversational systems. We find failures & risks that relevance, accuracy, or actionability alone do not capture. Web Search and conversational systems frequently return relevant information; Reddit responses are less consistently relevant and actionable. In our evaluated accuracy sample, 17.3% of webpages, 13.3% of Reddit threads, and 19.6% of conversational AI responses contained damaging guidance. Further, 65.5% of victim queries led to a webpage with a secondary URL flagged by multiple VirusTotal engines, over 20% received a toxic Reddit comment, and every conversational system produced guidance that overlooked escalation risk. Specialization did not guarantee better support: HopeChat underperformed general-purpose LLMs on several dimensions, while Ruth remained limited in trauma-informed support. These findings expose a gap between technical quality and contextual safety. Safe TFA assistance requires risk-aware recommendations, trustworthy sources, uncertainty communication, and human support, beyond technically plausible answers.

cs.CY↗
arXiv2026

Auditing Institutional Heterogeneity for Generative AI in Patient Education: A Large-Scale Study of 102 US Transplant Handbooks

Health systems are rapidly deploying generative-AI assistants that answer patient questions from institution-authored education materials, on the premise that grounding in local content yields consistent guidance. Do the underlying documents themselves agree? We use a structured-output large-language-model judge to audit 1{,}772{,}261 pairwise comparisons across 102 patient-education handbooks from 23 US solid-organ transplant centers, paired with 1{,}115 patient-derived questions (TransplantQA). Four findings bear directly on deployment: (1) same-center cross-organ agreement exceeds cross-center same-organ agreement by $0.024$ in the primary analysis (Holm-adjusted $p=0.011$), with sensitivity to document selection; (2) information gaps concern topics relevant to underrepresented subgroups, with reproductive health a \emph{double jeopardy}: 82\% absence and 86\% judge-rated high significance among divergent/contradictory pairs; (3) judge-derived themes form 991 clusters, with immunosuppression and pregnancy timing among the highest judge-rated priorities; (4) question and observed-coverage features predict high-divergence questions retrospectively (AUC $0.77$). We discuss implications for deploying patient-facing generative AI in transplant care.

cs.CY↗