AI Scammers Now Outperform Humans at Building Trust With Victims

New research covered by Ars Technica finds that AI-powered scammers are now measurably more effective than human scammers at establishing trust with potential victims, based on experimental studies comparing AI-generated versus human-generated social engineering attempts. The AI systems were better at personalizing messages, maintaining conversational consistency, and avoiding the tells that typically alert savvy users to scams. For developers building consumer-facing AI communication tools, this research is a direct signal that trust-building capabilities in LLMs are a dual-use concern — the same fluency that makes an AI assistant useful also makes it a more effective manipulation tool. Platform and API providers may face increasing pressure to implement behavioral guardrails specifically targeting social engineering patterns. Developers should also consider how their own applications might be weaponized via prompt injection or API misuse to conduct trust-based attacks.