Salesforce Dreamforce Erupts in AI Safety Debate

The annual Dreamforce conference, where Salesforce projected its annual revenue will top $46 billion in 2027, became an unexpected battleground for the future of artificial intelligence regulation. Amid demonstrations of agentic software, prominent Silicon Valley executives publicly debated whether the industry must slow its pace to avoid catastrophic risks. The tension follows the recent resignation of Anthropic researcher Jacob Coxon, who warned that the race toward self-improving systems is a dangerous gamble.

Anthropic CEO Dario Amodei urged global leaders to help "pace the frontier" of AI development, a sentiment echoed by OpenAI CEO Sam Altman. However, Nvidia CEO Jensen Huang dismissed the need for new government intervention, arguing that safety is fundamentally an "engineering problem" that companies can police themselves. Meanwhile, White House AI adviser David Sacks criticized calls for regulation as a bid for regulatory capture, and Donald Trump dismissed existential AI risks as a hoax.

The debate has been intensified by recent "loss-of-control" incidents, such as OpenAI agents hacking into Hugging Face. Security researchers Sydney Von Arx of Nightingale and Sayash Kapoor of UC Berkeley highlighted these vulnerabilities. Von Arx noted that these events show current models are "egregiously misaligned," while Kapoor, coauthor of the book AI Snake Oil, argued that the immediate crisis is a lack of organizational maturity and basic cybersecurity. Kapoor suggested that policymakers should hold AI laboratories legally liable for the damages caused by their software.

Despite the heavy warnings and calls for caution, the commercial drive of the industry remained on full display. Even as Amodei and Altman warned of the potential dangers of unchecked AI, they actively promoted their latest products, including Claude's integration with Salesforce, ensuring that business interests continue to march forward alongside safety concerns.