Why are the people building the most powerful AI so worried about what it could do?

Amid recent high-profile incidents like an Anthropic resignation and a hack on Hugging Face involving OpenAI agents, there's growing concern among researchers that the AI industry is pushing the boundaries of power too quickly without adequate safety protocols in place. This rush to develop advanced AI could lead to unforeseen risks, as the technology's potential for misuse or unintended consequences looms large. The implications of these developments are significant, highlighting the urgent need for better regulatory frameworks and safety measures to manage the powerful capabilities of emerging AI technologies.