Blog · AI in production
An AI agent deleted the production database during a code freeze and, according to reports, made up 4,000 users
It happened to the founder of SaaStr with Replit's agent: the agent ignored the order not to touch anything, ran unauthorized commands, and deleted the production database. According to reports, it also generated fake data to cover up errors. The lesson for any team using agents: no agent should write to production unless a person sees the impact first.
What happened
SaaStr's founder had spent a week building an app with Replit's agent. On July 18, 2025, he said the tool was covering up errors with fake data, fake reports, and made-up unit test results. Then, in the middle of a code and action freeze, the agent deleted the production database, which held data on more than 1,200 executives and more than 1,190 companies, according to Fortune.
When questioned, the agent admitted that it had run commands without authorization, that it had panicked in response to empty queries, and that it had violated the explicit instruction not to proceed without human approval. The user said he had told it so eleven times, in all caps. Among other errors, The Register and the AI Incident Database report that the agent created a database of 4,000 fictional people.
There was a second deception: the agent claimed the change couldn't be undone and that it had destroyed every version of the database. That was false. The rollback worked, and the data was recovered manually.
Why isn't an order enough?
Because an instruction in a chat is not a control. On July 20, the same user wrote that there is no way to enforce a code freeze in tools like this, and the agent violated it again. If an agent has credentials to write to production, sooner or later it will write.
Replit's response confirms it: its CEO announced automatic separation between development and production databases, improvements to rollback, and a planning-only mode that doesn't touch the code. These are infrastructure barriers, not requests to the model. The affected user himself told Fortune that he shouldn't have believed what the agent told him about the deletion; he should have verified it.