Production readiness in the AI factory: what to check when agents write the code

Assume the environment is already what many teams are currently building. Requirements are no longer prose in a ticket but structured so an agent can work with them: acceptance criteria, domain terms, examples. There is a maintained arc42, a domain model, a record of architecture decisions. Agents implement, propose, refactor, decide in the small. The pipeline is green before anyone looks.

In that world the classic pre-deployment question — does it run, are the tests green, is there a rollback — has largely become trivial. It is answered automatically, often better than before. What has not become trivial: how do we know a change is right when nobody read every line?

That is not a question for the technology. It is the question an auditor, a large customer or a court will ask later — and one you cannot answer retroactively.

Five areas that matter then. Each with the question and what you can produce afterwards.

1. Provenance: where did this change come from?

A commit hash with a name beside it was the answer as long as a human wrote the code. Now an agent stands there, and the name beside it belongs to whoever pressed merge.

The relevant questions: what context was present — which version of the requirement, which slice of the domain model? Which model at which version? Which instruction triggered the run? And: will that still be reconstructable in six months, once the model has long been replaced?

What you can produce: an immutable trail per change — trigger, context version, model version, result, approval. Anyone already using an event stream as the foundation for business processes knows the pattern; here it applies to the development process itself. More under event sourcing.

2. Context is now a production artefact

While arc42, glossary and decision records were documentation, letting them go stale was merely annoying. Now they are input. A stale domain model no longer produces a misunderstanding — it produces code. Consistent, fast, and wrong.

Questions: who maintains these artefacts, and is that a named responsibility or a good intention? How do you notice the generation drifting away from business reality? Is there a moment where the business side confirms the model still holds?

What you can produce: a versioned context package with change history and a name behind it. This is exactly where event modeling earns its keep: it produces a picture the business and the team agree on — and that an agent can read.

3. Where the human decides

Not every change needs a review. Some do. The real work is naming those classes before the first incident names them for you.

Typical candidates: anything that moves money. Anything that decides access. Anything touching personal data. Anything under a duty of proof. Anything that changes an external commitment.

The real question is not whether the boundary exists, but where it is anchored. A team agreement holds until things get urgent. A boundary enforced in the pipeline holds then too.

What you can produce: a list of change classes requiring human intervention, technically enforced, with logged approvals.

4. Test behaviour, not implementation

When code becomes cheap, the value shifts into the test suite. And that is where this environment produces its subtlest problem: if the same agent generates implementation and test, the test may only verify that the implementation does what it does.

Questions: do your tests describe the expected business behaviour in the language of the domain — or do they mirror the structure of the code? Do the examples come from the business or from the model? Are there tests nobody may change without the business agreeing?

What you can produce: a set of business-worded acceptance scenarios derived from the requirement rather than the code — the only part of the suite you can still trust after generation.

5. Reversibility and pace

More changes per week means more opportunities to ship something wrong. That is not an argument against pace, but one for reversibility.

Questions: is reverting as cheap as shipping? Do you notice a fault before the customer does — or only in the ticket? How small is the smallest change you can roll back on its own?

What you can produce: a measured time from ship to detected deviation, and a tested revert per unit of change. Same question as in the classic production readiness view — it just applies several times a day now.

What does not go away

Operations. On-call. The question of who gets phoned at night. And above all: accountability. An agent can produce a change but cannot stand behind it. Who is really responsible after go-live is still decided by people, not tools — see ownership instead of roles.

The duty of proof does not disappear either. It gets stricter, because the volume of changes rises and the individual human behind each one becomes blurrier.

When you do not need this yet

If agents at your place produce suggestions a human reads in full before merging, then your existing review is the control point and this article describes a future, not your present. For internal tools without business impact the effort is not justified either.

The tipping point is easy to spot: it is the day of the first merge whose diff nobody read in full. From there on, the five questions above apply.

Short version

In an AI-driven development environment, production readiness shifts from does it run to can we prove how it came to be. Provenance of the change, maintained context, anchored decision boundaries, business-worded tests and cheap reversibility are the five places where that is decided. All of them are architecture questions — and none of them can be settled two weeks before deployment.