Measurement and experimentation

Agentic software factory

Also known as Agent-driven software factory, Agentic delivery system

By WeavePublished 1 min read

Definition

An agentic software factory is a software delivery system in which software agents perform one or more engineering tasks with access to tools, repositories, or environments under defined instructions and permissions. It requires observable outputs, human or automated review, and controls for failure and recovery.

An agent needs a defined work boundary

An agentic factory does not mean that an agent receives unlimited authority. A reliable workflow defines the task, context, tools, data, permissions, allowed outputs, checks, and stop conditions. An agent might inspect a repository, propose a patch, update a test, or open a pull request. Each action has a different risk and review boundary.

The factory should preserve an audit trail. Record the task, instructions, relevant context, tool calls, outputs, checks, approvals, retries, and final outcome. This makes a successful run understandable and a failed run recoverable.

Measure task outcomes and system effects

Track task success, time to useful result, human review effort, acceptance and rework, quality findings, failure recovery, cost, and delivery impact. A faster agent run is not useful if the resulting change is harder to review or creates more production risk. Compare similar task types and keep agent and human work distinguishable without pretending the boundary is perfectly observable.

How Weave can help

Weave can show how agent-associated changes move through the repository and delivery workflow. That gives teams evidence about the factory effect. Agent telemetry and explicit permission logs establish what the agent actually did and whether the workflow stayed within its intended boundary.

How this relates to Weave

Weave can help measure the software changes produced or assisted by agents through code output, review, quality, rework, and delivery context. Agent run telemetry, tool permissions, cost, task success, and human review records are needed to evaluate the agent itself.

Explore Engineering intelligence

Sources and further reading

  1. Continuous AI in practice: agentic CI, GitHub
  2. DORA 2025 State of AI-assisted Software Development Report, Google Research