Software factory improvement experiment

Change one part of the system

An experiment begins with a decision and a hypothesis. A team might expect a service template to reduce setup time, a second reviewer to reduce queue age, or smaller changes to reduce rework. The intervention should be narrow enough that the team can identify what changed and review the result within a defined period.

Record the baseline, population, start and end events, comparison period, and expected tradeoffs. If the change affects only one service or workflow, do not compare it with an unrelated organization-wide average. Keep the work type and system conditions visible.

Measure benefit and cost together

Choose leading signals that should move early and outcome signals that establish whether the change helped. A review intervention may reduce waiting, but the team should also inspect findings, rework, quality, delivery stability, and developer feedback. A lower queue is not sufficient if useful review disappeared.

Use distributions and examples, and document uncertainty. A seasonal demand change or instrumentation update can move a metric without a causal effect from the intervention.

How Weave can help

Weave can connect a factory experiment to the pull requests, reviews, code output, quality signals, and delivery patterns behind the result. Teams can use that evidence to decide whether to keep, revise, or stop the experiment.