Configuration drift is the gap between what a machine is declared to be and what it is actually running. The declaration is the desired state: the code, kept in version control, that says what every machine should hold. Drift is anything on the machine that the declaration does not account for, and anything in the declaration that never reached the machine.
Drift is easy to create and hard to see. A change made by hand, a change merged and never applied, a file written by a different version of the tooling: each leaves a machine that still works, and a declaration that no longer describes it.
None of this is new
In 2002, Steve Traugott and Lance Brown sorted every way of managing machines as divergent, convergent or congruent. The first is the one most places live with: “Divergence is characterized by the configuration of live hosts drifting away from any desired or assumed baseline disk content.”
The second pulls machines back: “Convergence is characterized by the configuration of live hosts moving towards an ideal baseline.” They credit the idea where it is due: “Convergence is a feature of Mark Burgess’ Computer Immunology principles”. The third, congruence, is “the practice of maintaining production hosts in complete compliance with a fully descriptive baseline”, and it has a particular answer to drift: “Congruent infrastructures rely on monitoring to detect deviations, and generally call for a rebuild when they have occurred.”
That sentence describes what we are building toward, twenty-four years later: a check that detects, and a rebuild from the declaration as the cure.
Why it matters more with an agent
An agent reads the declaration and believes it. It has no memory of the change someone made by hand last week, or of the merge nobody applied. A person who was there might remember that one machine is different. A fresh session sees only the code, and drift is exactly the part of the estate the code does not show.
A check only sees what it reads. When we retired an old key, the audit read every key file and reported clean, while the key was still registered in a database the audit never read. It stayed there for two weeks after being declared gone.
What finding it bought here
- A destroy guard declared on and running off. In the working session of 7 September 2026, the machine that holds the estate’s secrets was found protected in the code and unprotected on the hypervisor. It had never been applied. It is now on, verified on the machine.
- A plan that could finally say “no changes”. Until that same session, the provisioning code reported one pending change forever, so it could never tell us that the machines and the code agreed. Since 7 September 2026 it can, for the first time since 20 August 2026, and a difference now means something.
- A check that looks every hour. Since 14 September 2026 it runs four parts of the code in a mode that changes nothing and reports whether anything would change. Its first run found drift on two machines and nowhere else, fixed by the code that owns them, and the alert cleared at 20:24.
- Rebuilds. Deleting a machine and building it from the declaration removes every kind of drift at once, including the kinds no check reads.
Where we fall short
We are convergent, not congruent. Between rebuilds a machine is only as current as the last time someone ran its code against it. Rebuilds happen in campaigns, not continuously.
The hourly check covers four parts of the code. Everything else is compared with the running estate only when someone runs it.
Detection is not correction. When the check finds drift, a person runs the fix. Nothing converges on its own, and merging a change still does not apply it.