Outline of operation of the autobuilder network
At the heart of the system is the wanna-build database, which keeps track of package versions and states. quinn-diff compares the package lists for the target architecture against the list of source packages after every archive update and feeds a list of packages that need re-compilation into the database where they enter state Needs-Build.
All the build daemons (there can be more than one) query the database regularly for such packages and take some of them so that they go into state Building. Of course, humans also can take packages, e.g. in special cases where automatic compilation isn't possible. Here we also see the second purpose of wanna-build: It ensures that the same version of a package won't be built twice.
If everything goes well, a finished package can be uploaded later, which is another state Uploaded. After that it will eventually be installed into the Debian archive so it appears in the updated package list for the target architecture. This list will be merged into the database, so the package will go to state Installed and remains there until the next version of the source package.
There are several other states; they include: Failed is for packages that failed to build due to errors in the sources, and the errors are expected to be fixed in a successor version (after reporting the problem, of course). So a new version will directly enter Needs-Build, but with a warning that something was wrong with the previous version. Along with this state an error description is stored. State Dep-Wait is used when a package needs some other packages to be compiled but those aren't available yet and must be built before. This state stores a list of required packages and maybe versions, and if all of them are known to be installed the state changes back to Needs-Build.
As we have already seen, the build daemon takes packages from the database for compiling them. Let's look a bit closer: If it has some packages to build, it uses sbuild for the actual compilation process, and for each build a log is mailed to the maintainer of the daemon. He reviews the log and decides what to do with the package: upload it, set it to Failed or Dep-Wait and retry it, etc... If a positive acknowledgement (a signed .changes file) is received, the daemon moves it to an upload directory, from where all packages are uploaded by a cron job.
Looking at the log files is the only human intervention in the whole process if no errors happen. There are two good reasons for not further automating this: First, sometimes builds end with an ``OK'' result but the build nevertheless failed for reasons that are invisible to the machine. And second, directly uploading would require to automatically OpenPGP-sign the resulting files with a key without passphrase on the build machine. I considered this an unacceptable security hole.
The build script sbuild more or less just calls some standard Debian tools to compile the sources. It also helps with some common tasks and bookkeeping and with automatically installing the build-dependencies as requested by the package being built.
Content developed by Roman Hodek for the 6th International Linux-Kongress 1999; it was gently updated to reflect the current reality a bit more by Philipp Kern in 2009