25.9 Tracking for Multiple Machines

Contributed by Mike Meyer.

When multiple machines need to track the same source tree, it is a waste of disk space, network bandwidth, and CPU cycles to have each system download the sources and rebuild everything. The solution is to have one machine do most of the work, while the rest of the machines mount that work via NFS. This section outlines a method of doing so.

25.9.1 Preliminaries

First, identify a set of machines which will run the same set of binaries, known as a build set. Each machine can have a custom kernel, but will run the same userland binaries. From that set, choose a machine to be the build machine that the world and kernel are built on. Ideally, this is a fast machine that has sufficient spare CPU to run make buildworld and make buildkernel. Select a machine to be the test machine, which will test software updates before they are put into production. This must be a machine that can afford to be down for an extended period of time. It can be the build machine, but need not be.

All the machines in this build set need to mount /usr/obj and /usr/src from the same machine, and at the same point. Ideally, those directories are on two different drives on the build machine, but they can be NFS mounted on that machine as well. For multiple build sets, /usr/src should be on one build machine, and NFS mounted on the rest.

Finally, ensure that /etc/make.conf and /etc/src.conf on all the machines in the build set agree with the build machine. That means that the build machine must build all the parts of the base system that any machine in the build set is going to install. Also, each build machine should have its kernel name set with KERNCONF in /etc/make.conf, and the build machine should list them all in KERNCONF, listing its own kernel first. The build machine must have the kernel configuration files for each machine in /usr/src/sys/arch/conf if it is going to build their kernels.

25.9.2 The Base System

On the build machine, build the kernel and world as described in Section 25.7.8.2, but do not install anything. After the build has finished, go to the test machine, and install the built kernel. If this machine mounts /usr/src and /usr/obj via NFS, enable the network and mount these directories after rebooting to single user mode. The easiest way to do this is to boot to multi-user, then run shutdown now to go to single user mode. Once there, install the new kernel and world and run mergemaster as usual. When done, reboot to return to normal multi-user operations for this machine.

After verifying that everything on the test machine is working properly, use the same procedure to install the new software on each of the other machines in the build set.

25.9.3 Ports

The same ideas can be used for the ports tree. The first critical step is to mount /usr/ports from the same machine to all the machines in the build set. Then, configure /etc/make.conf properly to share distfiles. Set DISTDIR to a common shared directory that is writable by whichever user root is mapped to by the NFS mounts. Each machine should set WRKDIRPREFIX to a local build directory. Finally, if the system is to build and distribute packages, set PACKAGES to a directory similar to DISTDIR.