25.2 FreeBSD Update

Written by Tom Rhodes. Based on notes provided by Colin Percival.

Applying security patches is an important part of maintaining computer software, especially the operating system. For the longest time on FreeBSD, this process was not an easy one. Patches had to be applied to the source code, the code rebuilt into binaries, and then the binaries had to be re-installed.

This is no longer the case as FreeBSD now includes a utility called freebsd-update. This utility provides two separate functions. First, it allows for binary security and errata updates to be applied to the FreeBSD base system without the build and install requirements. Second, the utility supports minor and major release upgrades.

Note: Binary updates are available for all architectures and releases currently supported by the security team. Before updating to a new release, its release announcement should be reviewed as it contains important information pertinent to the release. Release announcements are available from http://www.FreeBSD.org/releases/.

If a crontab utilizing the features of freebsd-update(8) exists, it must be disabled before the following operation is started.

25.2.1 The Configuration File

Some users may wish to tweak the default configuration in /etc/freebsd-update.conf, allowing better control of the process. The options are well documented, but the following may require a bit more explanation:

# Components of the base system which should be kept updated.
Components src world kernel

This parameter controls which parts of FreeBSD will be kept up-to-date. The default is to update the source code, the entire base system, and the kernel. Components are the same as those available during installation. For instance, adding world/games would allow game patches to be applied. Using src/bin would allow the source code in src/bin to be updated.

The best option is to leave this at the default as changing it to include specific items requires the user to list every item to be updated. This could have disastrous consequences as source code and binaries may become out of sync.

# Paths which start with anything matching an entry in an IgnorePaths
# statement will be ignored.
IgnorePaths

To leave specified directories, such as /bin or /sbin, untouched during the update process, add their paths to this statement. This option may be used to prevent freebsd-update from overwriting local modifications.

# Paths which start with anything matching an entry in an UpdateIfUnmodified
# statement will only be updated if the contents of the file have not been
# modified by the user (unless changes are merged; see below).
UpdateIfUnmodified /etc/ /var/ /root/ /.cshrc /.profile

This option will only update unmodified configuration files in the specified directories. Any changes made by the user will invalidate the automatic updating of these files. There is another option, KeepModifiedMetadata, which will instruct freebsd-update to save the changes during the merge.

# When upgrading to a new FreeBSD release, files which match MergeChanges
# will have any local changes merged into the version from the new release.
MergeChanges /etc/ /var/named/etc/

List of directories with configuration files that freebsd-update should attempt to merge. The file merge process is a series of diff(1) patches similar to mergemaster(8), but with fewer options. Merges are either accepted, open an editor, or freebsd-update will abort. When in doubt, backup /etc and just accept the merges. See Section 25.7.12.1 for more information about mergemaster.

# Directory in which to store downloaded updates and temporary
# files used by FreeBSD Update.
# WorkDir /var/db/freebsd-update

This directory is where all patches and temporary files are placed. In cases where the user is doing a version upgrade, this location should have a least a gigabyte of disk space available.

# When upgrading between releases, should the list of Components be
# read strictly (StrictComponents yes) or merely as a list of components
# which *might* be installed of which FreeBSD Update should figure out
# which actually are installed and upgrade those (StrictComponents no)?
# StrictComponents no

When this option is set to yes, freebsd-update will assume that the Components list is complete and will not attempt to make changes outside of the list. Effectively, freebsd-update will attempt to update every file which belongs to the Components list.

25.2.2 Security Patches

FreeBSD security patches may be downloaded and installed using the following command:

# freebsd-update fetch
# freebsd-update install

If the update applied any kernel patches, the system will need a reboot in order to boot into the patched kernel. Otherwise, the system should be patched and freebsd-update may be run as a nightly cron(8) job by adding this entry to /etc/crontab:

@daily                                  root    freebsd-update cron

This entry states that freebsd-update will run once every day. When run with cron, freebsd-update will only check if updates exist. If patches exist, they will automatically be downloaded to the local disk but will not be applied. The root user will be sent an email so that they may be reviewed and manually installed.

If anything goes wrong, freebsd-update has the ability to roll back the last set of changes with the following command:

# freebsd-update rollback

Once complete, the system should be restarted if the kernel or any kernel modules were modified. This will allow FreeBSD to load the new binaries into memory.

Only the GENERIC kernel can be automatically updated by freebsd-update. If a custom kernel is installed, it will have to be rebuilt and reinstalled after freebsd-update finishes installing the rest of the updates. However, freebsd-update will detect and update the GENERIC kernel if /boot/GENERIC exists, even if it is not the current running kernel of the system.

Note: It is a good idea to always keep a copy of the GENERIC kernel in /boot/GENERIC. It will be helpful in diagnosing a variety of problems, and in performing version upgrades using freebsd-update as described in Section 25.2.3.

Unless the default configuration in /etc/freebsd-update.conf has been changed, freebsd-update will install the updated kernel sources along with the rest of the updates. Rebuilding and reinstalling a new custom kernel can then be performed in the usual way.

Note: The updates distributed by freebsd-update do not always involve the kernel. It is not necessary to rebuild a custom kernel if the kernel sources have not been modified by the execution of freebsd-update install. However, freebsd-update will always update /usr/src/sys/conf/newvers.sh. The current patch level, as indicated by the -p number reported by uname -r, is obtained from this file. Rebuilding a custom kernel, even if nothing else changed, allows uname(1) to accurately report the current patch level of the system. This is particularly helpful when maintaining multiple systems, as it allows for a quick assessment of the updates installed in each one.

25.2.3 Major and Minor Version Upgrades

Upgrades from one minor version of FreeBSD to another, like from FreeBSD 9.0 to FreeBSD 9.1, are called minor version upgrades. Generally, installed applications will continue to work without problems after minor version upgrades.

Major version upgrades occur when FreeBSD is upgraded from one major version to another, like from FreeBSD 8.X to FreeBSD 9.X. Major version upgrades remove old object files and libraries which will break most third party applications. It is recommended that all installed ports either be removed and re-installed or upgraded after a major version upgrade using a utility such as ports-mgmt/portmaster. A brute-force rebuild of all installed applications can be accomplished with this command:

# portmaster -f

This will ensure everything will be re-installed correctly. Note that setting the BATCH environment variable to yes will answer yes to any prompts during this process, removing the need for manual intervention during the build process.

25.2.3.1 Dealing with Custom Kernels

If a custom kernel is in use, the upgrade process is slightly more involved, and the procedure varies depending on the version of FreeBSD.

25.2.3.1.1 Custom Kernels with FreeBSD 8.X and Earlier

A copy of the GENERIC kernel is needed, and should be placed in /boot/GENERIC. If the GENERIC kernel is not present in the system, it may be obtained using one of the following methods:

  • If a custom kernel has only been built once, the kernel in /boot/kernel.old is actually GENERIC. Rename this directory to /boot/GENERIC.

  • Assuming physical access to the machine is possible, a copy of the GENERIC kernel can be installed from the installation media using the following commands:

    # mount /cdrom
    # cd /cdrom/X.Y-RELEASE/kernels
    # ./install.sh GENERIC
    

    Replace X.Y-RELEASE with the actual version of the release being used. The GENERIC kernel will be installed in /boot/GENERIC by default.

  • Failing all the above, the GENERIC kernel may be rebuilt and installed from source:

    # cd /usr/src
    # env DESTDIR=/boot/GENERIC make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null
    # mv /boot/GENERIC/boot/kernel/* /boot/GENERIC
    # rm -rf /boot/GENERIC/boot
    

    For this kernel to be picked up as GENERIC by freebsd-update, the GENERIC configuration file must not have been modified in any way. It is also suggested that it is built without any other special options.

Rebooting to the GENERIC kernel is not required at this stage.

25.2.3.1.2 Custom Kernels with FreeBSD 9.X and Later

  • If a custom kernel has only been built once, the kernel in /boot/kernel.old is actually the GENERIC kernel. Rename this directory to /boot/kernel.

  • If physical access to the machine is available, a copy of the GENERIC kernel can be installed from the installation media using these commands:

    # mount /cdrom
    # cd /cdrom/usr/freebsd-dist
    # tar -C/ -xvf kernel.txz boot/kernel/kernel
    
  • If the options above cannot be used, the GENERIC kernel may be rebuilt and installed from source:

    # cd /usr/src
    # make kernel __MAKE_CONF=/dev/null SRCCONF=/dev/null
    

    For this kernel to be identified as the GENERIC kernel by freebsd-update, the GENERIC configuration file must not have been modified in any way. It is also suggested that the kernel is built without any other special options.

Rebooting to the GENERIC kernel is not required at this stage.

25.2.3.2 Performing the Upgrade

Major and minor version upgrades may be performed by providing freebsd-update with a release version target. The following command will update to FreeBSD 9.1:

# freebsd-update -r 9.1-RELEASE upgrade

After the command has been received, freebsd-update will evaluate the configuration file and current system in an attempt to gather the information necessary to perform the upgrade. A screen listing will display which components have and have not been detected. For example:

Looking up update.FreeBSD.org mirrors... 1 mirrors found.
Fetching metadata signature for 9.0-RELEASE from update1.FreeBSD.org... done.
Fetching metadata index... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/smp src/base src/bin src/contrib src/crypto src/etc src/games
src/gnu src/include src/krb5 src/lib src/libexec src/release src/rescue
src/sbin src/secure src/share src/sys src/tools src/ubin src/usbin
world/base world/info world/lib32 world/manpages

The following components of FreeBSD do not seem to be installed:
kernel/generic world/catpages world/dict world/doc world/games
world/proflibs

Does this look reasonable (y/n)? y

At this point, freebsd-update will attempt to download all files required for the upgrade. In some cases, the user may be prompted with questions regarding what to install or how to proceed.

When using a custom kernel, the above step will produce a warning similar to the following:

WARNING: This system is running a "MYKERNEL" kernel, which is not a
kernel configuration distributed as part of FreeBSD 9.0-RELEASE.
This kernel will not be updated: you MUST update the kernel manually
before running "/usr/sbin/freebsd-update install"

This warning may be safely ignored at this point. The updated GENERIC kernel will be used as an intermediate step in the upgrade process.

Once all the patches have been downloaded to the local system, they will be applied. This process may take a while, depending on the speed and workload of the machine. Configuration files will then be merged. The merging process requires some user intervention as a file may be merged or an editor may appear on screen for a manual merge. The results of every successful merge will be shown to the user as the process continues. A failed or ignored merge will cause the process to abort. Users may wish to make a backup of /etc and manually merge important files, such as master.passwd or group at a later time.

Note: The system is not being altered yet as all patching and merging is happening in another directory. Once all patches have been applied successfully, all configuration files have been merged and it seems the process will go smoothly, the changes can be committed to disk by the user using the following command:

# freebsd-update install

The kernel and kernel modules will be patched first. At this point, the machine must be rebooted. If the system is running with a custom kernel, use nextboot(8) to set the kernel for the next boot to the updated /boot/GENERIC:

# nextboot -k GENERIC

Warning: Before rebooting with the GENERIC kernel, make sure it contains all the drivers required for the system to boot properly and connect to the network, if the machine being updated is accessed remotely. In particular, if the running custom kernel contains built-in functionality usually provided by kernel modules, make sure to temporarily load these modules into the GENERIC kernel using the /boot/loader.conf facility. It is recommended to disable non-essential services as well as any disk and network mounts until the upgrade process is complete.

The machine should now be restarted with the updated kernel:

# shutdown -r now

Once the system has come back online, restart freebsd-update using the following command. The state of the process has been saved and thus, freebsd-update will not start from the beginning, but will remove all old shared libraries and object files.

# freebsd-update install

Note: Depending upon whether any library version numbers were bumped, there may only be two install phases instead of three.

25.2.3.3 Rebuilding Ports After a Major Version Upgrade

After a major version upgrade, all third party software needs to be rebuilt and re-installed. This is required as installed software may depend on libraries which have been removed during the upgrade process. This process can be automated using ports-mgmt/portmaster:

# portmaster -f

Once this has completed, finish the upgrade process with a final call to freebsd-update in order to tie up all the loose ends in the upgrade process:

# freebsd-update install

If the GENERIC kernel was temporarily used, this is the time to build and install a new custom kernel in the usual way.

Reboot the machine into the new FreeBSD version. The process is complete.

25.2.4 System State Comparison

freebsd-update can be used to test the state of the installed FreeBSD version against a known good copy. This option evaluates the current version of system utilities, libraries, and configuration files. To begin the comparison, issue the following command:

# freebsd-update IDS >> outfile.ids

Warning: While the command name is IDS it is not a replacement for a real intrusion detection system such as security/snort. As freebsd-update stores data on disk, the possibility of tampering is evident. While this possibility may be reduced using kern.securelevel and by storing the freebsd-update data on a read only file system when not in use, a better solution would be to compare the system against a secure disk, such as a DVD or securely stored external USB disk device.

The system will now be inspected, and a lengthy listing of files, along with the sha256(1) hash values for both the known value in the release and the current installation, will be sent to the specified outfile.ids file.

The entries in the listing are extremely long, but the output format may be easily parsed. For instance, to obtain a list of all files which differ from those in the release, issue the following command:

# cat outfile.ids | awk '{ print $1 }' | more
/etc/master.passwd
/etc/motd
/etc/passwd
/etc/pf.conf

This sample output has been truncated as many more files exist. Some files have natural modifications. For example, /etc/passwd has been modified because users have been added to the system. Other files, such as kernel modules, may differ as freebsd-update may have updated them. To exclude specific files or directories, add them to the IDSIgnorePaths option in /etc/freebsd-update.conf.

This system may be used as part of an elaborate upgrade method, aside from the previously discussed version.