Once the local source tree is synchronized against a particular version of FreeBSD such as FreeBSD-STABLE or FreeBSD-CURRENT, the source tree can be used to rebuild the system.
Make a Backup: It cannot be stressed enough how important it is to make a backup of the system before rebuilding the system. While rebuilding the world is an easy task, there will inevitably be times when mistakes in the source tree render the system unbootable.
Create and verify a backup and have a bootable installation media at hand. You will probably never have to use it, but it is better to be safe than sorry!
Subscribe to the Right Mailing List: The FreeBSD-STABLE and FreeBSD-CURRENT branches are, by their nature, in development. People that contribute to FreeBSD are human, and mistakes occasionally happen.
Sometimes these mistakes can be quite harmless, just causing the system to print a new diagnostic warning. Or the change may be catastrophic, and render the system unbootable or destroy file systems.
When problems occur, a “heads up” is posted to the appropriate mailing list, explaining the nature of the problem and which systems it affects. An “all clear” announcement is posted when the problem has been solved.
Users who track FreeBSD-STABLE or FreeBSD-CURRENT and do not read FreeBSD-STABLE mailing list or FreeBSD-CURRENT mailing list respectively, are asking for trouble.
Do Not Use make world: Some older documentation recommends using make world. However, that command skips some important steps and should only be used by experts. For almost all circumstances make world is the wrong thing to do, and the procedure described here should be used instead.
Before updating the system, read /usr/src/UPDATING for any pre-buildworld steps necessary for that version of the sources. Then, use the procedure outlined here.
These upgrade steps assume an upgrade from an older FreeBSD version, consisting of an old compiler, old kernel, old world, and old configuration files. “World” includes the core system binaries, libraries, and programming files. The compiler is part of “world”, but has a few special concerns.
These steps also assume that the sources to a newer version have already been obtained. If the sources are not up-to-date, refer to Section 25.6 for detailed help about synchronizing to a newer version.
Updating the system from source is a more subtle process than it might initially seem to be, and the FreeBSD developers have found it necessary over the years to change the recommended approach fairly dramatically as new kinds of unavoidable dependencies come to light. The rest of this section describes the rationale behind the currently recommended upgrade sequence.
Any successful update sequence must deal with the following issues:
The old compiler might have a bug and not be able to compile the new kernel. So, the new kernel should be built with the new compiler, meaning that the new compiler must be built before the new kernel is built. This does not necessarily mean that the new compiler must be installed before building the new kernel.
The new world might rely on new kernel features. So, the new kernel must be installed before the new world is installed.
These first two issues are the basis for the core buildworld, buildkernel, installkernel, installworld sequence described in the following paragraphs. Other reasons for using these steps are listed below:
The old world might not run correctly on the new kernel, so the new world must be installed immediately upon installing the new kernel.
Some configuration changes must be made before the new world is installed, but others might break the old world. Hence, two different configuration upgrade steps are generally needed.
For the most part, the update process only replaces or adds files and existing old files are not deleted. In a few cases, this can cause problems. As a result, the update procedure will sometimes specify certain files that should be manually deleted at certain steps. This may or may not be automated in the future.
These concerns have led to the following recommended sequence. Note that the detailed sequence for particular updates may require additional steps, but this core process should remain unchanged for some time:
make buildworld
This first compiles the new compiler and a few related tools, then uses the new compiler to compile the rest of the new world. The result ends up in /usr/obj.
make buildkernel
This uses the new compiler residing in /usr/obj in order to protect against compiler-kernel mismatches.
make installkernel
Place the new kernel and kernel modules onto the disk, making it possible to boot with the newly updated kernel.
Reboot into single user mode.
Single user mode minimizes problems from updating software that is already running. It also minimizes any problems from running the old world on a new kernel.
mergemaster -p
This does some initial configuration file updates in preparation for the new world. For instance, it may add new user groups to the system, or new user names to the password database. This is often necessary when new groups or special system-user accounts have been added since the last update, so that the installworld step will be able to use the newly installed system user or system group names without problems.
make installworld
Copies the world from /usr/obj. The new kernel and new world are now installed on disk.
mergemaster
Repeated to update the remaining configuration files, now that the new world is on disk.
Reboot.
A full machine reboot is needed now to load the new kernel and new world with new configuration files.
Upgrades from one release of the same FreeBSD branch to a more recent release of the same branch, such as from 9.0 to 9.1, may not need this procedure since it is less likely to run into serious mismatches between compiler, kernel, userland, and configuration files. The approach of make world followed by building and installing a new kernel might work well enough for minor updates.
When upgrading across major releases, people who do not follow this procedure should expect some problems.
It is also worth noting that many upgrades may require specific additional steps such as renaming or deleting specific files prior to installworld. Read /usr/src/UPDATING carefully, especially at the end, where the currently recommended upgrade sequence is explicitly spelled out.
This procedure has evolved over time as the developers have found it impossible to completely prevent certain kinds of mismatch problems. Hopefully, the current procedure will remain stable for a long time.
To summarize, the currently recommended way of upgrading FreeBSD from sources is:
# cd /usr/src # make buildworld # make buildkernel # make installkernel # shutdown -r now
Note: There are a few rare cases when an extra run of mergemaster -p is needed before the buildworld step. These are described in UPDATING. In general, though, this step can safely be omitted when not updating across one or more major FreeBSD versions.
After installkernel finishes successfully, boot into single user mode using boot -s from the loader prompt. Then run:
# mount -u / # mount -a -t ufs # adjkerntz -i # mergemaster -p # cd /usr/src # make installworld # mergemaster # reboot
Read Further Explanations: The following sections clearly describe each step, especially when using a custom kernel configuration.
Before updating, read /usr/src/UPDATING. This file contains important information about potential problems and may specify the order to run certain commands. If UPDATING contradicts the procedure in this section, UPDATING takes precedence.
Important: Reading UPDATING is not an acceptable substitute for subscribing to the correct mailing list. The two requirements are complementary, not exclusive.
Available make(1) options are shown in make.conf(5) and /usr/share/examples/etc/make.conf. These settings can be added to /etc/make.conf to control the way make(1) runs and how it builds programs. Changes to some settings can have far-reaching and potentially surprising effects. Read the comments in both locations and keep in mind that the defaults have been chosen for a combination of performance and safety.
Options set in /etc/make.conf take effect every time make(1) is used, including compiling applications from the Ports Collection or user-written C programs, or building the FreeBSD operating system.
/etc/src.conf controls the building of the operating system from source code. Unlike /etc/make.conf, the contents of /etc/src.conf only take effect when the FreeBSD operating system itself is being built. Descriptions of the many options available for this file are shown in src.conf(5). Be cautious about disabling seemingly unneeded kernel modules and build options. Sometimes there are unexpected or subtle interactions.
/etc contains a large part of the system's configuration information, as well as scripts that are run at system startup. Some of these scripts change between FreeBSD versions.
Some of the configuration files are used in the day to day running of the system, such as /etc/group.
There have been occasions when the installation part of make installworld expected certain usernames or groups to exist. When performing an upgrade, it is likely that these users or groups do not yet exist. In some cases make buildworld will check to see if these users or groups exist.
The solution is to run mergemaster(8) in
pre-buildworld mode with -p. This compares only
those files that are essential for the success of buildworld or installworld.
Tip: To check which files are owned by the group being renamed or deleted:
# find / -group GID -printThis command will show all files owned by group GID, which can be either a group name or a numeric group ID.
Consider compiling the system in single user mode. Reinstalling the system touches a lot of important system files, all the standard system binaries, libraries, and include files. Changing these on a running system, particularly one with active users, is asking for trouble.
Another method is to compile the system in multi-user mode, and then drop into single user mode for the installation. With this method, hold off on the following steps until the build has completed. Drop to single user mode in order to run installkernel or installworld.
To enter single user mode from a running system:
# shutdown now
Alternatively, reboot the system, and at the boot prompt, select the “single user” option. Once at the single user mode shell prompt, run:
# fsck -p # mount -u / # mount -a -t ufs # swapon -a
This checks the file systems, remounts / read/write, mounts all the other UFS file systems referenced in /etc/fstab, and turns swapping on.
Note: If the CMOS clock is set to local time and not to GMT (this is true if the output of date(1) does not show the correct time and zone), run the following command:
# adjkerntz -iThis ensures that the local time-zone settings get set up correctly.
As parts of the system are rebuilt, they are, by default, placed in subdirectories of /usr/obj. The directories shadow those under /usr/src.
To speed up the make buildworld process, and possibly save some dependency headaches, remove this directory if it already exists.
Some files below /usr/obj may have the immutable flag set which must be removed first using chflags(1).
# cd /usr/obj # chflags -R noschg * # rm -rf *
It is a good idea to save the output from running make(1) to a file. If something goes wrong, a copy of the error message can be posted to one of the FreeBSD mailing lists.
The easiest way to do this is to use script(1) with a parameter that specifies the name of the file to save all output to. Run this command immediately before rebuilding the world, and then type exit when the process has finished:
# script /var/tmp/mw.out
Script started, output file is /var/tmp/mw.out
# make TARGET
… compile, compile, compile …
# exit
Script done, …
Do not save the output in /tmp as this directory may be cleared at next reboot. A better place to save the file is /var/tmp or in root's home directory.
While in /usr/src type:
# cd /usr/src
To rebuild the world, use make(1). This command reads instructions from the Makefile, which describes how the programs that comprise FreeBSD should be built and the order in which they should be built.
The general format of the command is as follows:
# make -x -DVARIABLE target
In this example, -x is an option passed to make(1). Refer to make(1) for an
examples of available options.
-DVARIABLE
passes a variable to the Makefile. The behavior of the
Makefile is controlled by these variables. These are the
same variables as are set in /etc/make.conf, and this
provides another way of setting them. For example:
# make -DNO_PROFILE target
is another way of specifying that profiled libraries should not be built, and corresponds with the
NO_PROFILE= true # Avoid compiling profiled libraries
line in /etc/make.conf.
target tells make(1) what to do. Each Makefile defines a number of different “targets”, and the choice of target determines what happens.
Some targets listed in the Makefile are used by the build process to break out the steps necessary to rebuild the system into a number of sub-steps.
Most of the time, no parameters need to be passed to make(1) and the command looks like this:
# make target
Where target is one of many build options. The first target should always be buildworld.
As the names imply, buildworld builds a complete new tree under /usr/obj and installworld installs this tree on the current machine.
Having separate options is useful for two reasons. First, it allows for a “self hosted” build that does not affect any components of a running system. Because of this, buildworld can be run on a machine running in multi-user mode with no fear of ill-effects. It is still recommended that installworld be run in part in single user mode, though.
Secondly, it allows NFS mounts to be used to upgrade multiple machines on a network. If order to upgrade three machines, A, B and C, run make buildworld and make installworld on A. B and C should then NFS mount /usr/src and /usr/obj from A, and run make installworld to install the results of the build on B and C.
Although the world target still exists, users are strongly encouraged not to use it.
Instead, run:
# make buildworld
It is possible to specify -j which will cause make to spawn several simultaneous processes. This is most
useful on multi-CPU machines. However, since much of the compiling process is
I/O bound rather than CPU bound, it is also useful on single CPU machines.
On a typical single-CPU machine, run:
# make -j4 buildworld
make(1) will then have up to 4 processes running at any one time. Empirical evidence posted to the mailing lists shows this generally gives the best performance benefit.
On a multi-CPU machine using an SMP configured kernel, try values between 6 and 10 and see how they speed things up.
Many factors influence the build time, but fairly recent machines may only take a one or two hours to build the FreeBSD-STABLE tree, with no tricks or shortcuts used during the process. A FreeBSD-CURRENT tree will take somewhat longer.
To take full advantage of the new system, recompile the kernel. This is practically a necessity, as certain memory structures may have changed, and programs like ps(1) and top(1) will fail to work until the kernel and source code versions are the same.
The simplest, safest way to do this is to build and install a kernel based on GENERIC. While GENERIC may not have all the necessary devices for the system, it should contain everything necessary to boot the system back to single user mode. This is a good test that the new system works properly. After booting from GENERIC and verifying that the system works, a new kernel can be built based on a custom kernel configuration file.
On FreeBSD it is important to build world before building a new kernel.
Note: To build a custom kernel with an existing customized configuration file, use KERNCONF=MYKERNEL:
# cd /usr/src # make buildkernel KERNCONF=MYKERNEL # make installkernel KERNCONF=MYKERNEL
If kern.securelevel has been raised above 1 and noschg or
similar flags have been set on the kernel binary, drop into single user mode to
use installkernel. Otherwise, both these commands
can be run from multi user mode without problems. See init(8) for details
about kern.securelevel and chflags(1) for
details about the various file flags.
Reboot into single user mode to test that the new kernel works using the instructions in Section 25.7.6.
Next, use installworld to install the new system binaries:
# cd /usr/src # make installworld
Note: If variables were specified to make buildworld, specify the same variables to make installworld. However,
-jmust never be used with installworld.For example, if you ran:
# make -DNO_PROFILE buildworldinstall the results with:
# make -DNO_PROFILE installworldotherwise, the command will try to install profiled libraries that were not built during the make buildworld phase.
Remaking the world will not update certain directories, such as /etc, /var and /usr, with new or changed configuration files.
The simplest way to update the files in these directories is to use mergemaster(8). Be sure to first make a backup of /etc in case anything goes wrong.
mergemaster(8) is a Bourne script to aid in determining the differences between the configuration files in /etc, and the configuration files in the source tree /usr/src/etc. This is the recommended solution for keeping the system configuration files up to date with those located in the source tree.
To begin, type mergemaster and it will build a
temporary root environment, from / down, and populate it
with various system configuration files. Those files are then compared to the
ones currently installed in the system. Files that differ will be shown in diff(1) format, with
the + sign representing added or modified lines,
and - representing lines that will be either removed
completely, or replaced with a new file. Refer to diff(1) for more
information about the diff(1) syntax and how
file differences are shown.
mergemaster(8) will then display each file that differs, and present the options of either deleting the new file, referred to as the temporary file, installing the temporary file in its unmodified state, merging the temporary file with the currently installed file, or viewing the diff(1) results again.
Choosing to delete the temporary file will tell mergemaster(8) to keep the current file unchanged and to delete the new version. This option is not recommended, unless there is no reason to change the current file. To get help at any time, type ? at the mergemaster(8) prompt. If the user chooses to skip a file, it will be presented again after all other files have been dealt with.
Choosing to install the unmodified temporary file will replace the current file with the new one. For most unmodified files, this is the best option.
Choosing to merge the file will present a text editor, and the contents of both files. The files can be merged by reviewing both files side by side on the screen, and choosing parts from both to create a finished product. When the files are compared side by side, l selects the left contents and r selects contents from the right. The final output will be a file consisting of both parts, which can then be installed. This option is customarily used for files where settings have been modified by the user.
Choosing to view the diff(1) results again will display the file differences just like mergemaster(8) did before prompting an option.
After mergemaster(8) is done with the system files, it will prompt for other options. mergemaster(8) may prompt to rebuild the password file and will finish up with an option to remove left-over temporary files.
To perform the update manually instead, do not just copy over the files from /usr/src/etc to /etc and expect it to work. Some files must be “installed” first as /usr/src/etc is not a copy of what /etc should look like. In addition, some files that should be in /etc are not in /usr/src/etc.
If you are using mergemaster(8) (as recommended), you can skip forward to the next section.
The simplest way to merge files by hand is to install the files into a new directory, and then work through them looking for differences.
Backup Your Existing /etc: It is recommended to first copy the existing /etc somewhere safe, like so:
# cp -Rp /etc /etc.oldwhere
-Rdoes a recursive copy and-ppreserves times and the ownerships on files.
Next, build a dummy set of directories to install the new /etc and other files into /var/tmp/root is a reasonable choice:
# mkdir /var/tmp/root # cd /usr/src/etc # make DESTDIR=/var/tmp/root distrib-dirs distribution
This will build the necessary directory structure and install the files. A lot of the subdirectories that have been created under /var/tmp/root are empty and should be deleted. The simplest way to do this is to:
# cd /var/tmp/root # find -d . -type d | xargs rmdir 2>/dev/null
This will remove all empty directories while redirecting standard error to /dev/null to prevent the warnings about the directories that are not empty.
/var/tmp/root now contains all the files that should be placed in appropriate locations below /. Go through each of these files, determining how they differ from the system's existing files.
Some of the files installed into /var/tmp/root have a leading “.”. Make sure to use ls -a in order to catch them.
The simplest way to compare files is to use diff(1):
# diff /etc/shells /var/tmp/root/etc/shells
This command will show the differences between the existing /etc/shellsand the new /var/tmp/root/etc/shells. Review the differences to decide whether to merge in custom changes or to replace the existing file with the new one.
Name the New Root Directory (/var/tmp/root) with a Time Stamp, so You Can Easily Compare Differences Between Versions: Frequently rebuilding world entails frequently updating /etc as well, which can be a bit of a chore.
To speed up this process, use the following procedure to keep a copy of the last set of changed files that were merged into /etc.
Make the world as normal. When updating /etc and the other directories, give the target directory a name based on the current date:
# mkdir /var/tmp/root-20130214 # cd /usr/src/etc # make DESTDIR=/var/tmp/root-20130214 \ distrib-dirs distributionMerge in the changes from this directory as outlined above. Do not remove the /var/tmp/root-20130214 directory when you have finished.
After downloading the latest version of the source and remaking it, follow step 1. Create a new directory, which reflects the new date. This example uses /var/tmp/root-20130221.
Use diff(1) to see the differences that have been made in the intervening week by creating a recursive diff between the two directories:
# cd /var/tmp # diff -r root-20130214 root-20130221Typically, this will be a much smaller set of differences than those between /var/tmp/root-20130221/etc and /etc. Because the set of differences is smaller, it is easier to migrate those changes across into /etc.
When finished, remove the older of the two /var/tmp/root-* directories:
# rm -rf /var/tmp/root-20130214Repeat this process whenever merging in changes to /etc.
Use date(1) to automate the generation of the directory names:
# mkdir /var/tmp/root-`date "+%Y%m%d"`
Verify that everything appears to be in the right place, then reboot the system using shutdown(8):
# shutdown -r now
You should now have successfully upgraded the FreeBSD system. Congratulations.
If things went slightly wrong, it is easy to rebuild a particular piece of the system. For example, if /etc/magic was accidentally deleted as part of the upgrade or merge of /etc, file(1) will stop working. To fix this, run:
# cd /usr/src/usr.bin/file # make all install
There is no easy answer, as it depends on the nature of the change. For example, if running svn only shows the following files as being updated:
src/games/cribbage/instr.c src/games/sail/pl_main.c src/release/sysinstall/config.c src/release/sysinstall/media.c src/share/mk/bsd.port.mk
it probably is not worth rebuilding the entire world. Instead, go into the appropriate sub-directories and run make all install. But if something major changed, such as src/lib/libc/stdlib, either re-make world, or at least those parts of it that are statically linked.
At the end of the day, it is your call. Some users re-make the world every fortnight and let changes accumulate over that fortnight. Others only re-make those things that have changed and are careful to spot all the dependencies.
It all depends on how often a user wants to upgrade and whether they are tracking FreeBSD-STABLE or FreeBSD-CURRENT.
This normally indicates hardware problems. (Re)making world is an effective way to stress test hardware, and will frequently throw up memory problems which normally manifest themselves as the compiler mysteriously aborts.
A sure indicator of this occurs when make is restarted and it dies at a different point in the process.
To resolve this error, start swapping around the components in the machine to determine which one is failing.
The short answer is yes.
/usr/obj contains all the object files that were produced during the compilation phase. Normally, one of the first steps in the make buildworld process is to remove this directory and start afresh. Keeping /usr/obj around when finished makes little sense, and its removal frees up a approximately 2 GB of disk space.
Advances users can instruct make buildworld to skip this step. This speeds up subsequent builds, since most of the sources will not need to be recompiled. The flip side is that subtle dependency problems can creep in, causing the build to fail in odd ways. This frequently generates noise on the FreeBSD mailing lists, when one person complains that their build has failed, not realizing that it is because they have tried to cut corners.
This depends on how far into the process the problem occurs.
In general, make buildworld builds new copies of essential tools, such as gcc(1) and make(1), and the system libraries. These tools and libraries are then installed, used to rebuild themselves, and are installed again. The entire system, including regular user programs such as ls(1) or grep(1), is then rebuilt with the new system files.
During the last stage, it is fairly safe to:
… fix the problem …
# cd /usr/src
# make -DNO_CLEAN all
This will not undo the work of the previous make buildworld.
If you see the message:
-------------------------------------------------------------- Building everything.. --------------------------------------------------------------
in the make buildworld output, it is probably fairly safe to do so.
If that message is not displayed, or you are not sure, it is always better to be safe than sorry, and restart the build from scratch.
Run it in single user mode.
Put /usr/src and /usr/obj on separate file systems held on separate disks. If possible, put these disks on separate disk controllers.
Alternately, put these file systems across multiple disks using ccd(4).
Turn off profiling by setting “NO_PROFILE=true” in /etc/make.conf.
Pass -jn
to make(1) to run
multiple processes in parallel. This usually helps on both single and
multi processor machines.
The file system holding /usr/src can
be mounted or remounted with noatime.
This prevents the file system from recording the file access time which is
probably not needed.
# mount -u -o noatime /usr/src
Warning: This example assumes /usr/src is on its own file system. If it is part of /usr, then use that file system mount point instead.
The file system holding /usr/obj can be mounted
or remounted with async so that disk writes happen
asynchronously. The write completes immediately, and the data is written
to the disk a few seconds later. This allows writes to be clustered
together, and can provide a dramatic performance boost.
Warning: Keep in mind that this option makes the file system more fragile. With this option, there is an increased chance that, should power fail, the file system will be in an unrecoverable state when the machine restarts.
If /usr/obj is the only directory on this file system, this is not a problem. If you have other, valuable data on the same file system, ensure that there are verified backups before enabling this option.
# mount -u -o async /usr/obj
Warning: If /usr/obj is not on its own file system, replace it in the example with the name of the appropriate mount point.
Make absolutely sure that the environment has no extraneous cruft from earlier builds:
# chflags -R noschg /usr/obj/usr # rm -rf /usr/obj/usr # cd /usr/src # make cleandir # make cleandir
Yes, make cleandir really should be run twice.
Then, restart the whole process, starting with make buildworld.
If problems persist, send the error and the output of uname -a to FreeBSD general questions mailing list. Be prepared to answer other questions about the setup!