Please talk to Mark Linimon before making any changes to this section.
Install FreeBSD.
Create a user to own the portbuild repository, such as portbuild. It should have the '*' password.
Similarly, create a user to own the administration functions and manage the svn repositories, such as srcbuild. It should have the '*' password.
Add the following to /boot/loader.conf:
console="vidconsole,comconsole"
You should run the cluster on UTC. If you have not set the clock to UTC:
# cp -p /usr/share/zoneinfo/Etc/UTC /etc/localtime
Create the appropriate /etc/rc.conf.
Required entries:
hostname="${hostname}"
sshd_enable="YES"
zfs_enable="YES"
Recommended entries:
background_fsck="NO" clear_tmp_enable="YES" dumpdev="AUTO" fsck_y_enable="YES" apache22_enable="YES" apache_flags="" apache_pidfile="/var/run/httpd.pid" gmetad_enable="YES" gmond_enable="YES" inetd_enable="YES" inetd_flags="-l -w" mountd_enable="YES" nfs_server_enable="YES" nfs_server_flags="-u -t -n 12" nfs_remote_port_only="YES" ntpd_enable="YES" rpcbind_enable="YES" rpc_lockd_enable="NO" rpc_statd_enable="YES" sendmail_enable="NONE" smartd_enable="YES"
Create /etc/resolv.conf, if necessary.
Create the appropriate files in /etc/ssh/.
Add the following to /etc/sysctl.conf:
kern.maxfiles=40000 kern.maxfilesperproc=38000 sysctl vfs.usermount=1 sysctl vfs.zfs.super_owner=1
Make sure the following change is made to /etc/ttys:
ttyu0 "/usr/libexec/getty std.9600" vt100 on secure
You should be able to install from the most recent release using only the default kernel configuration.
The following ports (or their latest successors) are required:
databases/py-sqlite3 databases/py-sqlalchemy (only SQLITE is needed) devel/git (WITH_SVN) devel/py-configobj devel/py-setuptools devel/subversion net/nc net/rsync sysutils/ganglia-monitor-core (with GMETAD off) sysutils/ganglia-webfrontend (compile with -DWITHOUT_X11) www/apache22 (with EXT_FILTER)
Expect those to bring in, among others:
databases/sqlite3 lang/perl-5.14 (or successor) lang/python27 (or sucessor)
The following ports (or their latest successors) are strongly suggested:
devel/ccache mail/postfix net/isc-dhcp41-server ports-mgmt/pkg ports-mgmt/portaudit ports-mgmt/portmaster shells/bash shells/zsh sysutils/screen
Note: The use of sudo on the master, which was formerly required, is no longer recommended.
The following ports (or their latest successors) are handy:
benchmarks/bonnie++ ports-mgmt/pkg_tree sysutils/dmidecode sysutils/smartmontools sysutils/zfs-stats
The following steps need to be done as euid root.
Here is a quick example:
Example 10. The contents of example file portbuild/tools/example_install
#!/bin/sh
#
# example script to drive the "mkportbuild" kickstart file
#
export PORTBUILD_USER=portbuild
export SRCBUILD_USER=srcbuild
export ZFS_VOLUME=a
export ZFS_MOUNTPOINT=/a
export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org
#
# create the zpool. the examples here are just suggestions and need to be
# customized for your site.
#
# simple examples:
# zpool create ${ZFS_VOLUME} da1
# zpool create ${ZFS_VOLUME} gprootfs
# more complex example:
# zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
#
# check out the kickstart file and run it
#
mkdir -p tmp
svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/admin/tools tmp
sh -x ./tmp/mkportbuild
Here is a detailed explanation of the example:
Export the value of PORTBUILD_USER:
# export PORTBUILD_USER=portbuild
Export the value of SRCBUILD_USER:
# export SRCBUILD_USER=srcbuild
Pick a zfs volume name and export it. We have used a so far to date.
# export ZFS_VOLUME=a
Pick a mountpoint and export it. We have used /a so far to date.
# export ZFS_MOUNTPOINT=/a
Create the zfs volume and mount it.
Example 11. Creating a zfs volume for portbuild
# zpool create ${ZFS_VOLUME} mirror da1 da2 mirror da3 da4 mirror da5 da6 mirror da7 da8
Note: The kickstart script defines zfs permission sets, so that the srcbuild user and portbuild user may administer subdirectories of this volume without having to have root privileges.
Select an svn repository and export it. See the FreeBSD Handbook for the currently supported list.
# export VCS_REPOSITORY=svn://svn0.us-east.FreeBSD.org
Obtain a copy of the kickstart script into a temporary directory. (You will not need to keep this directory later.)
# mkdir -p /home/portbuild/tmp
# svn checkout ${VCS_REPOSITORY}/base/projects/portbuild/admin/tools /home/portbuild/tmp
Run the kickstart script:
# sh /home/portbuild/tmp/mkportbuild
This will accomplish all the following steps:
Create the portbuild directory
Create and mount a new zfs filesystem on it
Set up the directory
Set up the initial repository:
Set up the zfs permission sets.
Split ownerships of subdirectories such that PORTBUILD_USER owns, and only owns, files that are used to manage builds and interact with slaves. The more trustable user SRCBUILD_USER now owns everything else.
Configure the server by making the following changes to /a/portbuild/admin/conf/admin.conf:
Set SUPPORTED_ARCHS to the list of architectures you wish to build packages for.
For each source branch you will be building for, set SRC_BRANCHES and SRC_BRANCH_branch_SUBDIR as detailed in Section 14.1. You should not need to change SRC_BRANCHES_PATTERN.
Set ZFS_VOLUME and ZFS_MOUNTPOINT to whatever you chose above.
Set VCS_REPOSITORY to whatever you chose above.
Set MASTER_URL to the http URL of your server. This will be stamped into the package build logs and the indices thereof.
Most of the other default values should be fine.
Configure how build slaves will talk to your server by making the following changes to /a/portbuild/conf/client.conf:
Set CLIENT_NFS_MASTER to wherever your build slaves will PXE boot from. (Possibly, the hostname of your server.)
Set CLIENT_BACKUP_FTP_SITE to a backup site for FTP fetches; again, possibly the hostname of your server.
Set CLIENT_UPLOAD_HOST to where completed packages will be uploaded.
Most of the other default values should be fine.
Most of the default values in /a/portbuild/conf/common.conf should be fine. This file holds definitions used by both the server and all its clients.
Configure the server by making the following changes to /a/portbuild/conf/server.conf:
Set UPLOAD_DIRECTORY, UPLOAD_TARGET, and UPLOAD_USER as appropriate for your site.
Most of the other default values should be fine.
For each architecture, follow the steps in Section 17.1.
As root, copy the following files from /a/portbuild/admin/etc/rc.d/ to /usr/local/etc/rc.d/:
pollmachine qmanager
As root, start each one of them. You may find it handy to start each under screen for debugging purposes.
Initialize the qmanager database's acl list:
Note: This should now be automatically done for you by the first build command.
# python /a/portbuild/qmanager/qclient add_acl name=deny_all uidlist= gidlist= sense=0
As the srcbuild user, run the following commands manually to create the src and ports repositories, respectively:
% /a/portbuild/admin/scripts/updatesnap.ports % /a/portbuild/admin/scripts/updatesnap
These will be periodically run from the srcbuild crontab, which you will install below.
Configure /usr/local/etc/apache22/httpd.conf as appropriate for your site.
Copy /a/portbuild/admin/conf/apache.conf to the appropriate Includes/ subdirectory, e.g., /usr/local/etc/apache22/Includes/portbuild.conf. Configure it as appropriate for your site.
Install /a/portbuild/admin/crontabs/portbuild as the portbuild crontab via crontab -u portbuild -e. If you do not support all the archs listed there, make sure to comment out the appropriate dologs entries.
Install /a/portbuild/admin/crontabs/srcbuild as the srcbuild crontab via crontab -u srcbuild -e.
If your build slaves will be pxebooted, make sure to enable the tftp entries in /etc/inetd.conf.
Configure mail by doing the following:
newaliases.
For each architecture, follow the steps in Section 17.2.
You will probably find it handy to append the following to the PATH definition for the portbuild user:
/a/portbuild/scripts:/a/portbuild/tools
You will also probably find it handy to append the following to the PATH definition for the srcbuild user:
/a/portbuild/admin/scripts:/a/portbuild/admin/tools
You should now be ready to build packages.