3 Subversion Primer

It is assumed that you are already familiar with the basic operation of the version control systems in use. Traditionally this was CVS. Subversion is used for the src tree as of May 2008, the doc/www tree as of May 2012 and the ports tree as of July 2012.

There is a list of things missing in Subversion when compared to CVS . The notes at http://people.freebsd.org/~peter/svn_notes.txt might also be useful.

3.1 Introduction

The FreeBSD source repository switched from CVS to Subversion on May 31st, 2008. The first real SVN commit is r179447.

The FreeBSD doc/www repository switched from CVS to Subversion on May 19th, 2012. The first real SVN commit is r38821.

Note: Part of the doc/www CVS to SVN conversion included an infrastructural change to the build process. The most notable change is the location of the FreeBSD website www tree, which has been moved from www/lang/ to head/lang/htdocs/.

The FreeBSD ports repository switched from CVS to Subversion on July 14th, 2012. The first real SVN commit is r300894.

There are mechanisms in place to automatically merge changes back from the Subversion repository to the CVS one, so regular users should not notice a difference, however developers most certainly will.

Subversion is not that different from CVS when it comes to daily use, but there are differences. Subversion has a number of features that should make developers' lives easier. The most important advantage to Subversion (and the reason why FreeBSD switched) is that it handles branches and merging much better than CVS does. Some of the principal differences are:

Subversion can be installed from the FreeBSD Ports Collection, by issuing the following commands:

# cd /usr/ports/devel/subversion
# make clean install

3.2 Getting Started

There are a few ways to obtain a working copy of the tree from Subversion. This section will explain them.

3.2.1 Direct Checkout

The first is to check out directly from the main repository. For the src tree, use:

% svn checkout svn+ssh://svn.freebsd.org/base/head /usr/src

For the doc tree, use:

% svn checkout svn+ssh://svn.freebsd.org/doc/head /usr/doc

For the ports tree, use:

% svn checkout svn+ssh://svn.freebsd.org/ports/head /usr/ports

Note: Though the remaining examples in this document are written with the workflow of working with the src tree in mind, the underlying concepts are the same for working with the doc and the ports tree. Ports related Subversion operations are listed in Section 13.

The above command will check out a CURRENT source tree as /usr/src/, which can be any target directory on the local filesystem. Omitting the final argument of that command causes the working copy, in this case, to be named “head”, but that can be renamed safely.

svn+ssh means the SVN protocol tunnelled over SSH. The name of the server is svn.freebsd.org, base is the path to the repository, and head is the subdirectory within the repository.

If your FreeBSD login name is different from your login name on your local machine, you must either include it in the URL (for example svn+ssh://jarjar@svn.freebsd.org/base/head), or add an entry to your ~/.ssh/config in the form:

Host svn.freebsd.org
	User jarjar

This is the simplest method, but it's hard to tell just yet how much load it will place on the repository. Subversion is much faster than CVS, however.

Note: The svn diff does not require access to the server as SVN stores a reference copy of every file in the working copy. This, however, means that Subversion working copies are very large in size.

3.2.2 Checkout from a Mirror

You can check out a working copy from a mirror by simply substituting the mirror's URL for svn+ssh://svn.freebsd.org/base. This can be an official mirror or a mirror you maintain yourself using svnsync or similar.

There is a serious disadvantage to this method: every time something is to be committed, a svn switch --relocate to the master repository has to be done, remembering to svn switch back to the mirror after the commit. Also, since svn switch only works between repositories that have the same UUID, some hacking of the local repository's UUID has to occur before it is possible to start using it.

Unlike with CVS and csup, the hassle of a local svnsync mirror probably is not worth it unless the network connectivity situation or other factors demand it. If it is needed, see the end of this chapter for information on how to set one up.

3.2.3 RELENG_* Branches and General Layout

In svn+ssh://svn.freebsd.org/base, base refers to the source tree. Similarly, ports refers to the ports tree, and so on. These are separate repositories with their own change number sequences, access controls and commit mail.

For the base repository, HEAD refers to the -CURRENT tree. For example, head/bin/ls is what would go into /usr/src/bin/ls in a release. Some other key locations are:

  • /stable/n which corresponds to RELENG_n.

  • /releng/n.n which corresponds to RELENG_n_n.

  • /release/n.n.n which corresponds to RELENG_n_n_n_RELEASE.

  • /vendor* is the vendor branch import work area. This directory itself does not contain branches, however its subdirectories do. This contrasts with the stable, releng and release directories.

  • /projects and /user feature a branch work area, like in Perforce. As above, the /user directory does not contain branches itself.

3.2.4 FreeBSD Documentation Project Branches and Layout

In svn+ssh://svn.freebsd.org/doc, doc refers repository root of the source tree.

In general, most FreeBSD Documentation Project work will be done within the head/ branch of the source tree.

FreeBSD documentation is written and/or translated to various languages, each of which within a separate directory within the head/ branch.

Each translation set contains several subdirectories for the various parts of the FreeBSD Documentation Project. A few noteworthy directories are:

  • /articles/ contains the source code for articles written by various FreeBSD contributors.

  • /books/ contains the source code for the different books, such as the FreeBSD Handbook.

  • /htdocs/ contains the source code for the FreeBSD website.

3.2.5 FreeBSD Ports Tree Branches and Layout

In svn+ssh://svn.freebsd.org/ports, ports refers repository root of the ports tree.

In general, most FreeBSD port work will be done within the head/ branch of the ports tree which is the actual ports tree used to install software. Some other key locations are:

  • /branches/RELENG_n_n_n which corresponds to RELENG_n_n_n is used to merge back security updates in preparation for a release.

  • /tags/RELEASE_n_n_n which corresponds to RELEASE_n_n_n represents a release tag of the ports tree.

  • /tags/RELEASE_n_EOL represents the end of life tag of a specific FreeBSD branch.

3.3 Daily Use

This section will explain how to perform common day-to-day operations with Subversion. There should be no difference between SVN and SVK in daily use, except for the revision renumbering mentioned earlier.

3.3.1 Help

Both SVN and SVK have built in help documentation. It can be accessed by typing the following command:

% svn help

3.3.2 Checkout

As seen earlier, to check out the FreeBSD head branch:

% svn checkout svn+ssh://svn.freebsd.org/base/head /usr/src

At some point, more than just HEAD will probably be useful, for instance when merging changes to stable/7. Therefore, it may be useful to have a partial checkout of the complete tree (a full checkout would be very painful).

To do this, first check out the root of the repository:

% svn checkout --depth=immediates svn+ssh://svn.freebsd.org/base

This will give base with all the files it contains (at the time of writing, just ROADMAP.txt) and empty subdirectories for head, stable, vendor and so on.

Expanding the working copy is possible. Just change the depth of the various subdirectories:

% svn up --set-depth=infinity base/head
% svn up --set-depth=immediates base/release base/releng base/stable

The above command will pull down a full copy of head, plus empty copies of every release tag, every releng branch, and every stable branch.

If at a later date merging to 7-STABLE is required, expand the working copy:

% svn up --set-depth=infinity base/stable/7

Subtrees do not have to be expanded completely. For instance, expanding only stable/7/sys and then later expand the rest of stable/7:

% svn up --set-depth=infinity base/stable/7/sys
% svn up --set-depth=infinity base/stable/7

Updating the tree with svn update will only update what was previously asked for (in this case, head and stable/7; it will not pull down the whole tree.

Note: Decreasing the depth of a working copy is not possible.

3.3.3 Anonymous Checkout

It is possible to anonymously check out the FreeBSD repository with Subversion. This will give access to a read-only tree that can be updated, but not committed to. To do this, use one of the following commands:

% svn co svn://svn.freebsd.org/base/head /usr/src
% svn co http://svn.freebsd.org/base/head /usr/src

3.3.4 Updating the Tree

To update a working copy to either the latest revision, or a specific revision:

% svn update
% svn update -r12345

3.3.5 Status

To view the local changes that have been made to the working copy:

% svn status

To show local changes and files that are out-of-date do:

% svn status --show-updates

3.3.6 Editing and Committing

Unlike Perforce, SVN and SVK do not need to be told in advance about file editing.

svn commit works like the equivalent CVS command. To commit all changes in the current directory and all subdirectories:

% svn commit

To commit all changes in, for example, lib/libfetch/ and usr/bin/fetch/ in a single operation:

% svn commit lib/libfetch usr/bin/fetch

There is also a commit wrapper for the ports tree to handle the properties and sanity checking your changes:

% /usr/ports/Tools/scripts/psvn commit
	 

3.3.7 Adding and Removing Files

Note: Before adding files, get a copy of auto-props.txt (there is also a ports tree specific version) and add it to ~/.subversion/config according to the instructions in the file. If you added something before you've read this, you may use svn rm --keep-local for just added files, fix your config file and re-add them again. The initial config file is created when you first run a svn command, even something as simple as svn help.

Files are added to a SVN repository with svn add. To add a file named foo, edit it, then:

% svn add foo

Files can be removed with svn remove:

% svn remove foo

Subversion does not require rming the file before svn rming it, and indeed complains if that happens.

It is possible to add directories with svn add:

% mkdir bar
% svn add bar

Although svn mkdir makes this easier by combining the creation of the directory and the adding of it:

% svn mkdir bar

The directory is not immediately created in the repository when you use svn mkdir. Subversion allows directories to be removed using svn rm, however there is no svn rmdir:

% svn rm bar

3.3.8 Copying and Moving Files

The following (obviously) creates a copy of foo.c, named bar.c:

% svn copy foo.c bar.c

To move and rename a file:

% svn move foo.c bar.c

The above command is the exact equivalent of:

% svn copy foo.c bar.c
% svn remove foo.c

3.3.9 Log and Annotate

svn log will show all the revisions that affect a directory and files within that directory in reverse chronological order, if run on a directory. This contrasts with cvs log in that CVS shows the complete log for each file in the directory, including duplicate entries for revisions that affect multiple files.

svn annotate, or equally svn praise or svn blame, is equivalent to cvs annotate in everything but output format.

3.3.10 Diffs

svn diff displays changes to the working copy of the repository. Diffs generated by SVN are unified and include new files by default in the diff output.

svn diff can show the changes between two revisions of the same file:

% svn diff -r179453:179454 ROADMAP.txt

It can also show all changes for a specific changeset. The following will show what changes were made to the current directory and all subdirectories in changeset 179454:

% svn diff -c179454 .

3.3.11 Reverting

Local changes (including additions and deletions) can be reverted using svn revert. It does not update out-of-date files—it just replaces them with pristine copies of the original version.

3.3.12 Conflicts

If a svn update resulted in a merge conflict, Subversion will remember which files have conflicts and refuse to commit any changes to those files until explicitly told that the conflicts have been resolved. The simple, not yet deprecated procedure is the following:

% svn resolved foo

However, the preferred procedure is:

% svn resolve --accept=working foo

The two examples are equivalent. Possible values for --accept are:

  • working: use the version in your working directory (which one presumes has been edited to resolve the conflicts).

  • base: use a pristine copy of the version you had before svn update, discarding your own changes, the conflicting changes, and possibly other intervening changes as well.

  • mine-full: use what you had before svn update, including your own changes, but discarding the conflicting changes, and possibly other intervening changes as well.

  • theirs-full: use the version that was retrieved when you did svn update, discarding your own changes.

3.4 Advanced Use

3.4.1 Sparse Checkouts

The equivalent to cvs checkout -l, which checks out a directory without its subdirectories, is svn checkout -N. Unlike CVS, SVN remembers the -N so that a svn update does not end up pulling down the subdirectories. In Subversion 1.5 and newer, -N has been deprecated in favour of the --depth option which allows for precise control. Therefore:

% svn checkout -N svn+ssh://svn.freebsd.org/base ~/freebsd

is equivalent to:

% svn checkout --depth=empty svn+ssh://svn.freebsd.org/base ~/freebsd

Valid arguments to --depth are:

  • empty: the directory itself without any of its contents.

  • files: the directory and any files it contains.

  • immediates: the directory and any files and directories it contains, but none of the subdirectories' contents.

  • infinity: anything.

The --depth option applies to many other commands, including svn commit, svn revert, and svn diff.

Since --depth is sticky, there is a --set-depth option for svn update that will change the selected depth. Thus, given the working copy produced by the previous example:

% cd ~/freebsd
% svn update --set-depth=immediates .

The above command will populate the working copy in ~/freebsd with ROADMAP.txt and empty subdirectories, and nothing will happen when svn update is executed on the subdirectories. However, the following command will set the depth for head (in this case) to infinity, and fully populate it:

% svn update --set-depth=infinity head

3.4.2 Direct Operation

Certain operations can be performed directly on the repository, without touching the working copy. Specifically, this applies to any operation that does not require editing a file, including:

  • log, diff.

  • mkdir.

  • remove, copy, rename.

  • propset, propedit, propdel.

  • merge.

Branching is very fast. The following command would be used to branch RELENG_8:

% svn copy svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/stable/8

This is equivalent to the following set of commands which take minutes and hours as opposed to seconds, depending on your network connection:

% svn checkout --depth=immediates svn+ssh://svn.freebsd.org/base
% cd base
% svn update --depth=infinity head
% svn copy head stable/8
% svn commit stable/8

3.4.3 Merging with SVN

This section deals with merging code from one branch to another (typically, from head to a stable branch).

Note: In all examples below, $FSVN refers to the location of the FreeBSD Subversion repository, svn+ssh://svn.freebsd.org/base/.

3.4.3.1 About Merge Tracking

From the user's perspective, merge tracking information (or mergeinfo) is stored in a property called svn:mergeinfo, which is a comma-separated list of revisions and ranges of revisions that have been merged. When set on a file, it applies only to that file. When set on a directory, it applies to that directory and its descendants (files and directories) except for those that have their own svn:mergeinfo.

It is not inherited. For instance, stable/6/contrib/openpam/ does not implicitly inherit mergeinfo from stable/6/, or stable/6/contrib/. Doing so would make partial checkouts very hard to manage. Instead, mergeinfo is explicitly propagated down the tree. For merging something into branch/foo/bar/, the following rules apply:

  1. If branch/foo/bar/ doesn't already have a mergeinfo record, but a direct ancestor (for instance, branch/foo/) does, then that record will be propagated down to branch/foo/bar/ before information about the current merge is recorded.

  2. Information about the current merge will not be propagated back up that ancestor.

  3. If a direct descendant of branch/foo/bar/ (for instance, branch/foo/bar/baz/) already has a mergeinfo record, information about the current merge will be propagated down to it.

If you consider the case where a revision changes several separate parts of the tree (for example, branch/foo/bar/ and branch/foo/quux/), but you only want to merge some of it (for example, branch/foo/bar/), you will see that these rules make sense. If mergeinfo was propagated up, it would seem like that revision had also been merged to branch/foo/quux/, when in fact it had not been.

3.4.3.2 Selecting the Source and Target

Because of mergeinfo propagation, it is important to choose the source and target for the merge carefully to minimise property changes on unrelated directories.

The rules for selecting the merge target (the directory that you will merge the changes to) can be summarized as follows:

  1. Never merge directly to a file.

  2. Never, ever merge directly to a file.

  3. Never, ever, ever merge directly to a file.

  4. Changes to kernel code should be merged to sys/. For instance, a change to the ichwd(4) driver should be merged to sys/, not sys/dev/ichwd/. Likewise, a change to the TCP/IP stack should be merged to sys/, not sys/netinet/.

  5. Changes to code under etc/ should be merged at etc/, not below it.

  6. Changes to vendor code (code in contrib/, crypto/ and so on) should be merged to the directory where vendor imports happen. For instance, a change to crypto/openssl/util/ should be merged to crypto/openssl/. This is rarely an issue, however, since changes to vendor code are usually merged wholesale.

  7. Changes to userland programs should as a general rule be merged to the directory that contains the Makefile for that program. For instance, a change to usr.bin/xlint/arch/i386/ should be merged to usr.bin/xlint/.

  8. Changes to userland libraries should as a general rule be merged to the directory that contains the Makefile for that library. For instance, a change to lib/libc/gen/ should be merged to lib/libc/.

  9. There may be cases where it makes sense to deviate from the rules for userland programs and libraries. For instance, everything under lib/libpam/ is merged to lib/libpam/, even though the library itself and all of the modules each have their own Makefile.

  10. Changes to manual pages should be merged to share/man/manN/, for the appropriate value of N.

  11. Other changes to share/ should be merged to the appropriate subdirectory and not to share/ directly.

  12. Changes to a top-level file in the source tree such as UPDATING or Makefile.inc1 should be merged directly to that file rather than to the root of the whole tree. Yes, this is an exception to the first three rules.

  13. When in doubt, ask.

If you need to merge changes to several places at once (for instance, changing a kernel interface and every userland program that uses it), merge each target separately, then commit them together. For instance, if you merge a revision that changed a kernel API and updated all the userland bits that used that API, you would merge the kernel change to sys, and the userland bits to the appropriate userland directories, then commit all of these in one go.

The source will almost invariably be the same as the target. For instance, you will always merge stable/7/lib/libc/ from head/lib/libc/. The only exception would be when merging changes to code that has moved in the source branch but not in the parent branch. For instance, a change to pkill(1) would be merged from bin/pkill/ in head to usr.bin/pkill/ in stable/7.

3.4.3.3 Preparing the Merge Target

Because of the mergeinfo propagation issues described earlier, it is very important that you never merge changes into a sparse working copy. You must always have a full checkout of the branch you will merge into. For instance, when merging from HEAD to 7, you must have a full checkout of stable/7:

% cd stable/7
% svn up --set-depth=infinity

The target directory must also be up-to-date and must not contain any uncommitted changes or stray files.

3.4.3.4 Identifying Revisions

Identifying revisions to be merged is a must. If the target already has complete mergeinfo, ask SVN for a list:

% cd stable/6/contrib/openpam
% svn mergeinfo --show-revs=eligible $FSVN/head/contrib/openpam

If the target does not have complete mergeinfo, check the log for the merge source.

3.4.3.5 Merging

Now, let's start merging!

3.4.3.5.1 The Principles

Say you would like to merge:

  • revision $R.

  • in directory $target in stable branch $B.

  • from directory $source in head.

  • $FSVN is svn+ssh://svn.freebsd.org/base.

Assuming that revisions $P and $Q have already been merged, and that the current directory is an up-to-date working copy of stable/$B, the existing mergeinfo looks like this:

% svn propget svn:mergeinfo -R $target
$target - /head/$source:$P,$Q

Merging is done like so:

% svn merge -c$R $FSVN/head/$source $target

Checking the results of this is possible with svn diff.

The svn:mergeinfo now looks like:

% svn propget svn:mergeinfo -R $target
$target - head/$source:$P,$Q,$R

If the results are not exactly as shown, assistance may be required before committing as mistakes may have been made, or there may be something wrong with the existing mergeinfo, or there may be a bug in Subversion.

3.4.3.5.2 Practical Example

As a practical example, consider the following scenario: The changes to netmap.4 in r238987 is to be merged from CURRENT to 9-STABLE. The file resides in head/share/man/man4 and according to Section 3.4.3 this is also where to do the merge. Note that in this example all paths are relative to the top of the svn repository. for more information on the directory layout, see Section 3.2.3.

The first step is to inspect the existing mergeinfo.

% svn propget svn:mergeinfo -R stable/9/share/man/man4

Take a quick note of how it looks before moving on to the next step; doing the actual merge:

% svn merge -c r238987 svn+ssh://svn.freebsd.org/base/head/share/man/man4 stable/9/share/man/man4
--- Merging r238987 into 'stable/9/share/man/man4':
U    stable/9/share/man/man4/netmap.4
--- Recording mergeinfo for merge of r238987 into
'stable/9/share/man/man4':
 U   stable/9/share/man/man4

Check that the revision number of the merged revision has been added. Once this is verified, the only thing left is the actual commit.

% svn commit stable/9/share/man/man4
3.4.3.5.3 Merging into the Kernel (sys/)

As stated above, merging into the kernel is different from merging in the rest of the tree. In many ways merging to the kernel is simpler because there is always the same merge target (sys/).

Once svn merge has been executed, svn diff has to be run on the directory to check the changes. This may show some unrelated property changes, but these can be ignored. Next, build and test the kernel, and, once the tests are complete, commit the code as normal, making sure that the commit message starts with “Merge r226222 from head”, or similar.

3.4.3.6 Precautions Before Committing

As always, build world (or appropriate parts of it).

Check the changes with svn diff and svn stat. Make sure all the files that should have been added or deleted were in fact added or deleted.

Take a closer look at any property change (marked by a M in the second column of svn stat). Normally, no svn:mergeinfo properties should be anywhere except the target directory (or directories).

If something looks fishy, ask for help.

3.4.3.7 Committing

Make sure to commit a top level directory to have the mergeinfo included as well. Do not specify individual files on the command line. For more information about committing files in general, see the relevant section of this primer.

3.4.4 Vendor imports with SVN

Important: Please read this entire section before starting a vendor import.

Note: Patches to vendor code fall into two categories:

  • Vendor patches: these are patches that have been issued by the vendor, or that have been extracted from the vendor's version control system, which address issues which in your opinion can't wait until the next vendor release.

  • FreeBSD patches: these are patches that modify the vendor code to address FreeBSD-specific issues.

The nature of a patch dictates where it should be committed:

  • Vendor patches should be committed to the vendor branch, and merged from there to head. If the patch addresses an issue in a new release that is currently being imported, it must not be committed along with the new release: the release must be imported and tagged first, then the patch can be applied and committed. There is no need to re-tag the vendor sources after committing the patch.

  • FreeBSD patches should be committed directly to head.

3.4.4.1 Preparing the tree

If importing for the first time after the switch to Subversion, flattening and cleaning up the vendor tree is necessary, as well as bootstrapping the merge history in the main tree.

3.4.4.1.1 Flattening

During the conversion from CVS to Subversion, vendor branches were imported with the same layout as the main tree. This means that the pf vendor sources ended up in vendor/pf/dist/contrib/pf. The vendor source is best directly in vendor/pf/dist.

To flatten the pf tree:

% cd vendor/pf/dist/contrib/pf
% svn mv $(svn list) ../..
% cd ../..
% svn rm contrib
% svn propdel -R svn:mergeinfo .
% svn commit

The propdel bit is necessary because starting with 1.5, Subversion will automatically add svn:mergeinfo to any directory that is copied or moved. In this case, as nothing is being merged from the deleted tree, they just get in the way.

Tags may be flattened as well (3, 4, 3.5 etc.); the procedure is exactly the same, only changing dist to 3.5 or similar, and putting the svn commit off until the end of the process.

3.4.4.1.2 Cleaning up

The dist tree can be cleaned up as necessary. Disabling keyword expansion is recommended, as it makes no sense on unmodified vendor code and in some cases it can even be harmful. OpenSSH, for example, includes two files that originated with FreeBSD and still contain the original version tags. To do this:

% svn propdel svn:keywords -R .
# svn commit
3.4.4.1.3 Bootstrapping merge history

If importing for the first time after the switch to Subversion, bootstrap svn:mergeinfo on the target directory in the main tree to the revision that corresponds to the last related change to the vendor tree, prior to importing new sources:

% cd head/contrib/pf
% svn merge --record-only svn+ssh://svn.freebsd.org/base/vendor/pf/dist@180876 .
% svn commit

3.4.4.2 Importing new sources

With two commits—one for the import itself and one for the tag—this step can optionally be repeated for every upstream release between the last import and the current import.

3.4.4.2.1 Preparing the vendor sources

Unlike in CVS where only the needed parts were imported into the vendor tree to avoid bloating the main tree, Subversion is able to store a full distribution in the vendor tree. So, import everything, but merge only what is required.

A svn add is required to add any files that were added since the last vendor import, and svn rm is required to remove any that were removed since. Preparing sorted lists of the contents of the vendor tree and of the sources that are about to be imported is recommended, to facilitate the process.

% cd vendor/pf/dist
% svn list -R | grep -v '/$' | sort >../old
% cd ../pf-4.3
% find . -type f | cut -c 3- | sort >../new

With these two files, comm -23 ../old ../new will list removed files (files only in old), while comm -13 ../old ../new will list added files only in new.

3.4.4.2.2 Importing into the vendor tree

Now, the sources must be copied into dist and the svn add and svn rm commands should be used as needed:

% cd vendor/pf/pf-4.3
% tar cf - . | tar xf - -C ../dist
% cd ../dist
% comm -23 ../old ../new | xargs svn rm
% comm -13 ../old ../new | xargs svn --parents add

If any directories were removed, they will have to be svn rmed manually. Nothing will break if they are not, but they will remain in the tree.

Check properties on any new files. All text files should have svn:eol-style set to native. All binary files should have svn:mime-type set to application/octet-stream unless there is a more appropriate media type. Executable files should have svn:executable set to *. No other properties should exist on any file in the tree.

Committing is now possible, however it is good practice to make sure that everything is OK by using the svn stat and svn diff commands.

3.4.4.2.3 Tagging

Once committed, vendor releases should be tagged for future reference. The best and quickest way to do this is directly in the repository:

% svn cp svn+ssh://svn.freebsd.org/base/vendor/pf/dist svn+ssh://svn.freebsd.org/base/vendor/pf/4.3

Once that is complete, svn up the working copy of vendor/pf to get the new tag, although this is rarely needed.

If creating the tag in the working copy of the tree, svn:mergeinfo results must be removed:

% cd	vendor/pf
% svn cp dist 4.3
% svn propdel svn:mergeinfo -R 4.3

3.4.4.3 Merging to head

% cd head/contrib/pf
% svn up
% svn merge --accept=postpone svn+ssh://svn.freebsd.org/base/vendor/pf/dist .

The --accept=postpone tells Subversion that it shouldn't complain because merge conflicts will be taken care of manually.

It is necessary to resolve any merge conflicts. This process is the same in SVN as in CVS.

Make sure that any files that were added or removed in the vendor tree have been properly added or removed in the main tree. To check diffs against the vendor branch:

% svn diff --no-diff-deleted --old=svn+ssh://svn.freebsd.org/base/vendor/pf/dist --new=.

The --no-diff-deleted tells Subversion not to complain about files that are in the vendor tree but not in the main tree, i.e., things that would have previously been removed before the vendor import, like for example the like the vendor's makefiles and configure scripts.

Using CVS, once a file was off the vendor branch, it was not able to be put back. With Subversion, there is no concept of on or off the vendor branch. If a file that previously had local modifications, to make it not show up in diffs in the vendor tree, all that has to be done is remove any left-over cruft like FreeBSD version tags, which is much easier.

If any changes are required for the world to build with the new sources, make them now, and keep testing until everything builds and runs perfectly.

3.4.4.4 Committing the vendor import

Committing is now possible! Everything must be committed in one go. If done properly, the tree will move from a consistent state with old code, to a consistent state with new code.

3.4.4.5 From scratch

3.4.4.5.1 Importing into the vendor tree

This section is an example of importing and tagging byacc into head.

First, prepare the directory in vendor:

% svn co --depth immediates $FSVN/vendor
% cd vendor
% svn mkdir byacc
% svn mkdir byacc/dist

Now, import the sources into the dist directory. Once the files are in place, svn add the new ones, then svn commit and tag the imported version. To save time and bandwidth, direct remote committing and tagging is possible:

% svn cp -m "Tag byacc 20120115" $FSVN/vendor/byacc/dist $FSVN/vendor/byacc/20120115
3.4.4.5.2 Merging to head

Due to this being a new file, copy it for the merge:

% svn cp -m "Import byacc to contrib" $FSVN/vendor/byacc/dist $FSVN/head/contrib/byacc

Working normally on newly imported sources is still possible.

3.4.5 Reverting a Commit

Reverting a commit to a previous version is fairly easy:

% svn merge -r179454:179453 ROADMAP.txt
% svn commit

Change number syntax, with negative meaning a reverse change, can also be used:

% svn merge -c -179454 ROADMAP.txt
% svn commit

This can also be done directly in the repository:

% svn merge -r179454:179453 svn+ssh://svn.freebsd.org/base/ROADMAP.txt

Note: It is important to ensure that the mergeinfo is correct when reverting a file in order to permit svn mergeinfo --eligible to work as expected.

Reverting the deletion of a file is slightly different. Copying the version of the file that predates the deletion is required. For example, to restore a file that was deleted in revision N, restore version N-1:

% svn copy svn+ssh://svn.freebsd.org/base/ROADMAP.txt@179454
% svn commit

or, equally:

% svn copy svn+ssh://svn.freebsd.org/base/ROADMAP.txt@179454 svn+ssh://svn.freebsd.org/base

Do not simply recreate the file manually and svn add it—this will cause history to be lost.

3.4.6 Fixing Mistakes

While we can do surgery in an emergency, do not plan on having mistakes fixed behind the scenes. Plan on mistakes remaining in the logs forever. Be sure to check the output of svn status and svn diff before committing.

Mistakes will happen but, they can generally be fixed without disruption.

Take a case of adding a file in the wrong location. The right thing to do is to svn move the file to the correct location and commit. This causes just a couple of lines of metadata in the repository journal, and the logs are all linked up correctly.

The wrong thing to do is to delete the file and then svn add an independent copy in the correct location. Instead of a couple of lines of text, the repository journal grows an entire new copy of the file. This is a waste.

3.4.7 Setting up a svnsync Mirror

You probably do not want to do this unless there is a good reason for it. Such reasons might be to support many multiple local read-only client machines, or if your network bandwidth is limited. Starting a fresh mirror from empty would take a very long time. Expect a minimum of 10 hours for high speed connectivity. If you have international links, expect this to take 4 to 10 times longer.

A far better option is to grab a seed file. It is large (~1GB) but will consume less network traffic and take less time to fetch than a svnsync will. This is possible in one of the following three ways:

% rsync -va --partial --progress freefall:/home/peter/svnmirror-base-r179637.tbz2 .
% rsync -va --partial --progress rsync://repoman.freebsd.org:50873/svnseed/svnmirror-base-r215629.tar.xz .
% fetch ftp://ftp.freebsd.org/pub/FreeBSD/development/subversion/svnmirror-base-r221445.tar.xz

Once you have the file, extract it to somewhere like home/svnmirror/base/. Then, update it, so that it fetches changes since the last revision in the archive:

% svnsync sync file:///home/svnmirror/base

You can then set that up to run from cron(8), do checkouts locally, set up a svnserve server for your local machines to talk to, etc.

The seed mirror is set to fetch from svn://svn.freebsd.org/base. The configuration for the mirror is stored in revprop 0 on the local mirror. To see the configuration, try:

% svn proplist -v --revprop -r 0 file:///home/svnmirror/base

Use propset to change things.

3.4.8 Committing High-ASCII Data

Files that have high-ASCII bits are considered binary files in SVN, so the pre-commit checks fail and indicate that the mime-type property should be set to application/octet-stream. However, the use of this is discouraged, so please do not set it. The best way is always avoiding high-ASCII data, so that it can be read everywhere with any text editor but if it is not avoidable, instead of changing the mime-type, set the fbsd:notbinary property with propset:

% svn propset fbsd:notbinary yes foo.data

3.4.9 Maintaining a Project Branch

A project branch is one that's synced to head (or another branch) is used to develop a project then commit it back to head. In SVN, “dolphin” branching is used for this. A “dolphin” branch is one that diverges for a while and is finally committed back to the original branch. During development code migration in one direction (from head to the branch only). No code is committed back to head until the end. Once you commit back at the end, the branch is dead (although you can have a new branch with the same name after you delete the branch if you want).

As per http://people.freebsd.org/~peter/svn_notes.txt, work that is intended to be merged back into HEAD should be in base/projects/. If you are doing work that is beneficial to the FreeBSD community in some way but not intended to be merged directly back into HEAD then the proper location is base/user/your-name/. This page contains further details.

To create a project branch:

% svn copy svn+ssh://svn.freebsd.org/base/head svn+ssh://svn.freebsd.org/base/projects/spif

To merge changes from HEAD back into the project branch:

% cd copy_of_spif
% svn merge svn+ssh://svn.freebsd.org/base/head
% svn commit

It is important to resolve any merge conflicts before committing.

3.5 Some Tips

In commit logs etc., “rev 179872” should be spelled “r179872” as per convention.

Don't remove and re-add the same file in a single commit as this will break the CVS exporter.

Speeding up svn is possible by adding the following to ~/.ssh/config:

Host *
ControlPath ~/.ssh/sockets/master-%l-%r@%h:%p
ControlMaster auto
ControlPersist yes

and then typing

mkdir ~/.ssh/sockets

Checking out a working copy with a stock Subversion client without FreeBSD-specific patches (OPTIONS_SET=FREEBSD_TEMPLATE) will mean that $FreeBSD$ tags will not be expanded. Once the correct version has been installed, trick Subversion into expanding them like so:

% svn propdel -R svn:keywords .
% svn revert -R .

This will wipe out uncommitted patches.