Mirror a Subversion repository

Objective

To create and maintain a mirror of a Subversion repository

Scenario

Suppose that there is a Subversion repository that you wish to mirror at the URL http://svn.example.com/svn/foo. The copy is to be located at /var/lib/svn/foo.

Method

Overview

As of version 1.4, Subversion provides the svnsync for replicating repositories. There are four required steps to the replication process:

1. Create an empty destination repository.
2. Install a pre-revprop-change hook script in that repository.
3. Initialise the destination repository for use as a mirror.
4. Synchronise the destination repository with the source repository.

In principle svnsync, the source repository and the destination repository can be hosted on three separate machines. In practice it is beneficial to run svnsync on the same machine as the destination repository and to access it directly via the filesystem:

• Access control is simplified because it can be achieved using POSIX file permissions.
• svnsync cannot lose connectivity to the destination repository, therefore there is less chance of the repository being left in a locked state.

Create the destination repository

Create an empty Subversion repository in the normal manner.

cd /var/lib/svn
svnadmin create foo

Limit write access to the repository

svnsync makes the assumption that it has exclusive control of the destination repository. If any revisions are committed independently then subsequent attempts to update the mirror will fail. For this reason it is desirable that write access to the repository be limited to the user that will run svnsync.

If svnsync accesses the destination repository directly via the filesystem or via SSH then it may be sufficient to ensure that the relevant user is the owner of the repository. Most GNU/Linux distributions use a default umask of 0022, meaning that newly created files and directories are readable by anyone but writable only by their owner.

If access is via Apache then the configuration file should distinguish between operations which alter the repository and those which merely read from it. One way to do this is by means of a LimitExcept section. Suppose that write access to the repository is needed by the user ‘sync’

<LimitExcept GET PROPFIND OPTIONS REPORT>
 Require user sync
</LimitExcept>

Install a pre-revprop-change hook script

svnsync uses revision properties (revprops) to record housekeeping information in the destination repository, including the URL of the source repository the most recent revision number that has been copied.

By default, Subversion does not allow revprops to be created or modified. The destination repository must therefore be reconfigured to permit those operations. This is done by installing a hook script called pre-revprop-change. All this script needs to do is exit with a return code of 0 (which indicates success). A single line will suffice:

#!/bin/sh

If the root of the repository is /var/lib/svn/foo then the hook script should be written to /var/lib/svn/foo/hooks/pre-revprop-change. The script must be executable:

cd /var/lib/svn
echo '#!/bin/sh' > foo/hooks/pre-revprop-change
chmod 755 foo/hooks/pre-revprop-change

Note the use of single quotes to prevent the exclamation mark from triggering history expansion. (Double quotes would not have the required effect.)

You may encounter a more elaborate pre-revprop-change script which allows only the user performing the synchronisation to commit revprops. This is a superficially reasonable restriction, but it is too narrow to be useful. To protect the integrity of the mirror it is necessary to prevent other users from committing anything. Once this has been done, a restriction specific to revprops would be redundant.

Initialise the repository as a mirror

Once the pre-revprop-change script is in place, the repository can be initialised for use as a mirror by means of the svnsync init command:

svnsync init file:///var/lib/svn/foo http://svn.example.com/svn/foo

What this does is initialise the housekeeping information described above. The init subcommand does not itself perform any copying.

Update the mirror

Once the destination repository has been initialised as a mirror, it can be brought into synchronisation with the source repository at any time by means of the svnsync sync command:

svnsync sync file:///var/lib/svn/foo

It is a good idea to issue the first synchronisation command manually in order to check that the mirror has been configured correctly. After that you can synchronise manually or automatically at your preference. In the latter case you should use the --non-interactive option to suppress questions:

svnsync --non-interactive sync file:///var/lib/svn/foo

Resynchronising when there is no work to be done places only a small amount of load on the servers hosting the repositories. If you own both servers then it is not unreasonable to resynchronise every five minutes, or even every minute if there is a reason to do so. Alternatively, you may prefer to resynchronise once per day in order to take advantage of periods when the servers would otherwise be idle.

If svnsync is scheduled very agressively then there is a risk of it being started before the previous instance has finished. This does not cause any harmful effects because svnsync locks the destination repository while it is running. If it cannot acquire the lock then it will eventually give up and exit.

Errors

PROPFIND returns 405 Method Not Allowed

An error of the form:

svnsync: PROPFIND request failed on '/svn/foo'
svnsync: PROPFIND of '/svn/foo': 405 Method Not Allowed (http://svn.example.com/svn/foo)

or

svnsync: Server sent unexpected return value (405 Method Not Allowed) in response to PROPFIND request for '/svn/foo'

in response to svnsync init may indicate that the specified source URL is not a Subversion repository (or if it is a repository, is not being served correctly using WebDAV).

OPTIONS of URL: 200 OK

An error of the form:

svnsync: OPTIONS of 'http://svn.example.com/svn/foo': 200 OK (http://svn.example.com)

in response to svnsync init could indicate the same problem as above, that the source URL is not a repository.

Failed to get lock on destination repos

An error of the form:

Failed to get lock on destination repos, currently held by 'test:c1ae7d34-e0de-4f4c-96b9-e983fe7c3317'

indicates that either:

• there is another instance of svnsync running, or
• that a previous instance of svnsync has terminated without unlocking the repository.

It is necessary to find out which of these is applicable. You can search for processes called ‘svnsync’ using the ps command:

ps -C svnsync

If svnsync is running and you believe it has hung then you will need to kill it before proceeding.

If svnsync has died or been killed, and the destination repository remains locked, you can unlock it with the command:

svn pd svn:sync-lock --revprop -r 0 file:///var/lib/svn/foo

Do not remove the lock unless you are confident that there is no instance of svnsync running. If two svnsync processes are allowed to write to the destination repository concurrently then it is possible they could leave it in an unusable state.

(In particular, do not assume that a process will necessarily die following a kill command. You should check using ps, and if necessary issue a kill -9.)

Repository has not been enabled to accept revision propchanges

An error of the form:

svnsync: Repository has not been enabled to accept revision propchanges;
ask the administrator to create a pre-revprop-change hook

indicates that the pre-revprop-change hook has not been installed. You can install it using the commands:

cd /var/lib/svn
echo '#!/bin/sh' > foo/hooks/pre-revprop-change
chmod 755 foo/hooks/pre-revprop-change

Revprop change blocked by pre-revprop-change hook (exit code 255)

An error of the form:

svnsync: Revprop change blocked by pre-revprop-change hook (exit code 255) with no output.

probably indicates that the pre-revprop-change hook has been installed but has not been made executable. You can make it executable using the commands:

cd /var/lib/svn
chmod 755 foo/hooks/pre-revprop-change

Destination HEAD is not the last merged revision

An error of the form:

svnsync: Destination HEAD (1) is not the last merged revision (0); have you committed to the destination without using svnsync?

probably indicates, as the message suggests, that the destination repository has been modified independently of svnsync.

In principle you could back out the spurious revisions using svnadmin dump and svnadmin load, then reset the svn:sync-last-merged-rev property to an appropriate value. In practice it is usually simpler and safer to delete the mirror and rebuild it from scratch.

You can prevent this problem from reoccurring by restricting write access to the repository, as described above.

• Conventions
• Privacy
• Legal