Following gphoto SVN development with GIT

Since I started developing the Capa photo capture application, I’ve been following development of gphoto much more closely. Unfortunately gphoto is using subversion for source control. There are many things wrong with subversion in comparison to modern SCM systems like Mercurial or GIT. In this particular case though, the main problem is speed, or lack thereof. gphoto uses sourceforge as its hosting service and sf.net subversion servers are slower than you can possibly imagine. As an example, run ‘svn log’ to browse changes and you’ll be waiting 30 seconds for it to even start to give you an answer. Then run ‘svn diff’ to look at the contents of a change and you’ll be waiting another 30 seconds or more. Totally unacceptable. Once you’ve used a distributed SCM system like Mercurial or GIT, you cease to have tolerance for any operations which take longer than a 2-3 seconds.

Fortunately, GIT has the ability to checkout directly from SVN repository. The gphoto SVN repository actually contains many separate sub-projects in it and I didn’t want to import them all to my local GIT repository. This meant I couldn’t make use of the branch / tag tracking support directly and had todo things the long way. The good news is that the long way has already been blogged about and it isn’t hard.

There were two projects I was interested in getting, libgphoto (the main library) & gphoto (the command line frontend) and I wanted each to end up in their own GIT repository. For both, I wanted the trunk and 2.4.x branch. Starting with gphoto, since it has much less history, the first step was to clone the trunk

# git svn clone https://gphoto.svn.sourceforge.net/svnroot/gphoto/trunk/gphoto2 gphoto2

This takes a fairly long time because it pulls down every single SVN changeset in the repository. Once that’s complete though, the .git/config contains

[svn-remote "svn"]
        url = https://gphoto.svn.sourceforge.net/svnroot/gphoto/trunk/gphoto2
        fetch = :refs/remotes/git-svn

And the local ‘master’ branch is connected to the ‘git-svn’ remote.

$ git branch -a
* master
  remotes/git-svn

Anytime further changes are made in the SVN repository, those can be pulled down to the local GIT repository using git svn fetch git-svn. At this point it is possible to add in the branches. Simply edit the .git/config file and add another ‘svn-remote’ entry, this time pointing at the branch path.

[svn-remote "svn24"]
        url = https://gphoto.svn.sourceforge.net/svnroot/gphoto/branches/libgphoto2-2_4/gphoto2
        fetch = :refs/remotes/git-svn-2.4

And then pull down all the changes for that branch, and create a local branch for this

# git svn fetch svn24
# git checkout -b v2.4 git-svn-2.4

This leaves a local branch ‘v2.4’ and a remote branch ‘git-svn-2.4’

$ git branch -a
  master
* v2.4
  remotes/git-svn
  remotes/git-svn-2.4

That takes care of the gphoto2 frontend command line app codebase. It is then a simply matter to repeat the same thing substituting libgphoto2 into the SVN paths to checkout the library codebase. Though this takes a little longer because it has much much more history. This little upfront pain to clone the SVN repo to GIT will be paid back many hundreds of times over thanks to the speed that GIT brings to SCM operation.

The moral of the story is simple: Don’t ever choose subversion. If you have the choice, use GIT. If you don’t have the choice, then mirror SVN to GIT anyway.

Edit: One thing I forgot to mention is that after setting up all branches, run a git gc on the repo. This will dramatically reduce the disk usage & speed up GIT operations further

$ du -h -c -s .
45M .
45M total
$ git gc
Counting objects: 3695, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3663/3663), done.
Writing objects: 100% (3695/3695), done.
Total 3695 (delta 3081), reused 0 (delta 0)
$ du -h -c -s .
5.0M .
5.0M total

Going from 45 MB to 5 MB is quite impressive !