2007-06-15
Using Git to track the Firefox CVS repository
Git is a distributed version control system that was written for managing the Linux kernel. I've been using it recently to make it easier to manage my code changes against a CVS repository for which I don't have write access.
The CVS repository is the codebase Firefox. Usually I'd check out a copy of the CVS repository, make my local changes, generate a patch and attach it to a Bugzilla entry for review. When it passes this then gets committed by someone with write access.
What becames hard to manage is dealing with checkouts that require a number of patches, keeping track of what has been applied, keeping patches around to be able to reverse them out and handling multiple copies of trees.
An even bigger issue is how to create patches that add directories or files when you don't have write access to CVS. 'cvs diff' only includes new files if they have been added via 'cvs add'. This requires write access to CVS. You can work around this by manually editing the local CVS files, or using a utility like 'cvsdo'. Unfortunately this still doesn't work for directories. The only way to add a directory to CVS, to have it picked up with 'cvs diff', is with 'cvs add' and write access to the repository. This issue and workarounds is described in the Mozilla documentation.
This problem will go away as Mozilla migrates to a new version control system, but in the meantime I decided to try tracking things with Git after reading an article by Diane Trout who is doing something similar.
I did a standard checkout of the sources for Firefox, following the Mozilla Build instructions. Within the checkout directory I created a git repository containing all the files, with relevant .gitignore instructions to not include CVS directories, etc. Something like this did the trick:
cvs -d ... checkout mozilla/client.mk
cd mozilla
make -f client.mk checkout MOZ_CO_PROJECT=browser
cd ..
git init
git add .
git commit -m "Initial import"
I keep this repository as the main repository linked to CVS. When I work on a bug or some functionality I can clone this repository and use git's branch management functionality to work on different things.
Periodically I update the main git repository with the latest changes from CVS. I have a script that I run that does something like this:
cvs update mozilla/client.mk
cd mozilla
make -f client.mk checkout
git add .
git commit -a -m "Update to CVS"
This does a 'bulk commit'. It doesn't import the individual CVS checkins with comments but for my purposes I don't need this. I just need something quick that works.
Now that the main repository is updated I can merge in the changes with my 'patches in progress' easily. Here is a commented example of some workflow:
# Clone the firefox repository for working on a bug
git clone /git/firefox.git bugfixes
cd bugfixes
# I like to leave the 'master' branch created by the clone
# as being a pure copy of the original repository and work
# on a branch. The following creates a branch to work on.
git checkout -b bug_123
# Now I make changes, add files, remove files, etc. I commit
# to the branch frequently with:
git add some_file_i_added.txt
git commit -a -m "Some message"
# At some point I want to update the 'master' branch with the
# recent CVS updates.
git checkout master
git pull /git/firefox.git
# When in the bug_123 branch I can merge in the changes I
# got from the latest CVS that I keep in 'master'
git checkout bug_123
git pull . master
This workflow works quite well. At any time I can create additional branches very cheaply to try things out, reverting back to the original branch if I change my mind, or to pull in the changes if I want to keep them:
# In branch bug_123 at the moment
git checkout -b try_something
# Make changes
git commit -a -m "My changes"
# Go to the original branch and pull in those changes to keep them
git checkout bug_123
git pull . try_something
To generate a patch that can be applied to the original CVS checkout, I can generate a 'git diff' in my branch with committed changes, against the 'master' branch containing the up to date CVS:
# In branch bug_123
git diff master >~/bug_123.patch
I can also generate a patch against a specific git commit:
git diff d889a66ff046f3737b87bd3e4098dcb156b9836f >~/bug_123.patch
Some miscellaneous commands that are also helpfull:
# What has been changed since last commit
git status
# A diff of my recent edits
git diff
# A log of commits
git log
# Push my bug_123 changes to the 'master' branch in a remote repository
git push user@server.com/git/changes.git bug_123:master
I've published my regularly updated git repository of the Firefox CVS tree on my server. It can be retrieved using the 'git' or 'http' protocols using:
# Git protocol
git clone git://double.co.nz/git/firefox.git
# HTTP protocol
git clone http://double.co.nz/git/firefox.git
Note that you can't browse the repository using the HTTP URL due to me having directory browsing turned off, but 'git clone' or 'git pull', etc works. You can browse the repository using gitweb via http://www.double.co.nz/cgi-bin/gitweb.cgi. This is actually an excellent way to view recent changes made to the Firefox CVS.
By adding a .mozconfig file in the 'mozilla' directory and doing 'make -f client.mk build' you can build your own trunk version of Firefox. A simple .mozconfig that will work on most platforms is:
. $topsrcdir/browser/config/mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-@CONFIG_GUESS@
mk_add_options MOZ_CO_PROJECT=browser
ac_add_options --enable-debug
For Mac OS X users, add the following line:
ac_add_options --with-macosx-sdk=/Developer/SDKs/MacOSX10.4u.sdk
You can run the new build on Linux or Windows with:
obj-dir/dist/bin/firefox
Or on Mac OS X:
obj-dir/dist/MinefieldDebug.app/Contents/MacOS/firefox
Replace 'obj-dir' with the object directory created during the build, and I recommend using -ProfileManager or something similar to do your testing a profile separate from your main Firefox profile.
If you're keen to try playing around with the code, I wrote a short post on how to add a new DOM element a while back that may give a bit of a start.
If you have any additional git tips or suggestions, please post a comment.
