The right stuff: Part 1, Version Control - git and gitosis

I'm currently doing some work at a local startup that's looking to grow it's software engineering side. This has given me a unique opportunity to set up devops starting from a blank sheet. So I thought I'd blog about my progress along the way and share my experiences putting a toolset into place.

Part 1: Version Control - git and gitosis

As soon as you move beyond one developer version control becomes a necessity. Even alone, the advantages of being able to roll back your code, manage releases, and automate deployment are considerable. Version control is the foundation of any software engineering enterprise and so before anything else, we'll start here.

Things have progressed since the days of CVS, and now we have funky new distributed version control systems to play with. These allow developers to have a personal repository on their machines and push out changes to remote repositories for merging and release. Apart from being able to work more easily on the move, and encourage more regular commits of your code, this supposedly makes merges much less painless as well. So I'm going to run you through installing git, which is one of the most recognisable names in this category, and the server component, gitosis.

Before we start a note about environment - I'm working on a mac, but these instructions should be easily translatable to any platform. I'm also assuming that you have a 'management server' which is going to act as a persistant, backed up location for your repository and for other key software in later posts. In my case this is running Ubuntu, for other flavours of linux you will have to change the package install steps. I'm going to refer to these as local and mgt below and I'm assuming you also have an account called dev on mgt which we'll use to set things up.

Installation

Start by installing git on both local and mgt and setup your details correctly on local so commits are attributed:

git config --global user.name "John Doe"
git config --global user.email "john.doe@myco.com"

Next on mgt install gitosis, which on ubuntu is conveniently packaged:

sudo apt-get install gitosis

If you want to access your gitosis repository remotely, then ensure that your firewall/router has a rule setup to forward port 9418 to mgt. Next, create a ssh key setup for the dev user on mgt and initialise the repository:

ssh-keygen -t rsa
sudo -H -u gitosis gitosis-init <~/.ssh/id_rsa.pub

At this point, you're pretty much up and running, give it a go by checking out the admin repository on mgt:

mkdir ~/repo
cd ~/repo
git clone -o myco gitosis@mgt.myco.com:gitosis-admin.git

(Replace myco with a tag for your company; this option isn't strictly necessary, but I've found it useful when pushing changes to more than one place to be explicit about pushes to the company repository)

Adding a user

Now to create an account for yourself to access to your new repository. Generate yourself a ssh key if you don't have one already and copy it from local to the gitosis-admin project you just cloned:

ssh-keygen -t rsa
scp ~/.ssh/id_rsa.id dev@mgt.myco.com:~/repo/gitosis-admin/keydir/john.doe@myco.com.pub

Then edit ~/repo/gitosis-admin/gitosis.conf on mgt to create a new group for your team with your first project and add yourself to the members line. While you're there add yourself to the gitosis-admin team. Something like:

[gitosis]

[group gitosis-admin]
writeable = gitosis-admin
members = dev@mgt.myco.com john.doe@myco.com

[group teamawesome]
writable = myproject
members = john.doe@myco.com

Now for your first commit!

cd ~/repo/gitosis-admin
git add keydir/john.doe@myco.com.pub
git commit -am 'Created myproject and added John'
git push myco master

The git add does what it says and marks a file to be added to the repository - git won't add any files you don't tell it to, so you can have versioned and unversioned files in the same directory. The next line commits your changes to the local repository (in ~/repo/gitosis-admin/.git) with -a to tell git to commit all files which have changed. The last command then pushes the local repository to the gitosis server you cloned it from (which is also mgt in this first case).

Now you have access to the gitosis-admin project, let's add another user but this time locally. Get Jane to give you her public key and on local:

mkdir ~/repo
cd ~/repo
git clone -o myco gitosis@mgt.myco.com:gitosis-admin.git
cd gitosis-admin
cp janes-public-key.pub keydir/jane.doe@myco.com.pub
git add keydir/jane.doe@myco.com.pub

Edit gitosis.conf and add Jane to myproject members, then:

git commit -am 'Added Jane to myproject'
git push myco master

Now you've told gitosis-admin about myproject and added some users, lets actually make the project on local and initialise the local repository:

mkdir ~/repo/myproject
cd ~/repo/myproject
git init

And push it out to gitosis:

git remote add myco gitosis@mgt.mycom.com:myproject.git
git add .
git push myco master

Add a new file:

echo 'A test file' > test.txt
git add test.txt

Commit it locally (repository in ~/repo/myproject/.git):

git commit -am 'Added test file'

Push changes to gitosis for everyone else to grab:

git push myco master

And if Jane wants to work on the project from her machine she can do:

cd ~/repo
git clone -o myco gitosis@mgt.myco.com:myproject.git

One other useful command is if you're coming back to a repository that's already checked out and you want to make sure you have all the latest changes from gitosis:

git pull

So that's the basics, but there's one final thing I like to have setup... I've always liked to have a mailing list running which shows commits into the central repository so all developers can have an eye to what's going on and how projects are evolving. In this case I've setup a google group on our corporate google apps account and added John and Jane as members. Now to integrate gitosis with it. Unfortunately this requires editing a file in the actually repository itself on mgt:

cd /srv/gitosis/repositories/myproject.git
cat <<EOF |sudo tee -a config
[hooks]
mailinglist = git-commits-list@myco.com
announcelist = git-commits-list@myco.com
envelopesender = dev@myco.com
EOF
echo 'A meaningful description of my project' |sudo tee description

You also need to setup a hook. For future projects this will be rolled out by default by us dropping it into the templates directory so that you only have to edit the config file and provide a description as above:

sudo cp /usr/share/doc/git-core/contrib/hooks/post-receive-email /usr/share/git-core/templates/hooks/post-receive
sudo chmod +x /usr/share/git-core/templates/hooks/post-receive
cd /srv/gitosis/repositories/myproject.git/hooks
sudo -u gitosis ln -s sudo chmod +x /usr/share/git-core/templates/hooks/post-receive

That's it, try making a change and committing in your project - you should get an nice emailing telling you about the changes.

I hope that you've found this a useful, version control is one of the most worthwhile things to push for at any firm even though it seems complicated at first. Stick with it, soon it will seem like second nature. In the next post we'll get our teeth into something more complicated still - automating your infrastructure setup with Chef.

PS. Much of the above I'm typing up after the fact (note to self - document as I go along). So if you find any inaccuracies, missing steps, or the like I apologise and please let me know in the comments so I can fix.

Image courtesy of grendelkhan