There is a HPC git repository server that is open to all researchers. This enables you to push your code up to a server that you can then use to collaborate with other people, or just to back up a private repository to.
The main benefits of this over a major third-party service like github or bitbucket is:
- Unlimited private/public repositories (within reason... if your repo is multi-gigabytes please tell mps-support before uploading).
- University managed – collaboration within the university will be simple as it utilises your University credentials.
- Fine-grained access control if you wish. There is potential to have as detailed access control as you wish. You can have it completely private to just you, or a handful of your collaborators. Or you can have specific branches that some can write to but not others. If this sounds useful then please let us know and we can help you set this up.
The git server runs on gitolite.
- Access to the server is managed via SSH keys. If you don’t already have an SSH keypair, or know what that is, this page explains it well. You only need to do Steps 1 and 2 there.
- Once you have an ssh keypair email email@example.com with a copy of the public part of the key (this is usually named ~/.ssh/id_rsa.pub or sometimes ~/.ssh/id_dsa.pub). Also in that email, let us know your ITS username.
If you have lots of computers with different SSH keypairs, that you wish to work from then (at some point) you need to email over those public keypairs too. Alternatively, it might be worth reducing the number of keys you have if they don’t need to all be different
- After this, the support team will need to add your key to the server. They will let you know once this is done, usually it is very quick.
- Once you have access to the server, you can check that all is well by doing the following
$ ssh -T firstname.lastname@example.org hello testuser, this is git@malthael running gitolite3 184.108.40.206-1.el6 on git 1.7.1 C CREATOR/..* R sites/mps-computing
If this works you will see output like the above. What this command does is show you want repositories you have access to on the server.
- The line C CREATOR/..*, means that you can create any number of private repositories under the namespace, <ITS-username>/repository. See below for more about private repos.
- The line R sites/mps-computing refers to another repository that you have read access to, where the R above means the read permission. If it showed an W as well that would mean write access to the repository.
Types of Repositories¶
You now have access to the git server. You can create two types of repositories:
These are stored under your own namespace, which begins <its_username>/..., so from the output above, these will appear as:
R C CREATOR/..* R W username/repo R W username/sub_directory/another_repo
You can create as many of these as you like and no intervention is needed from the support team - you just push to the location and it sets up the repo for you. See below for the details on how to do this.
All personal repos default to being completely private by default. Only you have read and write access to them. If you wish to then allow others to colloborate with you on this repository, see the section below on how to do this.
You can arbitrarily nest your repositories under your namespace. Namespace here really just means that all your private repos start under the top level directory beginning with your $its_username. These repositories are stored on the server, literally in a directory hierarchy as shown above, so you can structure the repositories as you wish.
A project based repository really just means one that has it’s own top-level namespace, (i.e: it doesn’t start with your ITS username ) and that project permissions are managed more centrally by the git admin. You can still keep this private if you wish or have any other combination or access rights that you prefer.
Unlike personal repos above these can only be created by the support team, and their permissions will need to be controlled by them too. This is simple for them to do though, so it just requires sending an email to email@example.com with the name of the project, and who you want to collaborate on it with.
Quickstart: Personal Repository¶
If you have an existing git repository already that you just want to push to the server then the below is what you do on your workstation.
If you don’t have a git repo yet, setting one up is as simple as:
$ cd path/to/project $ git init $ git add file_name directory_name [...] # Add as many files/directories as you want to track $ git commit
- Start from the git repository on your workstation.
$ cd path/to/repo
- Check if you have any existing remote repositories configured.
$ git remote
Check the output here, as you may have an existing remote already if you already push to another git server. If so note the name of the remote (probably origin).
- Add the MPS git server as a remote repository for your project.
$ git remote add origin firstname.lastname@example.org:username/repo_name
‘origin’ here is a label to identify the server you will be pushing to. If origin is already taken from the step above, you need to rename this to something else, perhaps ‘susx’.
- Push your local repository to the server, and check it syncs correctly.
$ git push --set-upstream origin master $ git pull
This will push your master branch to the server and set up a remote branch called origin/master which you will push to and pull from. If you have many branches (for say different lines of development in your project) you can either push these individually too, substituting their branch names for master above, or you can push all your branches by doing: $ git push --set-upstream --all origin master
Your git repository is now synced to the server. You can now run git push and git pull to send and retrieve changes from the server.
Managing Repository Permissions/Access¶
How to share a personal repo with other users (both read-only access and read-write, or branch specific access)
Personal repositories you create like above, are private by default, so only you can see, clone and write to it.
However you can change this when you want to collaborate or share your repo with other people, and you can do this yourself at any time (or you can ask @mps-support to do it for you of course).
To list your repository’s permissions, you can do the following:
$ ssh email@example.com perms -l username/repo_name # By default this is what you get back OWNERS @admins
As the CREATOR of the repo, you have special rights that aren’t listed here to basically do what you want to the repo - you can read, write, modify history and change permissions.
The line OWNERS = @admins, grants the members of the @admins group, a similar level of access to your repo – this is unfortunately necessary so that we can provide support for things like changing permissions for you when you request it, or helping people out with git problems.
Granting READ-ONLY access¶
You grant read-only access (eg: the person can git clone ... your repo, but can’t git push ... anything back to you, by doing the following:
$ ssh firstname.lastname@example.org perms username/repo_name + READERS collaborator_username # eg: if your username is mb325 and you want to allow access to your repo 'puppet' to tku25 $ ssh email@example.com perms mb325/puppet + READERS tku25 $ ssh firstname.lastname@example.org perms -l mb325/puppet OWNERS @admins READERS tku25
To remove a permission, just change the + to a -:
$ ssh email@example.com perms username/repo_name - READERS collaborator_username
Note that this is already quite powerful for collaboration in a git context, as your collaborator can clone your repository and then push their clone to their own namespace on the git server, eg: tku25 above could then do:
# Inside their clone of mb325/puppet above $ git remote add my_fork firstname.lastname@example.org:tku25/puppet $ git push -u my_fork master $ git remote origin my_fork
tku25 now has two remotes for his clone or ‘fork’ of mb325/puppet. So importantly, he can make changes to the code and commit those changes to his fork of mb325/puppet, and then push these commits to his own remote repository. This means that he can then work in parallel to mb325 on the same code base affecting each other’s work. When he has something he wants mb325 to take notice of, maybe to merge back into mb325’s repository, he can contact mb325 to ask him to ‘fetch’ this new work from tku25’s repository. This is a common git workflow (github does this excellently with their pull-request feature) and something I will try to discuss at greater length below.
Granting WRITE access¶
Write access means they can git push ... to directly to your repo.
$ ssh email@example.com perms username/repo_name + WRITERS collaborator_username # and similarly to remove write permission $ ssh firstname.lastname@example.org perms username/repo_name - WRITERS collaborator_username
Note that this is more similar to the svn (subversion) style workflow where all collaborators point to one remote repository and everyone commits there. You need to be more dilligent about git pull-ing and git merge-ing work done on the remote repository. However git has excellent branching support, so you should use a workflow with your collaborators where everyone is working in their own branch, and then merging into the master branch only when ready.
I’ll try to cover this workflow along with others in detail below.
Research Collaboration: An Example¶
Full example of how to structure your team’s SCM workflow. Cover a couple of common setups such as an ‘SVN’ style single shared master repository with all developers committing into it, to a more ‘git’ decentralised model with multiple remotes and ‘pull requests’ into a central master.