Accessing the Cluster¶
Request for Access¶
Access to the HPC cluster is not granted automatically.
You need to have the following prerequisites:
- Be a registered member of University of Sussex.
- Have an active IT Services computing account.
To request for access, please email:
Please include your ITS username, department, research group and supervisor’s name (if applicable).
Login to Cluster¶
You can access the cluster from from any location on or off the campus. You just have to login to one of the login nodes:
- All users: apollo.hpc.susx.ac.uk
- EPP users: feynman.hpc.susx.ac.uk
- Economics users: econ.hpc.susx.ac.uk
These machines allow you to submit jobs to the batch queueing system, which in turn delegates the workload to the compute nodes.
From Linux & OS X¶
Once your account is set up, you can SSH to the login nodes using your ITS username and password:
e.g. ssh -X email@example.com, where <host> is apollo.hpc.susx.ac.uk (the login node) and the -X option is to forward X sessions, see man page for SSH (man ssh).
Using SSH keys
Entering a password every time you login gets tedious very quickly, so you’ll probably want to set up password-less SSH login. You will need to do this if you plan to use our git server, so it is highly recommended.
In order to do that you only need to follow a 2 step process
1. Generate your ssh-key pair
You’ll need first to think of a passphrase. That might sound like a contradiction - it supossed to be a password-less login, however, you need to protect your private key with a password in case it gets stolen, specially if your are generating it into a personal machine. The passphrase is then used to unlock the private key but you would usually only do this once a day. After that you’ll be able to use the unlocked key to perform password-less logins.
[usr123@imoen ~]$ ssh-keygen Generating public/private rsa key pair. Enter file in which to save the key (/home/usr123/.ssh/id_rsa): Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /home/usr123/.ssh/id_rsa. Your public key has been saved in /home/usr123/.ssh/id_rsa.pub. The key fingerprint is: 8e:e9:0f:75:32:79:ff:2f:bc:99:d5:74:14:47:26:5c firstname.lastname@example.org The key's randomart image is: +--[ RSA 2048]----+ | ..+E| | .oo| | .| | . . | | S o o| | = = . .o| | + . .. o| | . . .o+ | | ... =+.| +-----------------+
2. Copy it across to the the login node
[usr123@imoen ~]$ ssh-copy-id email@example.com /bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys firstname.lastname@example.org's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'email@example.com" and check to make sure that only the key(s) you wanted were added.
Now you can connect to the login node:
[usr123@imoen ~]$ ssh apollo.susx.ac.uk Enter passphrase for key '/home/usr123/.ssh/id_rsa': Last login: Thu Mar 9 01:15:54 2017 from imoen [usr123@apollo ~]$
Note though that it will ask yout for your passphrase avery time, unless you run an ssh-agent. Modern desktop Linux distribution such as Centos, Ubuntu, Debian, Fedora etc. have an SSH agent typically integrated into the standard GNOME session startup scripts, it is called gnome-keyring.
SSH config file
Another worthwhile tip that is highly recommended is to utilise an SSH config file to simplify your connections. Simply create the file (if it doesn’t already exist) ~/.ssh/config
Then add a block like the below to this file,
Host apollo hostname apollo-login.hpc.susx.ac.uk user <username> ForwardX11 yes
This allows you to type ssh apollo and it will connect you to the apollo-login with the username you specified. Similar to an alias.
This is just the tip of the iceberg for SSH configs. There is another one in remote-access.
SSH proxy configuration
Due to security issues, direct access to login nodes has been restricted from off-campus locations (since Dec 2016), so at the moment, external connections should pass trhough unix.sussex.ac.uk
if you do not want to do a 2-step process each time you want to connect, you can configure your client to go straight to the login node following the instructions bellow:
copy your id to unix.sussex.ac.uk
[your_work-station] $ ssh-copy-id [username]@unix.sussex.ac.uk
check you can login to unix.sussex.ac.uk without a password:
[your_work-station]$ ssh unix.sussex.ac.uk Last login: Wed Mar 8 12:19:19 2017 from ... Welcome to the University of Sussex
Into unix.sussex.ac.uk generate an id without passphrase (data on university provided storage can be considered safe, so you do not need to add a passphrase here):
[username@boole:~ ] ssh-keygen
Copy this new id into apollo
[username@boole:~ ] $ ssh-copy-id apollo.hpc.susx.ac.uk
Check you can login to apollo from boole without password
[username@boole:~ ] $ ssh apollo.hpc.susx.ac.uk
Go back to your work-station and add the config for the apollo-remote connection into $HOME/.ssh/config (change your username as apporpriate)
Host apollo-remote HostName apollo.hpc.sussex.ac.uk User [username] ProxyCommand ssh -Y [username]@unix.sussex.ac.uk nc %h %p
test using apollo-remote from your work-station, it should go straight into apollo:
[your_work-station] $ ssh -Y apollo-remote [your_work-station] $ touch test [your_work-station] $ scp test apollo-remote: [your_work-station] $ ssh apollo-remote "ls -l test"
Connected to the Login Gateway: Basic Setup¶
Once you have gained access and connected to one of the cluster’s login gateways, you should first set up your basic shell environment.
A lot of this section we will attempt to standardise and set up for new users, but for people with existing accounts it is worth checking here for things you might be missing Check if you have the following two files on the login gateway
At the shell prompt type -bash-4.1$ ls -al:
-bash-4.1$ ls -al total 40 drwxr-xr-x 17 its-username root 4096 Sep 18 12:52 . drwxr-xr-x 36 root root 4096 Jul 30 16:40 .. -rw------- 1 its-username its-username_g 10073 Sep 19 09:22 .bash_history -rw-r--r-- 1 its-username its-username_g 114 Sep 10 12:35 .bashrc drwx------ 3 its-username its-username_g 23 Aug 2 12:15 .bright computing drwxr-xr-x 3 its-username its-username_g 18 Aug 2 12:17 .cm drwxr-xr-x 2 its-username its-username_g 27 Jul 30 07:59 .config drwx------ 3 its-username its-username_g 24 Jul 28 10:22 .dbus drwxr-xr-x 2 its-username its-username_g 6 Aug 9 14:09 .felix drwxr-xr-x 2 its-username its-username_g 6 Aug 20 09:13 files drwx------ 2 its-username its-username_g 6 Sep 16 09:12 .gconf drwx------ 2 its-username its-username_g 24 Sep 16 09:13 .gconfd drwx------ 4 its-username its-username_g 32 Aug 2 12:15 .gnome2 drwx------ 2 its-username its-username_g 6 Aug 2 12:15 .gnome2_private drwxr-xr-x 3 its-username its-username_g 19 Jul 30 07:58 .matlab drwxr-xr-x 4 its-username its-username_g 37 Aug 8 15:26 .mozilla drwx------ 2 its-username its-username_g 53 Sep 4 15:24 .pulse -rw------- 1 its-username its-username_g 256 Aug 8 15:28 .pulse-cookie -rw------- 1 its-username its-username_g 1736 Sep 16 09:13 .recently-used.xbel drwx------ 2 its-username its-username_g 76 Jul 31 11:15 .ssh drwxr-xr-x 2 its-username its-username_g 50 Jul 30 07:58 .subversion -rw------- 1 its-username its-username_g 2595 Sep 10 12:35 .viminfo -rw------- 1 its-username its-username_g 1677 Sep 18 12:52 .Xauthority
Look for either of the following files .bashrc, .bashprofile and .profile
When you connect via ssh, your shell will read from .bash_profile and .profile if they are present (hence you only need one or the other of these).
As a bare minimum, your .bash_profile should contain::
module load sge module load gcc
The sge module contains the commands needed to submit jobs to the cluster and gcc is needed if you use gcc at all which is most people – note if you don’t load the gcc module, you will use the system’s gcc version which does not support the recent AMD cpus we have on certain nodes.
Once you can login to the login nodes, you then have a few options:
- See the software section for information about the cluster’s provided software, such as compilers, SDKs, common programming libraries and packages, and other useful scientific software such as Matlab, R, Mathematica.
- To jump right in and start compiling, or testing your code runs on the cluster, read interactive.
- To start writing batch, or headless jobs for submission to the cluster see running-jobs.