There are various storage options available on research computing facilities. You are advised to carefully consider the different features of each filesystem and how it might affects your usage scenarios.
Files in home directories are stored on a remote NFS server as opposed to locally on each machine. This enables you to access the same home directory which ever machine you login to.
Your home directory is, or should be, accessible only by you. It is securely backed up, so if you do happen to lose any data, like accidentally deleting a file, it is possible to recover it.
Home directories on research systems are not the same as the standard University’s N-drive.
5GB soft limit - You are free to use anything up to your allocated soft limit.
7GB hard limit - No write operation is allowed if hard limit is reached.
7 days grace period - The timer is triggered if the soft limit is reached but write operations is still allowed as long as it remains below the hard limit. Write operations will cease when the timer reaches the end of the grace period and will reset when usage is brought below the soft limit.
- You can check the status of your quota with the quota command.
- If you would like to increase your quota limits, contact MPS Support.
- A warning email will be sent to those who have exceeded their soft limit.
- Currently, quota is only implemented for Astronomy and TPP users. AMO, EPP and Maths home directories are not managed by quota.
Research filesystems are mounted on /research/<group> and access is controlled by Groups. Like for home directories, the research disks are also NFS, and therefore are not suitable for high I/O jobs. But data security is ensured by the backup regime.
The significance of this area is that it is shared so provides a convenient location for colaborative work, such as building custom software stacks or for storing shared code and data.
MPS research filesystems
|/research/astro||SAN/NFS||11TB||fs-astro||Incremental/Nightly||XFS Project Quota|
|/research/astrodata||SAN/NFS||16TB||fs-astro||Quarterly Snapshot||XFS Project Quota|
Apart from for Astronomy, research filesystems share a single RAID array on a single partition with the home directories. So XFS quota cannot be implemented because it would have to be applied to the entire filesystem, including the data in the shared research areas, this is not an option.
Astronomy have access to two separate SAN disks with different backup regimes and XFS project quota is applied to distribute the usage fairly amongst sub projects within the group. Quotas discourage users from doing high I/O operations, which should be reserved for Lustre.
Lustre is a distributed file system designed for use in High Performance Computing (HPC). The HPC cluster at Sussex utilises Lustre coupled with InfiniBand high speed interconnect to deliver maximum I/O performance. Lustre should be used for all data processing jobs on the HPC cluster, as opposed to your home or research disks.
See more about Lustre at Sussex here.