Cluster Usage Guidelines
This document aims to address some common questions about how to do things, and how NOT to do things on the "Cassini" computational cluster.
Getting your data on to and off of cassini
Copying data with rsync is the best choice. It's much more efficient than scp on small files. Run it as:
rsync -a -e "ssh -c blowfish" ewald:directory_to_copy ~/
The "blowfish" cipher provides weaker encryption but requires significantly less CPU power, so use it on the LAN!
NFS usage
There is one NFS server (cassini master) serving ALL computational nodes. Do your very best to avoid hammering it as it not only slows you and your jobs down, it also affects all other users and running jobs on the system.
If you will be using a set of data over and over, copy it to the local /scratch disk via rsync, either from within the job script or you may use cluster-fork or tentakel to copy data to all nodes:
cluster-fork rsync -a ~/directory_to_copy /scratch/
Jobs run time and queues
The job scheduler has been set up to evenly distribute available capacity to all users. That means that priority is given to users who did not use the cluster much recently (last 2 days). Make sure you submit your job to the correct queue as different queues have different run time limits, and also request the memory you need. Your job will be killed automatically if it exceeds any limit. If your jobs are short (less than an hour) then it is OK to flood the job queue with 5000+ jobs at once.
If your jobs interfere with other users because they use up all memory, cause extensive swapping, or hammer the NFS, we may kill them at our discretion. So don't do it.
Disk usage
There is a total of 6 TB of disk space attached to this system shared among all users. Please copy your data off of cassini when you are done processing it. This machine is NOT for permanent storage of your data. At this point we are monitoring but not enforcing disk space quotas, let's try to keep it that way.
Software
Common tools such as BLAST, interpreters like Perl, PHP and Python are installed locally on every node. Other shared software such as the CCP4 suite, PsiPred, DSSP, Rosetta and CHARMM are available in /software, which is visible to all nodes. An updated version of Python with common add-ons (numpy, gd, pylab) is available in /software/python. To request additional software to be added to this repository, email Luki.
Databases
A local copy of the PDB, Pfam, UniProt, the nr database for BLAST, as well as prokaryotic genomes are available in /databases. These are updated on a regular basis (typically monthly). Additional databases can be added. Please do not keep copies of databases you downloaded in your home directory.
Tips
To run a job interactively (for troubleshooting), reserve a node with qlogin -- you will get a shell to execute your commands in. See also other tips here.
