Halvan User Guide

This is the official users guide to Halvan, a high performance compute resource at UPPMAX.

Latest news

Please read this Users Guide for up-to-date information. All usage of the cluster must go through the batch system.

System configuration

As login node for Halvan, the login node for Kalkyl is used. The login node for Kalkyl is called kalkyl.uppmax.uu.se. (In fact, there may be two login nodes hidden behind this name; you will be automatically redirected to any one of these.)

The login node is an HP ProLiant SL170h G6 compute node with two Quad-core Intel® Xeon 5520 and 24GB RAM.

OS and software

See the OS user guide and installed software user guide for details about OS, compilers and installed software.

You will probably have good use of the following commands:

• uquota - telling you about your file system usage.
• projinfo -M halvan - telling you about the CPU hour usage of your projects. Please note that the default report interval for Halvan is half a year, and that core hour allocation overdrafts will be managed on that interval. The allocations for Halvan, Kalkyl and Grad are handled separately.
  
• jobinfo -M halvan - telling you about running and waiting jobs on Halvan.
• projmembers - telling you about project memberships.
  

Please note that most SLURM commands can be used for Halvan, if you add the "-M halvan" flag.

Accounts and log in

All access to this system is via secure shell interactive login to the login node, using the domain name kalkyl.uppmax.uu.se

   ssh -AX user@kalkyl.uppmax.uu.se

For questions concerning accounts and access to Kalkyl, please contact UPPMAX support.

Using the batch system

Please consult the Kalkyl User Guide for more information. Here you find mostly details that are special for Halvan.

To allow a fair and better usage of the system we use a resource manager to coordinate user demands. On Kalkyl we use the Slurm software.

Please note that most SLURM commands can be used for Halvan, if you add the "-M halvan" flag.

There is a job (walltime) timelimit of seven days (168 hours). If your job runs near this limit and you fear that it will crash into the it, please contact UPPMAX support.  and ask for a longer timelimit. Depending on various circumstances this request will be granted or denied.

We automatically clean the /scratch  directory. Please use the /scratch/<jobnr> subdirectory, also called $TMPDIR, only. We do the same with  the /dev/shm directory.

If you need exclusive use of Halvan, you must specify

• "-n 64", telling SLURM that you need all available cores.
  

Here are an exemple of how to get exclusive use of Halvan:

sbatch -M halvan -p halvan -A b2011999 -n 64 -t 5:00:00 jobscript

 "-n" specifices the minimum number of cores that you need.

Memory and cores are allocated together in Halvan, so each core allocated also allocates 32 GB of RAM, and so each 32 GB of RAM allocated also allocates one core.

So, if you need at most half of the memory on Halvan, i.e. 1024 GB of RAM, you should specify e.g.

sbatch -M halvan -p halvan -A b2011999 -n 32 -t 5-00:00:00 jobscript

or

sbatch -M halvan -p halvan -A b2011999 -n 1 --mem=1048576 -t 5-00:00:00 jobscript

where 1048576 is 1024 GB expressed in MB (1024 need to be multiplied with the GB-to-MB conversion factor 1024).

If you specify less than you use, one or two of these bad consequences might occur:

1. Together with other jobs that run on Halvan at the same time, you run into the memory limit and processes begin to crash.
2. The automatic memory watch on Halvan (not yet implemented) will abort your job as soon as it detects the problem.

 There are Kalkyl nodes containing 72 GB of RAM, so please use them if your need is not greater than that.