Spack on UPPMAX
Spack is a simple package management tool or installer that also installs dependencies automatically to the main software. Installing a new software version does not break existing installations, so many configurations can coexist on the same system.
It offers a simple spec syntax so that users can specify versions and configuration options concisely. Spack is also simple for package authors: package files are written in pure Python, and specs allow package authors to maintain a single file for many different builds of the same package.
The UPPMAX staff has already other ways to install most software applications. Please use Spack only if other ways to install your tool is not possible or very difficult, e.g. requiring very many dependencies and it is not available through, e.g. Easybuild (that the staff can manage centrally). One or the reasons is that SPACK produces very many small files and that having two parallel build systems centrally may make things a little complex.
This guide may change with time. Please come back and see updates.
- This version assumes no vailable SPACK module, which may come in the near future.
- You have your own instance of Spack but can get a configuration file provided by UPPMAX.
Installing you own instance of SPACK
You may want to use your project folder if you want your colleagues to be able to run the application. Then change directory to a giood place before installing Spack.
$ cd <good place>
Step 1: clone spack
$ module load git $ git clone -c feature.manyFiles=true https://github.com/spack/spack.git
$ cd spack
To get version v0.18:
$ git checkout releases/v0.18
Next, add Spack to your path. Spack has some nice command-line integration tools, so instead of simply appending to your
PATH variable, source the Spack setup script.
$ source <root dir of spack>/spack/share/spack/setup-env.sh
Adding this line to your ~/.bachrc as well will activate the "spack commands" each time you start a new terminal session.
Orientation of the SPACK files
The Spack oriented files are stored in two places:
- Spack directory
- the cloned git repo
- directories (important in bold)
- bin spack executables
- etc configuration files
- lib libraries
- share documentation, scripts etc...
- var other settings
- opt produced after first installation, contains all packages (tools, dependencies and libraries)
tools are found in a tree: ...opt/spack/linux-<arch>/<compiler>/tool/
- local config and packages files
- directories (important in bold)
The .yaml files in the .spack/linux directory contains information which tolls you want to include from the UPPMAX system.
- The compilers.yaml file lists the compilers (intel or gcc) modules available to build your software tool.
- The packages.yaml file lists tools available already as modules.
By default, these files are empty but you can copy working "central" files that can be extended for your needs. The content of the files can be larger than the needed packages/compilers, i.e. only the packages /dependencies needed for your installation will be "taken" from these files and the rest will be ignored. Therefore, the UPPMAX staff may update these central files once in a while.
Do the following to get these templates (be sure to not overwrite old versions of these .yaml files that you configured yourself and might need).
$ cp /sw/build/spack/0.17.1/src/spack/share/spack/templates/compilers.yaml ~/.spack/linux/
$ cp /sw/build/spack/0.17.1/src/spack/share/spack/templates/packages.yaml ~/.spack/linux/
Install your program
Check available software applications via Spack:
$ spack list
$ spack list <search string>
Check already installed software applications with spack
$ spack find
$ spack find <search string>
Some installations won't need any compilers or "large dependencies". The installation is straightforward:
$ spack install <tool>
$ spack install zlib
In other cases, for larger applications tools that require larger dependencies (that we might already have as modules), watch the installation documentation to see what is needed. Any recommended compiler? You can also check with a "dry run" before installing, to see what Spack "thinks" its needs to install. Use the spec command:
$ spack spec -I <tool>
To check the presently, for Spack, available compilers, type:
$ spack compilers
If your desired compiler is not there you can add it by first loading the module and then integrate it into the compilers.yaml file with a spack command:
$ module load intel/20.4
$ spack compiler add
You can check if the compiler was added, either in the .spack/linux/compilers.yaml file or directly by:
$ spack compilers
To install a tool with a certain compiler version, if there are several compilers added for Spack, use "%". For specific version of the software tool or package, use "@".
$ spack install <tool>%<compiler>@<compiler-version>
$ spack install firstname.lastname@example.org
Large application tools may take a couple of hours so might be good to run in an interactive session (4 cores, -n 4).
$ spack install -j 4 <tool>
Use dependencies already available from our environment module system ('module load').
$ cat .spack/linux/packages.yaml
- Fill it with text,defining the spack name and lmod module names (be careful with indentations)
- Then install you tool, as above.
To install a specific version of a dependency with Spack, use the command "^":
$ spack install <tool>%<compiler>@<compiler-version>^<dependency>@<version>
Here is a summarizing table
Use your tool
$ spack load <tool> # module load of the install dependencies will not be needed here, since their paths are integrated in spack
- <tool> [<arguments>]
More to come... Meanwhile:
The builds are by default located here: