This is an old revision of the document!
Table of Contents
Spack
Getting Started
Spack is a tool that makes it easy for you to configure and maintain your own modules for your projects. It is recommended to use Spack since its easy to use and has lots of capabilities. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.
To get started, Login to the Goethe-HLR and run the two following commands in the console:
git clone https://github.com/spack/spack.git
This will create a folder named “spack” into your home directory and clone the contents of the git repository into that newly created folder “spack”.
/scratch is only for temporary files. See here storage
To add the Spack commands to your shell use.
. spack/share/spack/setup-env.shTHIS STEP HAS TO BE MADE AFTER EACH LOGIN
If you want this to be permanent so that you can use the spack command after each login without executing the script first. You can add the execution of the script to your .bashrc. In order to do so use
Insert the following lines with an editor at the end of your ~/.bashrc
... SPACK_ROOT="/home/<your-project-name>/<your-user-name>/spack" export PATH=$SPACK_ROOT/bin:$PATH . $SPACK_ROOT/share/spack/setup-env.sh
/tmp
. This is shared with all users on the cluster and should not be used for your spack installation. In order to change the directory spack uses for its temporary files you have to set the new directory in the config.yaml
. This can be done by the following commands.
First open your config.yaml
located under ~/spack/etc/spack/defaults/
then change
build_stage: - $tempdir - $spack/var/spack/stage
to
build_stage: # - $tempdir # - $spack/var/spack/stage - /scratch/<your-project-name>/<your-user-name>/spack/tmp
/scratch PATH
has to exist!
Now you are ready to install your own software and use it in your projects!
Installing Software
With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.
To list all the available packages use
spack list
To check if Spack has a package you need you can either search through the list or use
spack list <my-package-name>
If you need to know if spack has a specific version of your package, use
spack versions <package-name>
For further information on a package, e.g. dependencies, use
spack info <package-name>
Finally you can install your package with the latest version with
spack install <package-name>
If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”
spack install <package-name>@version
If you also need it with a specific compiler, append
spack install <package-name>@version %<compiler-name>
For further details visit Specs and Dependencies
Managing Software Packages
When you install multiple software packages, Spack uses all the already installed packages to install the new one.
To see all your installed software packages with their version use
(This will show all the software packages with their installed dependency packages)
spack find
If you only want to see your directly installed packages (installed with “spack install <package-name>) use
spack find --explicit
Managing Modules
If you install many software packages, “module avail” will print out all the installed software packages. To clean things up, you can install lmod with
spack install lmod
After that you have to create a “modules.yaml” file in the ”.spack“ folder, e.g. by
nano ~/.spack/modules.yaml
There you can specify what modules will be listed by the “module avail” command.
Your “modules.yaml” file should look like this, if you only want to show directly installed modules and want to show only necessary data.
modules: tcl: # No hashs at the end of the module name hash_length: 0 # For example openmpi/3.1.2%intel@19.1.0.166 naming_scheme: '{name}/{version}%{compiler.name}@{compiler.version}' blacklist_implicits: true
Further details how to create your own flavor of modules.yaml can be found here: Module files
The last step is to apply your “modules.yaml” setting by using
spack module tcl refresh --delete-tree -y
Uninstalling Software
Uninstalling software packages is as easy as installing them.
First you should find the package you want to uninstall.
Then uninstall the package name with the version by using
spack uninstall <package-name>@version
If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.
You can check the dependencies of a package yourself by
spack find --deps gcc <package-name>
Adding new Compilers
Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers spack has found on your system run
spack compilers
any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available use.
module avail
then you can load the module of the compiler you want to add to spack
module load
When you have loaded the module of the compiler you want to add to spack, use
spack compiler find
to add the compiler to spack
Adding the Intel compiler
On the GOETHE HLR you can use the latest Intel compiler, you can load it with
module load comp/intel/2020.0
To add the compiler to spack, just type
spack compiler findWhich should add the intel compiler to your spack compilers.
Finally you have to edit the “compilers.yaml” file located at ”~/.spack/linux/“ by adding the two rpaths to the added Intel compiler. This example shows the configuration for “intel@19.1.0.166” which was previously added to spack.
... extra_rpaths: - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64 - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/ ...So your “compilers.yaml” should have the entry
- compiler: environment: {} extra_rpaths: - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64 - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/ flags: {} modules: [] operating_system: rhel7 paths: cc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icc cxx: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icpc f77: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort fc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort spec: intel@19.1.0.166 target: x86_64
Using Packages
You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.
spack location --install-dir [your package]
an other way to use a package is to load it
spack load [your package]
although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use. For more information see Spack-Generated Modules. For better ways to assemble a consistent set of packages for building application programs see Generated Load Scripts
Remove spack
If something went wrong and you want to start from the scratch or just remove spack, please delete both following directories.
#Think twice before using this! rm -rf /your-home-directory/spack rm -rf /your-home-directory/.spack