Spack v0.18

Spack is a tool that makes it easy for you to configure and maintain your own software modules. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.

To get started, login to the System and go to your home directory. As the very first step, create a tempdir for Spack builds (you will need it later):

mkdir -p /scratch/<path to your scratch folder>/spack/tmp

Then run the following commands on the command line:

git clone -c feature.manyFiles=true https://github.com/spack/spack.git ~/spack

This will create a folder named “spack” into your home directory and clone the contents of the git repository into that newly created folder “spack”. Now go to your spack directory.

$ cd ~/spack
$ git checkout releases/v0.18
Branch 'releases/v0.18' set up to track remote branch 'releases/v0.18' from 'origin'.
Switched to a new branch 'releases/v0.18'

To add the Spack commands to your environment run (this step has to be made after each login)

. ~/spack/share/spack/setup-env.sh

If you want this to be permanent, so that you can use the spack command after each login without executing the script first. You can add the execution of the script to your .bashrc file, e.g.:

...
SPACK_ROOT="/home/<your-project-name>/<your-user-name>/spack"
export PATH=$SPACK_ROOT/bin:$PATH
. $SPACK_ROOT/share/spack/setup-env.sh

First open your config.yaml located under ~/spack/etc/spack/defaults then change

  build_stage:
    - $tempdir/$user/spack-stage
    - $user_cache_path/stage
    # - $spack/var/spack/stage

to

  build_stage:
    # This path has to exist. Please create it, if not already done.
    - /scratch/<your-project-name>/<your-user-name>/spack/tmp

Now you are ready to install your own software and use it in your projects!

With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.

To list all the available packages use

spack list

To check if Spack has a package you need you can either search through the list or use

spack list <my-package-name>

If you need to know if spack has a specific version of your package, use

spack versions <package-name>

For further information on a package, e.g. dependencies, use

spack info <package-name>

Finally you can install your package with the latest version with

spack install <package-name>

If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”

spack install <package-name>@version

If you also need it with a specific compiler, append

spack install <package-name>@version %<compiler-name>

For further details visit Specs and Dependencies

You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.

spack location --install-dir <your package>

One way to use a package is to load it:

spack load <your package>

Although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use.

When you install multiple software packages, Spack uses all the already installed packages to install the new one.

To see all your installed software packages with their version use

(This will show all the software packages with their installed dependency packages)

spack find

If you only want to see your directly installed packages (installed with “spack install <package-name>) use

spack find --explicit

With spack install lmod your are able to customize your spack modules. Unfortunately spack tend to change the syntax of the /.spack/modules.yaml from time to time. Further details how to create your own flavor of modules.yaml can be found here: Module files

outdated

spack install lmod

nano ~/.spack/modules.yaml

 modules:
  tcl:
    # No hashs at the end of the module name
    hash_length: 0
    # For example openmpi/3.1.2%intel@19.1.0.166 
    naming_scheme: '{name}/{version}%{compiler.name}@{compiler.version}'
    blacklist_implicits: true

spack module tcl refresh --delete-tree -y

Uninstalling software packages is as easy as installing them.

First you should find the package you want to uninstall.

Then uninstall the package name with the version by using

spack uninstall <package-name>@version

If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.

You can check the dependencies of a package yourself by

spack find --deps gcc <package-name>

Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers spack has found on your system run

spack compilers

any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available use.

module avail

then you can load the module of the compiler you want to add to spack

module load

When you have loaded the module of the compiler you want to add to spack, use

spack compiler find

to add the compiler to spack

On the GOETHE HLR you can use the latest Intel compiler, you can load it with

module load comp/intel/2020.0

To add the compiler to spack, just type

spack compiler find

Finally you have to edit the “compilers.yaml” file located at ”~/.spack/linux/“ by adding the two rpaths to the added Intel compiler. This example shows the configuration for “intel@19.1.0.166” which was previously added to spack.

...   
extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
...

- compiler:
    spec: intel@19.1.0.166
    paths:
      cc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icc
      cxx: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icpc
      f77: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
      fc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
    flags: {}
    operating_system: scientific7
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/

In some cases you do not find with spack info <package> what you are looking for. Than you might want to update spack. Here are some steps how to do so.

spack -V                   # Your own version of spack

1. cd ~/spack              # Change into your spack directory
2. git pull                # If an error appears see next box
3. update your config.yaml # See top of the page

Following error appeared

error: Your local changes to the following files would be overwritten by merge:
        etc/spack/defaults/config.yaml
Please, commit your changes or stash them before you can merge.
Aborting

# This error appears because you have changed the tmp-directory to 
# /scratch/<your-project-name>/<your-user-name>/spack/tmp,
# as we advized (see section install), within the config.yaml.

1. cd ~/spack              # Previous step like above
2. git pull                # Error appears
3. git stash save ""       # New step to deal with the error
4. git pull                # Redo it again
5. cd ~/spack/etc/spack/defaults/
6. Edit your 'config.yaml' # See at the top how to alter

Now you should be able so see the lastest packages which spack provides.

If something went wrong and you want to start from the scratch or just remove spack, please delete both following directories.

#Think twice before using this!
rm -rf /your-home-directory/spack
rm -rf /your-home-directory/.spack