Spack

Getting Started

Spack is a tool that makes it easy for you to configure and maintain your own modules for your projects. It is recommended to use Spack since its easy to use and has lots of capabilities. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.

To get started, Login to the Goethe-HLR and run the two following commands in the console:

git clone https://github.com/spack/spack.git

This will create a folder named “spack” into your home directory and clone the contents of the git repository into that newly created folder “spack”.

We recommand to install Spack into your home directory.
/scratch is only for temporary files. See here storage

To add the Spack commands to your shell use.

. spack/share/spack/setup-env.sh
THIS STEP HAS TO BE MADE AFTER EACH LOGIN

If you want this to be permanent so that you can use the spack command after each login without executing the script first. You can add the execution of the script to your .bashrc. In order to do so use

Insert the following lines with an editor at the end of your ~/.bashrc

...
SPACK_ROOT="/home/<your-project-name>/<your-user-name>/spack"
export PATH=$SPACK_ROOT/bin:$PATH
. $SPACK_ROOT/share/spack/setup-env.sh

By default spack stores the temporary files under /tmp. This is shared with all users on the cluster and should not be used for your spack installation. In order to change the directory spack uses for its temporary files you have to set the new directory in the config.yaml. This can be done by the following commands.

First open your config.yaml located under ~/spack/etc/spack/defaults/ then change

  build_stage:
     - $tempdir
     - $spack/var/spack/stage

to

  build_stage:
#      - $tempdir
#      - $spack/var/spack/stage
      - /scratch/<your-project-name>/<your-user-name>/spack/tmp

The above /scratch PATH has to exist!

Now you are ready to install your own software and use it in your projects!

Installing Software

With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.

To list all the available packages use

spack list

To check if Spack has a package you need you can either search through the list or use

spack list <my-package-name>

If you need to know if spack has a specific version of your package, use

spack versions <package-name>

For further information on a package, e.g. dependencies, use

spack info <package-name>

Finally you can install your package with the latest version with

spack install <package-name>

If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”

spack install <package-name>@version

If you also need it with a specific compiler, append

spack install <package-name>@version %<compiler-name>

For further details visit Specs and Dependencies

Managing Software Packages

When you install multiple software packages, Spack uses all the already installed packages to install the new one.

To see all your installed software packages with their version use

(This will show all the software packages with their installed dependency packages)

spack find

If you only want to see your directly installed packages (installed with “spack install <package-name>) use

spack find --explicit

Managing Modules

If you install many software packages, “module avail” will print out all the installed software packages. To clean things up, you can install lmod with

spack install lmod

After that you have to create a “modules.yaml” file in the ”.spack“ folder, e.g. by

nano ~/.spack/modules.yaml

There you can specify what modules will be listed by the “module avail” command.

Your “modules.yaml” file should look like this, if you only want to show directly installed modules and want to show only necessary data.

 modules:
  tcl:
    # No hashs at the end of the module name
    hash_length: 0
    # For example openmpi/3.1.2%intel@19.1.0.166 
    naming_scheme: '{name}/{version}%{compiler.name}@{compiler.version}'
    blacklist_implicits: true

old nameing scheme syntax

Further details how to create your own flavor of modules.yaml can be found here: Module files

The last step is to apply your “modules.yaml” setting by using

spack module tcl refresh --delete-tree -y

Uninstalling Software

Uninstalling software packages is as easy as installing them.

First you should find the package you want to uninstall.

Then uninstall the package name with the version by using

spack uninstall <package-name>@version

If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.

You can check the dependencies of a package yourself by

spack find --deps gcc <package-name>

Adding new Compilers

Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers spack has found on your system run

spack compilers

any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available use.

module avail

then you can load the module of the compiler you want to add to spack

module load

When you have loaded the module of the compiler you want to add to spack, use

spack compiler find

to add the compiler to spack

Adding the Intel compiler

On the GOETHE HLR you can use the latest Intel compiler, you can load it with

module load comp/intel/2020.0

To add the compiler to spack, just type

spack compiler find
Which should add the intel compiler to your spack compilers.

Finally you have to edit the “compilers.yaml” file located at ”~/.spack/linux/“ by adding the two rpaths to the added Intel compiler. This example shows the configuration for “intel@19.1.0.166” which was previously added to spack.

If you intend to use a different version you need to change the paths, too.

...   
extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
...
So your “compilers.yaml” should have the entry
- compiler:
    environment: {}
    extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
    flags: {}
    modules: []
    operating_system: rhel7
    paths:
      cc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icc
      cxx: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icpc
      f77: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
      fc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
    spec: intel@19.1.0.166
    target: x86_64

Using Packages

You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.

spack location --install-dir [your package]

an other way to use a package is to load it

spack load [your package]

although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use. For more information see Spack-Generated Modules. For better ways to assemble a consistent set of packages for building application programs see Generated Load Scripts

Remove spack

If something went wrong and you want to start from the scratch or just remove spack, please delete both following directories.

#Think twice before using this!
rm -rf /your-home-directory/spack
rm -rf /your-home-directory/.spack

public/usage/spack.txt · Last modified: 2020/05/20 12:25 by jankowiak
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0