Spack v0.18

Getting Started

Spack is a tool that makes it easy for you to configure and maintain your own software modules. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.

To get started, login to the System and go to your home directory. As the very first step, create a tempdir for Spack builds (you will need it later):

mkdir -p /scratch/<path to your scratch folder>/spack/tmp
We recommend installing Spack into your home directory. Use /scratch for temporary data. If you don't know, what /scratch is, please read here or here.

Then run the following commands on the command line:

git clone -c feature.manyFiles=true https://github.com/spack/spack.git ~/spack

This will create a folder named “spack” into your home directory and clone the contents of the git repository into that newly created folder “spack”. Now go to your spack directory.

$ cd ~/spack
$ git checkout releases/v0.18
Branch 'releases/v0.18' set up to track remote branch 'releases/v0.18' from 'origin'.
Switched to a new branch 'releases/v0.18'
Spack does not update itself. After a while you might see only old versions with 'spack info <package>'. Use 'spack -V' to check your version. Please see section update spack.

To add the Spack commands to your environment run (this step has to be made after each login)

. ~/spack/share/spack/setup-env.sh

If you want this to be permanent, so that you can use the spack command after each login without executing the script first. You can add the execution of the script to your .bashrc file, e.g.:

Insert the following lines with an editor at the end of your ~/.bashrc

...
SPACK_ROOT="/home/<your-project-name>/<your-user-name>/spack"
export PATH=$SPACK_ROOT/bin:$PATH
. $SPACK_ROOT/share/spack/setup-env.sh

By default spack stores the temporary files under /tmp. This is shared with all users on the cluster and should not be used for your spack installation. Spack files in /tmp will be deleted on a regular base within 30 minutes after creation. In order to change the directory spack uses for its temporary files you have to set the new directory in the config.yaml. This can be done by the following commands.

First open your config.yaml located under ~/spack/etc/spack/defaults then change

  build_stage:
    - $tempdir/$user/spack-stage
    - $user_cache_path/stage
    # - $spack/var/spack/stage

to

  build_stage:
    # This path has to exist. Please create it, if not already done.
    - /scratch/<your-project-name>/<your-user-name>/spack/tmp

Now you are ready to install your own software and use it in your projects!

Installing Software

With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.

To list all the available packages use

spack list

To check if Spack has a package you need you can either search through the list or use

spack list <my-package-name>

If you need to know if spack has a specific version of your package, use

spack versions <package-name>

For further information on a package, e.g. dependencies, use

spack info <package-name>

Finally you can install your package with the latest version with

spack install <package-name>

If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”

spack install <package-name>@version

If you also need it with a specific compiler, append

spack install <package-name>@version %<compiler-name>

For further details visit Specs and Dependencies

Managing Software Packages

When you install multiple software packages, Spack uses all the already installed packages to install the new one.

To see all your installed software packages with their version use

(This will show all the software packages with their installed dependency packages)

spack find

If you only want to see your directly installed packages (installed with “spack install <package-name>) use

spack find --explicit

Managing Modules

If you install many software packages, “module avail” will print out all the installed software packages. To clean things up, you can install lmod with

spack install lmod

After that you have to create a “modules.yaml” file in the ”.spack“ folder, e.g. by

nano ~/.spack/modules.yaml

There you can specify what modules will be listed by the “module avail” command.

Your “modules.yaml” file should look like this, if you only want to show directly installed modules and want to show only necessary data.

 modules:
  tcl:
    # No hashs at the end of the module name
    hash_length: 0
    # For example openmpi/3.1.2%intel@19.1.0.166 
    naming_scheme: '{name}/{version}%{compiler.name}@{compiler.version}'
    blacklist_implicits: true

old nameing scheme syntax

Further details how to create your own flavor of modules.yaml can be found here: Module files

The last step is to apply your “modules.yaml” setting by using

spack module tcl refresh --delete-tree -y

Uninstalling Software

Uninstalling software packages is as easy as installing them.

First you should find the package you want to uninstall.

Then uninstall the package name with the version by using

spack uninstall <package-name>@version

If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.

You can check the dependencies of a package yourself by

spack find --deps gcc <package-name>

Adding new Compilers

Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers spack has found on your system run

spack compilers

any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available use.

module avail

then you can load the module of the compiler you want to add to spack

module load

When you have loaded the module of the compiler you want to add to spack, use

spack compiler find

to add the compiler to spack

Adding the Intel compiler

On the GOETHE HLR you can use the latest Intel compiler, you can load it with

module load comp/intel/2020.0

To add the compiler to spack, just type

spack compiler find
Which should add the intel compiler to your spack compilers.

Finally you have to edit the “compilers.yaml” file located at ”~/.spack/linux/“ by adding the two rpaths to the added Intel compiler. This example shows the configuration for “intel@19.1.0.166” which was previously added to spack.

If you intend to use a different version you need to change the paths, too.

...   
extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
...
So your “compilers.yaml” should have the entry
- compiler:
    spec: intel@19.1.0.166
    paths:
      cc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icc
      cxx: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icpc
      f77: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
      fc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort
    flags: {}
    operating_system: scientific7
    target: x86_64
    modules: []
    environment: {}
    extra_rpaths:
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64
    - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/

Using Packages

You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.

spack location --install-dir [your package]

an other way to use a package is to load it

spack load [your package]

although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use. For more information see Spack-Generated Modules. For better ways to assemble a consistent set of packages for building application programs see Generated Load Scripts

Update spack

In some cases you do not find with spack info <package> what you are looking for. Than you might want to update spack. Here are some steps how to do so.

update spack

spack -V                   # Your own version of spack

1. cd ~/spack              # Change into your spack directory
2. git pull                # If an error appears see next box
3. update your config.yaml # See top of the page

Following error appeared

git pull error

error: Your local changes to the following files would be overwritten by merge:
        etc/spack/defaults/config.yaml
Please, commit your changes or stash them before you can merge.
Aborting

# This error appears because you have changed the tmp-directory to 
# /scratch/<your-project-name>/<your-user-name>/spack/tmp,
# as we advized (see section install), within the config.yaml.
during the git pull process. Here you can find a whole discussion about it Link and how to solve it.

On possible way to go if you do not care about your old stuff

1. cd ~/spack              # Previous step like above
2. git pull                # Error appears
3. git stash save ""       # New step to deal with the error
4. git pull                # Redo it again
5. cd ~/spack/etc/spack/defaults/
6. Edit your 'config.yaml' # See at the top how to alter

Now you should be able so see the lastest packages which spack provides.

Remove spack

If something went wrong and you want to start from the scratch or just remove spack, please delete both following directories.

remove spack

#Think twice before using this!
rm -rf /your-home-directory/spack
rm -rf /your-home-directory/.spack

public/usage/spack.txt · Last modified: 2022/08/25 19:26 by geier
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0