Spack v0.18
Getting Started
Spack is a tool that makes it easy for you to configure and maintain your own software modules. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.
To get started, login to the System and go to your home directory. As the very first step, create a tempdir for Spack builds (you will need it later):
mkdir -p /scratch/<path to your scratch folder>/spack/tmp
/scratch
for temporary data. If you don't know, what /scratch
is, please read here or here.
Then run the following commands on the command line:
git clone -c feature.manyFiles=true https://github.com/spack/spack.git ~/spack
This will create a folder named “spack” into your home directory and clone the contents of the git repository into that newly created folder “spack”.
Now go to your spack
directory.
$ cd ~/spack $ git checkout releases/v0.18 Branch 'releases/v0.18' set up to track remote branch 'releases/v0.18' from 'origin'. Switched to a new branch 'releases/v0.18'
To add the Spack commands to your environment run (this step has to be made after each login)
. ~/spack/share/spack/setup-env.sh
If you want this to be permanent, so that you can use the spack command after each login without executing the script first. You can add the execution of the script to your .bashrc
file, e.g.:
Insert the following lines with an editor at the end of your ~/.bashrc
... SPACK_ROOT="/home/<your-project-name>/<your-user-name>/spack" export PATH=$SPACK_ROOT/bin:$PATH . $SPACK_ROOT/share/spack/setup-env.sh
/tmp
. This is shared with all users on the cluster and should not be used for your spack installation. Spack files in /tmp
will be deleted on a regular base within 30 minutes after creation. In order to change the directory spack uses for its temporary files you have to set the new directory in the config.yaml
. This can be done by the following commands.
First open your config.yaml
located under ~/spack/etc/spack/defaults
then change
build_stage: - $tempdir/$user/spack-stage - $user_cache_path/stage # - $spack/var/spack/stage
to
build_stage: # This path has to exist. Please create it, if not already done. - /scratch/<your-project-name>/<your-user-name>/spack/tmp
Now you are ready to install your own software and use it in your projects!
Installing Software
With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.
To list all the available packages use
spack list
To check if Spack has a package you need you can either search through the list or use
spack list <my-package-name>
If you need to know if spack has a specific version of your package, use
spack versions <package-name>
For further information on a package, e.g. dependencies, use
spack info <package-name>
Finally you can install your package with the latest version with
spack install <package-name>
If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”
spack install <package-name>@version
If you also need it with a specific compiler, append
spack install <package-name>@version %<compiler-name>
For further details visit Specs and Dependencies
Using Packages
You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.
spack location --install-dir <your package>
One way to use a package is to load it:
spack load <your package>
Although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use.
Managing Software Packages
When you install multiple software packages, Spack uses all the already installed packages to install the new one.
To see all your installed software packages with their version use
(This will show all the software packages with their installed dependency packages)
spack find
If you only want to see your directly installed packages (installed with “spack install <package-name>) use
spack find --explicit
Managing Modules
With spack install lmod
your are able to customize your spack modules. Unfortunately spack tend to change the syntax of the /.spack/modules.yaml
from time to time. Further details how to create your own flavor of modules.yaml can be found here: Module files
Uninstalling Software
Uninstalling software packages is as easy as installing them.
First you should find the package you want to uninstall.
Then uninstall the package name with the version by using
spack uninstall <package-name>@version
If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.
You can check the dependencies of a package yourself by
spack find --deps gcc <package-name>
Adding new Compilers
Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers spack has found on your system run
spack compilers
any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available use.
module avail
then you can load the module of the compiler you want to add to spack
module load
When you have loaded the module of the compiler you want to add to spack, use
spack compiler find
to add the compiler to spack
Adding the Intel compiler
On the GOETHE HLR you can use the latest Intel compiler, you can load it with
module load comp/intel/2020.0
To add the compiler to spack, just type
spack compiler findWhich should add the intel compiler to your spack compilers.
Finally you have to edit the “compilers.yaml” file located at ”~/.spack/linux/“ by adding the two rpaths to the added Intel compiler. This example shows the configuration for “intel@19.1.0.166” which was previously added to spack.
... extra_rpaths: - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64 - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/ ...So your “compilers.yaml” should have the entry
- compiler: spec: intel@19.1.0.166 paths: cc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icc cxx: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/icpc f77: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort fc: /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/bin/intel64/ifort flags: {} operating_system: scientific7 target: x86_64 modules: [] environment: {} extra_rpaths: - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64 - /cluster/intel/2020.0/compilers_and_libraries_2020.0.166/linux/compiler/lib/intel64_lin/
Update spack
In some cases you do not find with spack info <package>
what you are looking for. Than you might want to update spack.
Here are some steps how to do so.
update spack
spack -V # Your own version of spack 1. cd ~/spack # Change into your spack directory 2. git pull # If an error appears see next box 3. update your config.yaml # See top of the page
Following error appeared
git pull error
error: Your local changes to the following files would be overwritten by merge: etc/spack/defaults/config.yaml Please, commit your changes or stash them before you can merge. Aborting # This error appears because you have changed the tmp-directory to # /scratch/<your-project-name>/<your-user-name>/spack/tmp, # as we advized (see section install), within the config.yaml.
git pull
process. Here you can find a whole discussion about it Link and how to solve it.
On possible way to go if you do not care about your old stuff
1. cd ~/spack # Previous step like above 2. git pull # Error appears 3. git stash save "" # New step to deal with the error 4. git pull # Redo it again 5. cd ~/spack/etc/spack/defaults/ 6. Edit your 'config.yaml' # See at the top how to alter
Now you should be able so see the lastest packages which spack provides.
Remove spack
If something went wrong and you want to start from the scratch or just remove spack, please delete both following directories.
remove spack
#Think twice before using this! rm -rf /your-home-directory/spack rm -rf /your-home-directory/.spack