Spack v0.22

Before you begin, please check if you really need to build your own software packages, since a lot of software is already installed on the system. For example, do you require the GCC, Open MPI, HDF5, Python, CMake or R? They are readily available. Especially when considering building your own MPI, think twice.
Your are here for the first time? Jump to overview. You need to remove Spack? Jump to the bottom, follow the steps and then jump back to install Spack.

Overview - main goals

  1. Install Spack
  2. Access Spack anytime (aka sourcing - see below)
  3. Change build_stage directory from /tmp to /scratch
  4. Tell Spack what software is available on the cluster
  5. Connecting two worlds (environment modules with Spack)
  6. Basic Spack commands
  7. A warning
  8. Maintenance
    1. Which version do I use?
    2. Update Spack
    3. Remove Spack

Install Spack

Spack is a tool that makes it easy for you to configure and maintain your own software packages. Here we give a brief overview of the least necessary steps to get you going, but we advise to follow the Getting Started and Tutorial 101 of Spack. All of the following commands can be found at the Basic Usage chapter of Spack.

Run the following command on the command line (if you are reinstalling Spack, remove Spack first):

Download Spack

git clone -c feature.manyFiles=true https://github.com/spack/spack.git

This will create a folder named spack in your home directory and clone the contents of the Git repository into that newly created spack folder.

Access Spack anytime

Now source the appropriate script for your shell. Most users on the cluster use the Bash shell.

Sourcing

# Sourcing these files will put the ''spack'' command in your ''PATH'',
# set up your ''MODULEPATH'' to use Spack-generated modules, and add other
# useful shell integration for certain commands, environments and
# modules. For ''bash'' and ''zsh'', it also sets up tab completion.

# For bash/zsh/sh

. ~/spack/share/spack/setup-env.sh

# For tcsh/csh

source ~/spack/share/spack/setup-env.csh

Change build_stage directory from /tmp to /scratch

To get started, login to the system and go to your home directory. As the very first step, create a tempdir for Spack builds (you will need it later):

Create your temp directory on /scratch

# This is really important.

mkdir -p /scratch/<path to your scratch folder>/spack/tmp

We recommend installing Spack into your home directory. Use /scratch for temporary data. If you don't know, what /scratch is, please read here (or here for FUCHS).
By default Spack stores the temporary files under /tmp. This is shared with all users on the cluster and should not be used for your Spack installation. Spack files in /tmp will be deleted on a regular base within 30 minutes after creation. In order to change the directory Spack uses for its temporary files you have to set the new directory in the config.yaml. This can be done by the following commands.

The following steps show you what needs to be changed:

Access config.yaml

# For instance you can use the following command to alter config.yaml

nano ~/spack/etc/spack/defaults/config.yaml

~/spack/etc/spack/defaults/config.yaml (original file)

  ... (stuff above) ...
  
  build_stage:
    - $tempdir/$user/spack-stage
    - $user_cache_path/stage
    # - $spack/var/spack/stage
    
  ... (stuff below) ...   

to

~/spack/etc/spack/defaults/config.yaml (new file)

  ... (stuff above) ...
  
  build_stage:
    # This path must exist. Please create it if it's not already done.
    # For instance go to the folder /scratch/<your-project-name>/<your-user-name>/spack/tmp
    # and type ''pwd'' and copy it to
    # another terminal where you altering the config.yaml to avoid spelling errors.
    - /scratch/<your-project-name>/<your-user-name>/spack/tmp
  
  ...(stuff below)...

Tell Spack what software is available on the cluster

By default, Spack does not care about software that is already available on the system. You can tell Spack to locate it (please note that not everything can be found using this method, you may need to instruct Spack explicitly on what to find, as illustrated in Example 2).

Example 1:

Find externals and compilers

#
#  This will create .spack/packages.yaml

[<user>@login01 ~]$ spack external find && spack external find slurm
==> The following specs have been detected on this system and added to /home/<user>/.spack/packages.yaml
autoconf@2.69    bison@3.7.4     curl@7.76.1      flex@2.6.4  gmake@4.3      m4@1.4.19      pkgconf@1.7.3
automake@1.16.2  cmake@3.20.2    diffutils@3.7    gawk@5.1.0  groff@1.22.4   openssh@8.7p1  tar@1.34
binutils@2.35.2  coreutils@8.32  findutils@4.8.0  git@2.39.3  libtool@2.4.6  openssl@3.0.7  texinfo@6.7
==> The following specs have been detected on this system and added to /home/<user>/.spack/packages.yaml
slurm@23.02.6

[<user>@login01 ~]$ spack compiler find
==> Added 1 new compiler to /home/<user>/.spack/linux/compilers.yaml
  gcc@11.3.1
==> Compilers are defined in the following files:
  /home/<user>/.spack/linux/compilers.yaml

Example 2:

Add an already available Open MPI and build NetCDF against it

[<user>@login01 ~]$ module load mpi/openmpi/5.0.0
[<user>@login01 ~]$ spack external find openmpi
==> The following specs have been detected on this system and added to /home/<user>/.spack/packages.yaml
-- no arch / gcc@11.3.1 -----------------------------------------
openmpi@5.0.0
[<user>@login01 ~]$ spack install netcdf-c

Spack will use the external packages when building new software instead of building all dependencies from scratch. On the other hand, it is not always desirable to use an external package. Pre-installed software (e.g. Perl) might lack needed features and using it as a dependency will fail. Then it's better to let Spack build its own version.

Connecting two worlds

This section can be skipped if your are in a hurry, it just provides some convenience.

It is important to distinguish between the cluster modules (see Environment modules), your Spack packages, your private software and your private modules. You can install, list and load Spack packages using the spack command. Another way to use Spack packages are modules. If you don't need them or prefer not to use them, you can also skip this section. The cluster modules can be displayed with the module avail command, which looks like this:

Available cluster modules

[<user>@login02 ~]$ module avail
--------------------- /cluster/modulefiles ---------------------
intel/oneapi/2023.2.0  nvidia/hpc_sdk/nvhpc-byo-compiler/23.9  
intel/oneapi/latest    nvidia/hpc_sdk/nvhpc-hpcx-cuda12/23.9   
mpi/openmpi/4.1.6      nvidia/hpc_sdk/nvhpc-hpcx/23.9          
mpi/openmpi/5.0.0      nvidia/hpc_sdk/nvhpc-nompi/23.9         
nvidia/cuda/12.3.0     nvidia/hpc_sdk/nvhpc/23.9               

---------------- /usr/share/Modules/modulefiles ----------------
dot  module-git  module-info  modules  null  use.own  

-------------------- /usr/share/modulefiles --------------------
mpi/openmpi-x86_64  

Key:
modulepath  

Your Spack packages can be displayed with spack find. Now imagine you could combine module avail with your Spack packages. The following example illustrate how to:

Connecting two worlds

# In this example Python version 3.11.6 will be installed

spack info python              # Shows all available versions
spack install python@3.11.6    # Preferred version at the time of writing this

...after a while...

==> python: Successfully installed python-3.11.6-2zpso6kw573oyfw5cdqt5lq5sbsazv73
  Stage: 2.49s.  Configure: 44.94s.  Build: 21.50s.  Install: 16.89s.  Post-install: 22.83s.  Total: 1m 49.07s
[+] /home/<user>/spack/opt/spack/linux-almalinux9-skylake_avx512/gcc-11.3.1/python-3.11.6-2zpso6kw573oyfw5cdqt5lq5sbsazv73

# Here you can learn some interesting details.
# Spack knows that we have AlmaLinux on a Skylake compute node with AVX-512 instruction set,
# also Spack uses our default compiler gcc-11.3.1.
# It follows the desired python-3.11.6 version with a unique identifier

# Why is this pointed out here? Please see section "A warning".

# Show me what I already installed with Spack:

[<user>@login02 ~]$ spack find
-- linux-almalinux9-skylake_avx512 / gcc@11.3.1 -----------------
berkeley-db@18.1.40                 expat@2.5.0     libbsd@0.11.7  libxcrypt@4.4.35  perl@5.38.0    sqlite@3.43.2           zlib-ng@2.1.4
bzip2@1.0.8                         gdbm@1.23       libffi@3.4.4   libxml2@2.10.3    pkgconf@1.7.3  tar@1.34
ca-certificates-mozilla@2023-05-30  gettext@0.22.3  libiconv@1.17  ncurses@6.4       python@3.11.6  util-linux-uuid@2.38.1
diffutils@3.7                       gmake@4.3       libmd@1.0.4    openssl@3.1.3     readline@8.2   xz@5.4.1
==> 25 installed packages

# Refresh my modules to integrate them into the 'module avail' command.

[<user>@login02 ~]$ spack module tcl refresh
==> You are about to regenerate tcl module files for:

-- linux-almalinux9-skylake_avx512 / gcc@11.3.1 -----------------
gndqrbp berkeley-db@18.1.40                 d35sdln gdbm@1.23       uffpwpo libiconv@1.17     36lhb6f openssl@3.1.3  gsfue6h sqlite@3.43.2
plgyf6q bzip2@1.0.8                         oghfkbk gettext@0.22.3  53e2urm libmd@1.0.4       vthh2zs perl@5.38.0    7f6hcs7 tar@1.34
yuvg6t3 ca-certificates-mozilla@2023-05-30  krzlwmr gmake@4.3       hza4ixw libxcrypt@4.4.35  e4dbf5o pkgconf@1.7.3  l5vb4dw util-linux-uuid@2.38.1
o4iuzop diffutils@3.7                       siw2hgu libbsd@0.11.7   xzkdhcp libxml2@2.10.3    2zpso6k python@3.11.6  o2okcsd xz@5.4.1
q5lazqy expat@2.5.0                         ni4xv7l libffi@3.4.4    7cld3mz ncurses@6.4       46kn2s6 readline@8.2   nmtn65b zlib-ng@2.1.4

==> Do you want to proceed? [y/n] y

# Important, refresh your spack environment or logout and login again

. spack/share/spack/setup-env.sh

# Finally type

module avail

[<user>@login02 ~]$ module avail
------------------------------------------ /home/<user>/spack/share/spack/modules/linux-almalinux9-skylake_avx512 ------------------------------------------
python/3.11.6-gcc-11.3.1-2zpso6k

---------------------------------------------------------------------- /cluster/modulefiles -----------------------------------------------------------------------
intel/oneapi/2023.2.0  mpi/openmpi/4.1.6  nvidia/cuda/12.3.0                      nvidia/hpc_sdk/nvhpc-hpcx-cuda12/23.9  nvidia/hpc_sdk/nvhpc-nompi/23.9
intel/oneapi/latest    mpi/openmpi/5.0.0  nvidia/hpc_sdk/nvhpc-byo-compiler/23.9  nvidia/hpc_sdk/nvhpc-hpcx/23.9         nvidia/hpc_sdk/nvhpc/23.9

----------------------------------------------------------------- /usr/share/Modules/modulefiles ------------------------------------------------------------------
dot  module-git  module-info  modules  null  use.own

--------------------------------------------------------------------- /usr/share/modulefiles ----------------------------------------------------------------------
mpi/openmpi-x86_64

Key:
modulepath

Every time when you install a new Spack module it is necessary to redo these two steps.
spack module tcl refresh
. spack/share/spack/setup-env.sh

As mentioned, this is only an example that can be set up quickly. If you wish further adjustments of your own modules, for example different name schemes, please look here: Spack modules

Basic Spack commands

With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version.

To list all the available packages use

spack list

To check if Spack has a package you need you can either search through the list or use

spack list <my-package-name>

If you need to know if Spack has a specific version of your package, use

spack versions <package-name>

For further information on a package, e.g. dependencies, use

spack info <package-name>

Finally you can install your package with the latest version with

spack install <package-name>

If you need an older version you can use the “@” symbol appended to the package name and specify a version listed by “spack versions <package-name>”

spack install <package-name>@version

If you also need it with a specific compiler, append

spack install <package-name>@version %<compiler-name>

For further details visit Specs and Dependencies

Using packages

You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command.

spack location --install-dir <your package>

One way to use a package is to load it:

spack load <your package>

Although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use.

Managing software packages

When you install multiple software packages, Spack uses all the already installed packages to install the new one.

To see all your installed software packages with their version use

(This will show all the software packages with their installed dependency packages)

spack find

If you only want to see your directly installed packages (installed with spack install <package-name>) use

spack find --explicit

List loaded packages:

spack find --loaded

Uninstalling software

Uninstalling software packages is as easy as installing them.

First you should find the package you want to uninstall.

Then uninstall the package name with the version by using

spack uninstall <package-name>@version

If any other packages depend on it, Spack will print an error and list all the other packages that still depend on your package.

You can check the dependencies of a package yourself by

spack find --deps gcc <package-name>

Adding new compilers

Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers Spack has found on your system run

spack compilers

Any of these compilers can be used to build Spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for Spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available run

module avail

Then you can load the module of the compiler you want to add to Spack:

module load

When you have loaded a compiler module, run

spack compiler find

to add the compiler to Spack.

Adding the Intel compiler (deprecated)

A warning

Spack packages work on the machines on which they were built, but may not function on a different machine type1). If you need software packages for different nodes, e.g. GPU units, you need to compile them there. To accomplish this task we recommend to use SLURM's salloc command to work interactively on the desired node type. Please see here for further assistance: The salloc Command (or here for FUCHS).

Maintenance

Which version do I use?

Spack version - Update required?

# Spack does not update itself. After a while you might see only old versions with 'spack info <package>'.
# Use 'spack -V' to check your version.
# Example:
 
[user@login01 ~]$ spack -V
0.22.0.dev0  (d1fa23e9c69c15d7e29354a82c7814d6836e0fbd) # 2024/03/04

Update Spack

In some cases you do not find with spack info <package> what you are looking for. Then you might want to update Spack. Here are some steps how to do so.

Update Spack

spack -V                            # Your own version of Spack

1. cd ~/spack                       # Change into your Spack directory
2. git pull                         # If an error appears see next box
3. cd ..                            # Go one folder back
4. . spack/share/spack/setup-env.sh # Update your environment

Some hints to errors due the update prozess

Remove Spack

If something went wrong and you want to start from the scratch or just remove Spack, please delete both following directories.

Remove Spack

# Think twice before doing this! You need to remove two directories.
rm -rf /your-home-directory/spack
rm -rf /your-home-directory/.spack

1)
except a cross-compiler setting was used
public/usage/spack.txt · Last modified: 2024/03/04 11:40 by keiling
CC Attribution-Noncommercial-Share Alike 4.0 International
Driven by DokuWiki Recent changes RSS feed Valid CSS Valid XHTML 1.0