Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| public:usage:spack [2020/05/04 12:40] – [Managing Modules] keiling | public:usage:spack [2024/03/04 11:40] (current) – [Basic Spack commands] keiling | ||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Spack ====== | + | ====== Spack v0.22 ====== |
| - | ===== Getting Started | + | |
| - | [[https:// | + | <note warning> |
| - | To get started, [[doku: | + | <note tip>Your are here for the first time? Jump to overview. |
| - | < | + | You need to remove Spack? Jump to the bottom, follow the steps and then jump back to install Spack.</note> |
| - | This will create a folder named " | ||
| - | <note important> | + | ===== Overview - main goals ===== |
| - | /scratch is only for temporary files. See here [[public: | + | - Install |
| + | - Access Spack anytime (aka sourcing - see below) | ||
| + | - Change '' | ||
| + | - Tell Spack what software | ||
| + | | ||
| + | - Basic Spack commands | ||
| + | - A warning | ||
| + | - Maintenance | ||
| + | - Which version do I use? | ||
| + | - Update Spack | ||
| + | - Remove Spack | ||
| - | To add the Spack commands to your shell use. | ||
| - | < | + | ===== Install Spack ===== |
| - | <fc # | + | |
| - | If you want this to be permanent so that you can use the spack command after each login without executing the script first. You can add the execution | + | [[https:// |
| - | <code bash|Insert | + | Run the following |
| - | ... | + | |
| - | SPACK_ROOT="/ | + | <code|Download Spack> |
| - | export PATH=$SPACK_ROOT/bin:$PATH | + | git clone -c feature.manyFiles=true https://github.com/spack/spack.git |
| - | . $SPACK_ROOT/share/spack/setup-env.sh | + | |
| </ | </ | ||
| + | | ||
| + | This will create a folder named '' | ||
| - | <note warning> | + | ===== Access Spack anytime ===== |
| - | First open your '' | + | Now source the appropriate script for your shell. Most users on the cluster use the [[https:// |
| - | < | + | |
| - | - $tempdir | + | < |
| - | - $spack/var/spack/stage | + | # Sourcing these files will put the '' |
| + | # set up your '' | ||
| + | # useful shell integration for certain commands, environments and | ||
| + | # modules. For '' | ||
| + | |||
| + | # For bash/ | ||
| + | |||
| + | . ~/spack/share/spack/setup-env.sh | ||
| + | |||
| + | # For tcsh/csh | ||
| + | |||
| + | source ~/spack/share/spack/setup-env.csh | ||
| </ | </ | ||
| + | |||
| + | |||
| + | ===== Change build_stage directory from /tmp to /scratch ===== | ||
| + | To get started, [[goethe# | ||
| + | |||
| + | < | ||
| + | # This is really important. | ||
| + | |||
| + | mkdir -p / | ||
| + | </ | ||
| + | | ||
| + | <note important> | ||
| + | <note warning> | ||
| + | |||
| + | The following steps show you what needs to be changed: | ||
| + | < | ||
| + | # For instance you can use the following command to alter config.yaml | ||
| + | |||
| + | nano ~/ | ||
| + | </ | ||
| + | |||
| + | < | ||
| + | ... (stuff above) ... | ||
| + | | ||
| + | build_stage: | ||
| + | - $tempdir/ | ||
| + | - $user_cache_path/ | ||
| + | # - $spack/ | ||
| + | | ||
| + | ... (stuff below) ... | ||
| + | </ | ||
| to | to | ||
| - | <code> build_stage: | + | |
| - | # - $tempdir | + | <file|~/ |
| - | # - $spack/var/spack/stage | + | ... (stuff above) ... |
| - | - / | + | |
| + | | ||
| + | # This path must exist. Please create it if it's not already done. | ||
| + | # For instance go to the folder / | ||
| + | # and type '' | ||
| + | # another terminal where you altering the config.yaml to avoid spelling errors. | ||
| + | | ||
| + | |||
| + | ...(stuff below)... | ||
| + | </ | ||
| + | |||
| + | ===== Tell Spack what software is available on the cluster ===== | ||
| + | |||
| + | By default, Spack does not care about software that is already available on the system. You can tell Spack to locate it (please note that not everything can be found using this method, you may need to instruct Spack explicitly on what to find, as illustrated in Example 2). | ||
| + | |||
| + | **Example 1**: | ||
| + | |||
| + | < | ||
| + | # | ||
| + | # This will create .spack/ | ||
| + | |||
| + | [< | ||
| + | ==> The following specs have been detected on this system and added to / | ||
| + | autoconf@2.69 | ||
| + | automake@1.16.2 | ||
| + | binutils@2.35.2 | ||
| + | ==> The following specs have been detected on this system and added to / | ||
| + | slurm@23.02.6 | ||
| + | |||
| + | [< | ||
| + | ==> Added 1 new compiler to / | ||
| + | gcc@11.3.1 | ||
| + | ==> Compilers are defined in the following files: | ||
| + | / | ||
| </ | </ | ||
| - | < | + | **Example 2**: |
| + | |||
| + | < | ||
| + | [< | ||
| + | [< | ||
| + | ==> The following specs have been detected on this system and added to / | ||
| + | -- no arch / gcc@11.3.1 ----------------------------------------- | ||
| + | openmpi@5.0.0 | ||
| + | [< | ||
| + | </ | ||
| + | |||
| + | Spack will use the external packages when building new software instead of building all dependencies from scratch. On the other hand, it is not always desirable to use an external package. Pre-installed software (e.g. Perl) might lack needed features and using it as a dependency will fail. Then it's better to let Spack build its own version. | ||
| + | ===== Connecting two worlds ===== | ||
| + | |||
| + | < | ||
| + | |||
| + | It is important to distinguish between the cluster modules (see [[public: | ||
| + | |||
| + | < | ||
| + | [< | ||
| + | --------------------- | ||
| + | intel/ | ||
| + | intel/ | ||
| + | mpi/ | ||
| + | mpi/ | ||
| + | nvidia/ | ||
| + | |||
| + | ---------------- / | ||
| + | dot module-git | ||
| + | |||
| + | -------------------- / | ||
| + | mpi/ | ||
| + | |||
| + | Key: | ||
| + | modulepath | ||
| + | </ | ||
| + | |||
| + | Your Spack packages can be displayed with '' | ||
| + | |||
| + | <code|Connecting two worlds> | ||
| + | # In this example Python version 3.11.6 will be installed | ||
| + | |||
| + | spack info python | ||
| + | spack install python@3.11.6 | ||
| + | |||
| + | ...after a while... | ||
| + | |||
| + | ==> python: Successfully installed python-3.11.6-2zpso6kw573oyfw5cdqt5lq5sbsazv73 | ||
| + | Stage: 2.49s. | ||
| + | [+] /home/< | ||
| + | |||
| + | # Here you can learn some interesting details. | ||
| + | # Spack knows that we have AlmaLinux on a Skylake compute node with AVX-512 instruction set, | ||
| + | # also Spack uses our default compiler gcc-11.3.1. | ||
| + | # It follows the desired python-3.11.6 version with a unique identifier | ||
| + | |||
| + | # Why is this pointed out here? Please see section "A warning" | ||
| + | |||
| + | # Show me what I already installed with Spack: | ||
| + | |||
| + | [< | ||
| + | -- linux-almalinux9-skylake_avx512 / gcc@11.3.1 ----------------- | ||
| + | berkeley-db@18.1.40 | ||
| + | bzip2@1.0.8 | ||
| + | ca-certificates-mozilla@2023-05-30 | ||
| + | diffutils@3.7 | ||
| + | ==> 25 installed packages | ||
| + | |||
| + | # Refresh my modules to integrate them into the ' | ||
| + | |||
| + | [< | ||
| + | ==> You are about to regenerate tcl module files for: | ||
| + | |||
| + | -- linux-almalinux9-skylake_avx512 / gcc@11.3.1 ----------------- | ||
| + | gndqrbp berkeley-db@18.1.40 | ||
| + | plgyf6q bzip2@1.0.8 | ||
| + | yuvg6t3 ca-certificates-mozilla@2023-05-30 | ||
| + | o4iuzop diffutils@3.7 | ||
| + | q5lazqy expat@2.5.0 | ||
| + | |||
| + | ==> Do you want to proceed? [y/n] y | ||
| + | |||
| + | # Important, refresh your spack environment or logout and login again | ||
| + | |||
| + | . spack/ | ||
| + | |||
| + | # Finally type | ||
| + | |||
| + | module avail | ||
| + | |||
| + | [< | ||
| + | ------------------------------------------ / | ||
| + | python/ | ||
| + | |||
| + | ---------------------------------------------------------------------- / | ||
| + | intel/ | ||
| + | intel/ | ||
| + | |||
| + | ----------------------------------------------------------------- / | ||
| + | dot module-git | ||
| + | |||
| + | --------------------------------------------------------------------- / | ||
| + | mpi/ | ||
| + | |||
| + | Key: | ||
| + | modulepath | ||
| + | </ | ||
| + | |||
| + | <note tip>Every time when you install a new Spack module it is necessary to redo these two steps.\\ | ||
| + | '' | ||
| + | '' | ||
| + | </ | ||
| + | |||
| + | As mentioned, this is only an example that can be set up quickly. If you wish further adjustments of your own modules, for example different name schemes, please look here: [[https:// | ||
| - | Now you are ready to install your own software and use it in your projects! | ||
| + | ===== Basic Spack commands ===== | ||
| - | ===== Installing Software ===== | ||
| With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version. | With Spack it is easy to install the software needed for you project. But before you try install it, you should make sure Spack has the packages you need and with the correct version. | ||
| Line 55: | Line 250: | ||
| < | < | ||
| - | If you need to know if spack has a specific version of your package, use | + | If you need to know if Spack has a specific version of your package, use |
| < | < | ||
| Line 69: | Line 264: | ||
| If you also need it with a specific compiler, append | If you also need it with a specific compiler, append | ||
| < | < | ||
| - | |||
| For further details visit [[https:// | For further details visit [[https:// | ||
| + | ===== Using packages ===== | ||
| + | |||
| + | You can use your installed packages in different ways. In many cases it is enough to know where they are installed so you could run a binary. To get the installation location of a package you can use the following command. | ||
| + | < | ||
| + | |||
| + | One way to use a package is to load it: | ||
| + | |||
| + | < | ||
| - | ===== Managing | + | Although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use. |
| + | ===== Managing | ||
| When you install multiple software packages, Spack uses all the already installed packages to install the new one. | When you install multiple software packages, Spack uses all the already installed packages to install the new one. | ||
| Line 84: | Line 287: | ||
| - | If you only want to see your directly installed packages (installed with "spack install < | + | If you only want to see your directly installed packages (installed with '' |
| < | < | ||
| - | ===== Managing Modules ===== | + | List loaded |
| - | If you install many software | + | |
| - | To clean things up, you can install **lmod** with | + | |
| - | < | + | |
| - | After that you have to create a " | + | |
| - | < | + | ===== Uninstalling |
| - | + | ||
| - | There you can specify what modules will be listed by the " | + | |
| - | + | ||
| - | Your " | + | |
| - | < | + | |
| - | tcl: | + | |
| - | # No hashs at the end of the module name | + | |
| - | hash_length: | + | |
| - | # For example openmpi/ | + | |
| - | naming_scheme: | + | |
| - | blacklist_implicits: | + | |
| - | </ | + | |
| - | ++++old nameing scheme syntax| | + | |
| - | Spack changed the nameing scheme syntax for the module.yaml file. | + | |
| - | This example shows the **old** syntax. | + | |
| - | < | + | |
| - | modules: | + | |
| - | tcl: | + | |
| - | # No hashs at the end of the module name | + | |
| - | hash_length: | + | |
| - | # For example openmpi/ | + | |
| - | naming_scheme: | + | |
| - | blacklist_implicits: | + | |
| - | </ | + | |
| - | ++++ | + | |
| - | Further details how to create your own flavor of modules.yaml files can be found here: [[https:// | + | |
| - | + | ||
| - | The last step is to apply your " | + | |
| - | < | + | |
| - | ===== Uninstalling | + | |
| Uninstalling software packages is as easy as installing them. | Uninstalling software packages is as easy as installing them. | ||
| - | First you should [[doku: | + | First you should [[#Managing Software |
| Then uninstall the **package name with the version** | Then uninstall the **package name with the version** | ||
| Line 137: | Line 307: | ||
| < | < | ||
| - | ===== Adding new Compilers | + | ===== Adding new compilers |
| - | Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers | + | Spack has the ability to build packages with multiple compilers and compiler versions. To see which compilers |
| < | < | ||
| - | any of these compilers can be used to build spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available | + | Any of these compilers can be used to build Spack packages. If you want to use a compiler which is not in this list you have to add it to your PATH first in order for Spack to find it. You can add a new Compiler to your PATH by loading the module of the compiler. To see which modules are available |
| < | < | ||
| - | then you can load the module of the compiler you want to add to spack | + | Then you can load the module of the compiler you want to add to Spack: |
| < | < | ||
| - | When you have loaded | + | When you have loaded |
| < | < | ||
| - | to add the compiler to spack | + | to add the compiler to Spack. |
| + | ++++Adding the Intel compiler (deprecated)| | ||
| ===== Adding the Intel compiler ===== | ===== Adding the Intel compiler ===== | ||
| Line 175: | Line 346: | ||
| < | < | ||
| - compiler: | - compiler: | ||
| + | spec: intel@19.1.0.166 | ||
| + | paths: | ||
| + | cc: / | ||
| + | cxx: / | ||
| + | f77: / | ||
| + | fc: / | ||
| + | flags: {} | ||
| + | operating_system: | ||
| + | target: x86_64 | ||
| + | modules: [] | ||
| environment: | environment: | ||
| extra_rpaths: | extra_rpaths: | ||
| - / | - / | ||
| - / | - / | ||
| - | flags: {} | ||
| - | modules: [] | ||
| - | operating_system: | ||
| - | paths: | ||
| - | cc: / | ||
| - | cxx: / | ||
| - | f77: / | ||
| - | fc: / | ||
| - | spec: intel@19.0.1.144 | ||
| - | target: x86_64 | ||
| </ | </ | ||
| + | ++++ | ||
| - | ===== Using Packages | + | ===== A warning |
| - | You can use your installed | + | Spack packages |
| - | < | + | ===== Maintenance ===== |
| - | an other way to use a package is to load it | + | ==== Which version do I use? ==== |
| - | < | + | <code|Spack version - Update required?> |
| + | # Spack does not update itself. After a while you might see only old versions with 'spack info <package>' | ||
| + | # Use 'spack -V' to check your version. | ||
| + | # Example: | ||
| + | |||
| + | [user@login01 ~]$ spack -V | ||
| + | 0.22.0.dev0 | ||
| - | although a simple load is very useful in some cases it comes with some drawbacks, if there are more then one version of the package you want to load the load will fail, unless you specify exactly which of the packages you want to use. For more information see [[https:// | + | </code> |
| - | ===== Remove | + | ==== Update Spack ==== |
| - | If something went wrong and you want to start from the scratch or just remove | + | In some cases you do not find with '' |
| - | < | + | Here are some steps how to do so. |
| - | #Think twice before | + | |
| + | < | ||
| + | spack -V # Your own version of Spack | ||
| + | |||
| + | 1. cd ~/ | ||
| + | 2. git pull # If an error appears see next box | ||
| + | 3. cd .. # Go one folder back | ||
| + | 4. . spack/ | ||
| + | </ | ||
| + | |||
| + | ++++Some hints to errors due the update prozess| | ||
| + | |||
| + | Following error appeared < | ||
| + | error: Your local changes to the following files would be overwritten by merge: | ||
| + | etc/ | ||
| + | Please, commit your changes or stash them before you can merge. | ||
| + | Aborting | ||
| + | |||
| + | # This error appears because you have changed the tmp-directory to | ||
| + | # / | ||
| + | # as we advised (see section install), within the config.yaml. | ||
| + | </ | ||
| + | |||
| + | < | ||
| + | 1. cd ~/ | ||
| + | 2. git pull # Error appears | ||
| + | 3. git stash save "" | ||
| + | 4. git pull # Redo it again | ||
| + | 5. cd ~/ | ||
| + | 6. Edit your ' | ||
| + | </ | ||
| + | |||
| + | Now you should be able so see the latest packages which Spack provides. | ||
| + | ++++ | ||
| + | ==== Remove Spack ==== | ||
| + | |||
| + | If something went wrong and you want to start from the scratch or just remove | ||
| + | <code|Remove Spack> | ||
| + | # Think twice before | ||
| rm -rf / | rm -rf / | ||
| rm -rf / | rm -rf / | ||
| </ | </ | ||
| - | |||
