Skip to content

Build

Jump to bottom
Andrey Prokopenko edited this page Sep 3, 2024 · 21 revisions

Dependencies

Dependency Version Required Runtime
Kokkos 4.3.00 ✔️ ✔️
MPI 2 ✔️
CMake 3.16 ✔️

Build and installation

The build instructions for the Kokkos library can be found here. ArborX requires Kokkos CMake build to have Kokkos_ENABLE_CUDA_LAMBDA=ON if Kokkos_ENABLE_CUDA=ON.

Example: OLCF Summit

To build Kokkos on Summit (POWER9 CPU and Nvidia V100 GPU) with three backends (Serial, OpenMP and CUDA), configure it with

OPTIONS=(
    -D CMAKE_INSTALL_PREFIX="${KOKKOS_INSTALL_DIR}"
    -D CMAKE_CXX_COMPILER="${KOKKOS_SOURCE_DIR}/bin/nvcc_wrapper"
    -D Kokkos_ENABLE_SERIAL=ON
    -D Kokkos_ENABLE_OPENMP=ON
    -D Kokkos_ENABLE_CUDA=ON
        -D Kokkos_ENABLE_CUDA_LAMBDA=ON
    -D Kokkos_ARCH_POWER9=ON
    -D Kokkos_ARCH_VOLTA70=ON
    )
cmake "${OPTIONS[@]}" "${KOKKOS_SOURCE_DIR:-../}"

Example: OLCF Spock

To build Kokkos on Spock (AMD EPYC 7622 CPU with AMD MI100 GPU) with two backends (OpenMP and HIP), configure it with

OPTIONS=(
    -D CMAKE_INSTALL_PREFIX="${KOKKOS_INSTALL_DIR}"
    -D CMAKE_CXX_COMPILER=hipcc
    -D Kokkos_ENABLE_OPENMP=ON
    -D Kokkos_ENABLE_HIP=ON
    -D Kokkos_ARCH_ZEN2=ON
    -D Kokkos_ARCH_VEGA908=ON
    )
cmake "${OPTIONS[@]}" "${KOKKOS_SOURCE_DIR:-../}"

Example: ALCF Arcticus

To build Kokkos on Arcticus (Intel Xeon Gold 6336Y CPU with Intel GPU) with two backends (OpenMP and SYCL), configure it with

OPTIONS=(
    -D CMAKE_INSTALL_PREFIX="${KOKKOS_INSTALL_DIR}"
    -D CMAKE_CXX_COMPILER=icpx
    -D Kokkos_CXX_STANDARD=17
    -D Kokkos_ENABLE_OPENMP=ON
    -D Kokkos_ENABLE_SYCL=ON
    -D Kokkos_ARCH_INTEL_XEHP=ON
    )
cmake "${OPTIONS[@]}" "${KOKKOS_SOURCE_DIR:-../}"

Example: NERSC Perlmutter

To build Kokkos on Perlmutter (AMD EPYC 7763 CPU with Nvidia A100 GPU) with two backends (OpenMP and CUDA), configure it with

OPTIONS=(
    -D CMAKE_INSTALL_PREFIX="${KOKKOS_INSTALL_DIR}"
    -D CMAKE_CXX_COMPILER="${KOKKOS_SOURCE_DIR}/bin/nvcc_wrapper"
    -D Kokkos_ENABLE_OPENMP=ON
    -D Kokkos_ENABLE_CUDA=ON
        -D Kokkos_ENABLE_CUDA_LAMBDA=ON
    -D Kokkos_ARCH_ZEN2=ON
    -D Kokkos_ARCH_AMPERE80=ON
    )
cmake "${OPTIONS[@]}" "${KOKKOS_SOURCE_DIR:-../}"

Note: the actual architecture is Zen3, however the compiler support at the moment is spotty. The current successful build uses gcc/9.3.0 with cuda/11.3.0.

CMake

Assuming that Kokkos library is installed in $KOKKOS_INSTALL_DIR, configure ArborX as

OPTIONS=(
    -D CMAKE_INSTALL_PREFIX="${ARBORX_INSTALL_DIR}"
    -D ARBORX_ENABLE_MPI=ON
    -D Kokkos_ROOT="${KOKKOS_INSTALL_DIR}"
    -D CMAKE_CXX_COMPILER="${KOKKOS_INSTALL_DIR}/bin/nvcc_wrapper" # only for CUDA-enabled Kokkos
    -D CMAKE_CXX_EXTENSIONS=OFF # required by Kokkos
    )
cmake "${OPTIONS[@]}" "${ARBORX_DIR:-../}"

then run make install. ArborX will then be installed in $ARBORX_INSTALL_DIR. If Kokkos was built without CUDA support, remove the CMAKE_CXX_COMPILER configuration option.

To disable MPI, configure with

    -D ARBORX_ENABLE_MPI=OFF

To enable ArborX examples, configure with

    -D ARBORX_ENABLE_EXAMPLES=ON

To validate the ArborX build, install Boost (versions 1.67, 1.68, or 1.75+) ($BOOST_INSTALL_DIR), configure with

    -D Kokkos_ROOT="${KOKKOS_INSTALL_DIR}"
    -D Boost_ROOT="${BOOST_INSTALL_DIR}"
    -D ARBORX_ENABLE_TESTS=ON

and run ctest after completing the build.

To enable ArborX benchmarks, install Google Benchmark version 1.5.4 or later ($BENCHMARK_INSTALL_DIR), Boost version 1.67 or later ($BOOST_INSTALL_DIR), configure with

    -D Kokkos_ROOT="${KOKKOS_INSTALL_DIR}"
    -D Boost_ROOT="${BOOST_INSTALL_DIR}"
    -D benchmark_ROOT="${BENCHMARK_INSTALL_DIR}"
    -D ARBORX_ENABLE_BENCHMARKS=ON

The individual benchmarks can then be run from benchmarks directory.

ArborX also supports building against Kokkos built as part of Trilinos. This requires Trilinos hash de15ca5352 or later. In this case, $KOKKOS_INSTALL_DIR should point to the Trilinos installation directory.

Spack

ArborX can also be installed using the Spack package manager. A basic installation can be done as:

spack install arborx

Spack allows options and compilers to be turned on in the install command:

spack install [email protected] %[email protected] ~mpi+cuda cuda_arch=70

This example illustrates the most common parameters:

  • Variants are specified with ~ or + (e.g. +cuda) and enable or disable certain options
  • Version (@version) immediately follows arborx and can specify a specific ArborX version
  • Compiler (%compiler) can be set if an exact compiler is desired; otherwise, a default compiler is chosen

Note: building with CUDA (+cuda) requires cuda_arch speicification. Building with ROCm (+rocm) requires amdgpu_target specification.

For a complete list of available ArborX options, run:

spack info arborx

Building against ArborX

For projects using CMake, add to CMakeLists.txt

find_package(ArborX REQUIRED)

For any targets requiring ArborX, add

target_link_libraries(<target> ArborX::ArborX)

Note that because of the way CMake deals with dependencies, users of ArborX will need to make sure that Kokkos installation directory is visible to CMake. The easiest way to address it is to add both to the configuration options:

    -D CMAKE_PREFIX_PATH="$ARBORX_INSTALL_DIR;$KOKKOS_INSTALL_DIR"
Clone this wiki locally