Stokhos: unit test compilation errors with OpenMP and intell/2023.2.0, "error: Tpetra_CrsMatrix_MP_Amesos2_basker_UnitTest is not a template"

[ndellingwood]

Bug Report

@trilinos/stokhos @etphipp

Description

Stokhos unit tests fail to compile in OpenMP builds with intel/2023.2.0 (using icpc, intel classic compiler):

/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_OneAPI2023_2_0_ICPC_OpenMP-4T/Trilinos/packages/stokhos/test/UnitTest/Stokhos_TpetraCrsMatrixMPVectorUnitTest_OpenMP.cpp(53): error: Tpetra_CrsMatrix_MP_Amesos2_basker_UnitTest is not a template
  CRSMATRIX_MP_VECTOR_TESTS_N( OpenMPWrapperNode )
  ^

/home/jenkins/blake-new/workspace/KokkosEco_Trilinos_Blake_OneAPI2023_2_0_ICPC_OpenMP-4T/Trilinos/packages/stokhos/test/UnitTest/Stokhos_TpetraCrsMatrixMPVectorUnitTest_OpenMP.cpp(53): error: invalid explicit instantiation declaration
  CRSMATRIX_MP_VECTOR_TESTS_N( OpenMPWrapperNode )
  ^

Steps to Reproduce

1. SHA1: develop branch, e.g. 1c621fa
2. Configure script: Blake testbed "all" queue

module purge
module load cmake intel-oneapi-compilers/2023.2.0 intel-oneapi-mkl/2023.2.0

# Configure Trilinos
export BLAS_LIBRARIES="-mkl;${MKLROOT}/lib/intel64/libmkl_intel_lp64.a;${MKLROOT}/lib/intel64/libmkl_intel_thread.a;${MKLROOT}/lib/intel64/libmkl_core.a"
export LAPACK_LIBRARIES=${BLAS_LIBRARIES}
cmake \
  -D CMAKE_INSTALL_PREFIX="${TRILINOS_INSTALL_DIR}" \
  -D CMAKE_CXX_COMPILER="`which icpc`" \
  -D CMAKE_C_COMPILER="`which icc`" \
  -D CMAKE_Fortran_COMPILER="`which ifort`" \
  -D CMAKE_CXX_FLAGS="-g -no-ip" \
  -D CMAKE_C_FLAGS="-g -no-ip" \
  -DTPL_ENABLE_MPI=OFF \
  -DTPL_ENABLE_BLAS:BOOL=ON \
  -DTPL_BLAS_LIBRARIES:PATH="${BLAS_LIBRARIES}" \
  -DTPL_LAPACK_LIBRARIES:PATH="${LAPACK_LIBRARIES}" \
  -DTPL_ENABLE_LAPACK:BOOL=ON \
  -DTrilinos_ENABLE_ALL_PACKAGES=OFF \
  -DTrilinos_ENABLE_ALL_OPTIONAL_PACKAGES=OFF \
  -DTrilinos_ENABLE_TESTS=ON \
  -DTrilinos_MUST_FIND_ALL_TPL_LIBS=TRUE \
  -DTrilinos_ENABLE_COMPLEX=ON \
  -DTrilinos_ENABLE_OpenMP=ON \
  -DTrilinos_ENABLE_Kokkos=ON \
  -D Kokkos_ENABLE_OPENMP=ON \
   -D Kokkos_ENABLE_TESTS=ON \
  -D Kokkos_ARCH_SKX=ON \
  -DTrilinos_ENABLE_KokkosKernels=ON \
   -D KokkosKernels_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Tpetra=ON \
   -D Tpetra_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Sacado=ON \
   -D Sacado_ENABLE_TESTS=ON \
  -DTrilinos_ENABLE_Stokhos=ON \
   -D Stokhos_ENABLE_TESTS=ON \
\
  -DTPL_ENABLE_Matio=OFF \
\
-DTpetraCore_MatrixMarket_Tpetra_Map_InOutTest_DISABLE=ON \
-DTpetraCore_Bug5800_DISABLE=ON \
-DTpetraCore_BlockCrsMatrix_DISABLE=ON \
\
$TRILINOS_DIR

[ndellingwood]

Looks related to the changes here https://github.com/trilinos/Trilinos/pull/12324/files#diff-605a695df56ef47855f137277b3b36f7170b840dd4a6e13e2f7452fe5f829261 , compiler doesn't like the call guarded in the #else block TEUCHOS_UNIT_TEST_TEMPLATE_4_DECL(Tpetra_CrsMatrix_MP, Amesos2_basker, Storage, LocalOrdinal, GlobalOrdinal, Node) {} ?

[maartenarnst]

Hi @ndellingwood and @etphipp.

The problem does seem to come from my recent PR that Eric cited. Sorry about that.

I suspect the issue comes from the preprocessor conditional #if defined(HAVE_STOKHOS_AMESOS2). I think I put the corresponding #endif too far down in the code. Hence, when compiling without Amesos2, the macros don't get defined at all, i.e. the empty class guard that Eric mentioned doesn't get defined either.

• Trilinos/packages/stokhos/test/UnitTest/Stokhos_TpetraCrsMatrixMPVectorUnitTest.hpp

  Line 2529 in e135b58

  #endif // defined(HAVE_STOKHOS_AMESOS2)

I hadn't noticed the issue because all our builds are with Amesos2. Note that this guard with an empty class mimics what's done higher in the file for Belos sovers.

I think it can be fixed by organising the conditional better. I'll make a PR.

EDIT: it's PR #12354.