Merge pull request #249 from adanalis/2024.06.rocprof_sdk

ROCP_SDK: Enabling agent profiling mode.

src/components/rocp_sdk/README.md

@@ -0,0 +1,58 @@
+# ROCP\_SDK Component
+
+The ROCP\_SDK component exposes numerous performance events on AMD GPUs and APUs.
+The component is an adapter to the ROCm profiling library ROCprofiler-SDK which is included in a standard ROCM release.
+
+* [Enabling the ROCP\_SDK Component](#enabling-the-rocm-component)
+* [Environment Variables](#environment-variables)
+* [Known Limitations](#known-limitations)
+* [FAQ](#faq)
+***
+## Enabling the ROCP\_SDK Component
+
+To enable reading ROCP\_SDK events the user needs to link against a PAPI library that was configured with the ROCP\_SDK component enabled. As an example the following command: `./configure --with-components="rocp_sdk"` is sufficient to enable the component.
+
+Typically, the utility `papi_component_avail` (available in `papi/src/utils/papi_component_avail`) will display the components available to the user, and whether they are disabled, and when they are disabled why.
+
+## Library Version Limitations
+AMD ROCprofiler-SDK released before rocm-6.3.2 has known bugs.
+
+## Environment Variables
+
+PAPI requires the location of the ROCM install directory. This can be
+specified by one environment variable: **PAPI\_ROCP\_SDK\_ROOT**.
+
+Access to the rocm main directory is required at both compile (for include
+files) and at runtime (for libraries).
+
+Example:
+
+    export PAPI_ROCP_SDK_ROOT=/opt/rocm
+
+Within PAPI\_ROCP\_SDK\_ROOT, we expect the following standard directories:
+
+    PAPI_ROCP_SDK_ROOT/include
+    PAPI_ROCP_SDK_ROOT/include/rocprofiler-sdk
+    PAPI_ROCP_SDK_ROOT/lib
+
+### Counter Collection Modes
+
+The default mode is device sampling, which allows counter collection during the execution of a kernel. If a PAPI user wants to use dispatch mode, they must set the environment variable: **PAPI\_ROCP\_SDK\_DISPATCH\_MODE** before initializing PAPI.
+
+Example:
+
+    export PAPI_ROCP_SDK_DISPATCH_MODE=1
+
+### Unusual Installations
+
+For the ROCP\_SDK component to be operational, it must find the dynamic library `librocprofiler-sdk.so` at runtime. This is normally found in the standard directory structure mentioned above. For unusual installations that do not follow this structure, the user may provide the full path to the library using the environment variable: **PAPI\_ROCP\_SDK\_LIB**.
+
+Example:
+
+    export PAPI_ROCP_SDK_LIB=/opt/rocm-6.3.2/lib/librocprofiler-sdk.so.0
+
+Note that this variable takes precedence over PAPI\_ROCP\_SDK\_ROOT.
+
+## Known Limitations
+
+* In dispatch mode, PAPI may read zeros if reading takes place immediately after the return of a GPU kernel. This is not a PAPI bug. It may occur because calls such as hipDeviceSynchronize() do not guarantee that ROCprofiler has been called and all counter buffers have been flushed.  Therefore, it is recommended that the user code adds a delay between the return of a kernel and calls to PAPI_read(), PAPI_stop(), etc.

src/components/rocp_sdk/rocp_sdk.c

@@ -1,3 +1,18 @@
+/**
+ * @file    rocp_sdk.c
+ * @author  Anthony Danalis
+ *          [email protected]
+ *
+ * @ingroup papi_components
+ *
+ * @brief This implements a PAPI component that accesses hardware
+ *  monitoring counters for AMD GPU and APU devices through the
+ *  ROCprofiler-SDK library.
+ *
+ * The open source software license for PAPI conforms to the BSD
+ * License template.
+ */
+
 #include <stdio.h>
 #include <string.h>
 #include "papi.h"
@@ -57,7 +72,7 @@ papi_vector_t _rocp_sdk_vector = {
         .name = "rocp_sdk",
         .short_name = "rocp_sdk",
         .version = "1.0",
-        .description = "GPU events and metrics via AMD ROCprofiler-SDK API",
+        .description = "GPU events and metrics via AMD ROCprofiler-SDK",
         .initialized = 0,
         .num_mpx_cntrs = ROCPROF_SDK_MAX_COUNTERS,
     },