How to link a prebuilt device function in dpcpp?

[chlict]

Hi, apologize if this is not the correct place to ask. I want to link a dpcpp program with a prebuilt device function which is also written in C++. If I'm understanding correctly, I should annotate the external function with SYCL_EXTERNAL, and then call it from the kernal. Like this:

#include <CL/sycl.hpp>
using namespace cl::sycl;

SYCL_EXTERNAL int foo(int x);

int main() {
  queue Queue;
  Queue.submit([&](handler &cgh) {
    cgh.single_task<class Test>([=]() {
        foo(0);
    });
  });
}

Assume I have foo compiled in foo.o. So what command should I use to build the final executable?

In particular, do I need to use -fsycl-add-targets=mytarget:foo.o option? I notice it is deprecated in recent dpcpp code, what is its replacement?

Thanks.

[AGindinson]

Hi @chlict,

The interface for linking your code against an additional device-containing object is simpler - just pass your main.o/main.cpp and foo.o to the clang++ -fsycl command, and the compiler driver will handle the rest.

More detailed documentation on the subject can be found here:
Official oneAPI documentation & guides: https://software.intel.com/content/www/us/en/develop/documentation/oneapi-programming-guide/top/programming-interface/example-compilation.html
Developer-provided design description (advanced, may be overly thorough for mere usage purposes): https://github.com/intel/llvm/blob/sycl/sycl/doc/CompilerAndRuntimeDesign.md#sycl-support-in-the-driver

[chlict]

Hi AGindinson,

Thanks for your reply. Is it ok for foo.o being a pure native object file compiled from a regular C++ compiler (rather than a bundled llvm bitcode file compiled from sycl-clang++ -fscyl -c foo.cpp)?

[AGindinson]

For device-side usage of the foo() function, foo.cpp would have to be a SYCL object produced by the SYCL compiler. The linking stage of the SYCL app would treat the pure C++ object as a host input, and therefore the device-side foo call would end up unresolved. Meanwhile, the "full" SYCL compilation of the foo.cpp object would yield a so-called "fat object" foo.o with device and host objects bundled into it - for the SYCL linkage, this would signify that the device code for foo needs to be linked with the main application kernels.

[chlict]

So if I only have a foo.o compiled from a non-sycl compiler, what is the proper way to link against it? (assume ABI is compatible)

[AGindinson]

Unfortunately, I'm not sure if such a thing can be achieved. The code in the pure C++ object would be a binary for the host CPU target - while we could look to trick the SYCL compiler/runtime into treating that as a "CPU device" module compiled ahead of time, the ABI compatibility still feels like a non-realistic assumption.

@Fznamznon, @kbobrovs, could you please proofread/correct me on this?

[AlexeySachkov]

So if I only have a foo.o compiled from a non-sycl compiler, what is the proper way to link against it? (assume ABI is compatible)

Passing it as just one more input to clang++ -fsycl will work, but only the host part will be fully linked, because such foo.o doesn't provide definition of foo for device part of the app. It means that your app will be linked just fine, but you won't be able to launch the kernel, because JIT compilation will fail due to unresolved external symbol error.

I'm not exactly sure if we have a way to pass a device object file which needs to be linked to the rest of the device code in the app.

[kbobrovs]

I'm not exactly sure if we have a way to pass a device object file which needs to be linked to the rest of the device code in the app.

Right, this is a grey area. The standard way to use externally (not via DPC++)-provided functions is to use OpenCL interop (1.2.1) or BE-specific interfaces (SYCL2020, not specified yet).

But one can craft a fat object at his/her own risk from an existing thin host object and an .bc file (and a bit of luck :) ), then pass it to the host link command (which must be clang++ -fsycl, as it was previously mentioned). Running clang++ -fsycl -c -v when compiling some DPC++ sample will show all the necessary tool invocations - clang-offload-bundler in particular.

[kbobrovs]

In particular, do I need to use -fsycl-add-targets=mytarget:foo.o option? I notice it is deprecated in recent dpcpp code, what is its replacement?

-fsycl-add-targets served for a different purpose - it added fully linked device binary image into the host fat binary, not fat objects.

[chlict]

I see. Thanks.