[Example] Trace correlation between microservices using context

[etiennejournet]

Hello !

I'm a little bit lost on how to correlate traces between microservices using golang's context.

My use case is very simple: I have a backend and a frontend, the frontend sends request to the backend shipping the context with http.NewRequestWithContext. The objective is to correlate the two traces in Jaeger.

        tracer := otel.Tracer("hello-world-frontend")

        // Span Starts Now
        ctx, span := tracer.Start(ctx, "client")
        // Span Ends at the end of the function
        defer span.End()

        // Adding event to this spawn for follow up in the backend
        span.AddEvent("Launching Request to backend")
        req, _ := http.NewRequestWithContext(ctx, "GET", "http://127.0.0.1:3000", nil)
        httpclient := &http.Client{}
        resp, err = httpclient.Do(req)
        // etc...

The backend, receives the request and start a new span using the context from this request.

func handler(w http.ResponseWriter, r *http.Request) {
  // Span Starts Now
  span := trace.SpanFromContext(r.Context())

    // Span Ends at the end of the function
  defer span.End()

  code()
  span.AddEvent("Responding to api call")
}

I figured this would simply work, but it doesnt', I can retrieve traces for the two services separately, but can't correlate them.
The ecosystem is very rich in opentelemetry and there are a lot of libraries available, but I wasn't able to understand if I would need propagation or baggage (I still tested it with no results), or anything else.

Could you provide more examples or point me to the proper architecture documentation which will help me implement this ?

Thanks

[Aneurysm9]

For this to work you need the propagation API and an implementation of a Propagator to be used by the HTTP client and server at each end. The stdlib HTTP client doesn't serialize and ship a context.Context when you create a NewRequestWithContext(), it only makes it available to the http.Client and its associated RoundTripper. We provide the otelhttp.Transport and otelhttp.Handler types to assist with propagating trace context across HTTP requests. Its example is hopefully helpful, but your implementation requires very little change to incorporate them.

        tracer := otel.Tracer("hello-world-frontend")

        // Register a propagator
        otel.SetTextMapPropagator(propagation.TraceContext{})

        // Span Starts Now
        ctx, span := tracer.Start(ctx, "client")
        // Span Ends at the end of the function
        defer span.End()

        // Adding event to this spawn for follow up in the backend
        span.AddEvent("Launching Request to backend")
        req, _ := http.NewRequestWithContext(ctx, "GET", "http://127.0.0.1:3000", nil)
        // Include a tracing transport
        httpclient := &http.Client{Transport: otelhttp.NewTransport(http.DefaultTransport)}
        resp, err = httpclient.Do(req)
        // etc...

Then, when you register your HTTP handler you wrap it with otelhttp.Handler:

otelHandler := otelhttp.NewHandler(http.HandlerFunc(handler), "API call")

[etiennejournet]

Thank you for your answer, I was able to make it work quite easily with this.

I did try to implement some of the code that's in the example you linked.
But since it also shows baggage propagation, I was always unsure of what part of the code was needed for the baggage and what other part was used by the context propagation.