Use standardized vocabulary and specification for observability in Semantic Kernel

[TaoChenOSU]

Our goal is to make sure that SK provides the best developer experience while complying with the industry standards for observability in generative-AI-based applications.

ADR: https://github.com/microsoft/semantic-kernel/blob/main/docs/decisions/0044-OTel-semantic-convention.md

Tasks

Preview Give feedback

1. .Net: Add model diagnostics to non-streaming APIs #6150
   .NET kernel kernel.core +3
   
2. .Net: Update telemetry sample app and documentations #6174
   .NET sk team issue +2
   
3. .Net: Add model diagnostics to streaming APIs #6175
   .NET sk team issue +2
   
4. .Net: Wrap auto tool calls in activities #6176
   .NET sk team issue stale +3
   
5. .Net: Add tests to make sure we correctly and consistently produce the telemetry #6189
   .NET sk team issue stale +3
   
6. .Net: Add gen-ai metrics #6489
   .NET sk team issue +2
   

[TaoChenOSU]

This is an ongoing effort to align SK to the latest semantic conventions as best as we can. The GenAI semantic conventions are still under development and rapidly evolving. We will keep an eye on the following items:

1. Metrics
2. Vector database conventions
3. Conventions for multi-modality

[github-actions]

This issue is stale because it has been open for 90 days with no activity.

[markwallace-microsoft]

Closing as stale