Python: Streaming content for token usage

[TaoChenOSU]

Motivation and Context

OpenAI recently starts providing token usage information on their streaming chat completion API. This ADR opens the discussion on how we should consume that information within our StreamingChatMessageContent data structure.

Description

1. An ADR documenting 4 approaches.
2. Implementation of the selected approach.
3. Fix issues where data in the streaming contents are modified after concatenation (__add__).
4. Add streaming to model diagnostics operation names so that we can distinguish streaming and non-streaming operations.

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[TaoChenOSU]

Proposal 3 and 4 came from an offline discussion. They do include a refactoring of the existing content classes, which is not necessary for handling streaming usage information from OAI. However, I do like those two proposals as they further clean up our code base. Out of the two, I think proposal 4 is cleaner.

[TaoChenOSU]

Upon further deliberation, I think that deprecating StreamingXXXContent may be premature at this time due to the following reasons:

1. Many AI services yield objects of varying types in their streaming APIs. Implementing the abstraction on our end could potentially lead to confusion.
2. The deprecation of StreamingXXXContent would result in breaking changes to our AI connectors.
3. Such deprecation would also cause a misalignment with the .Net SDK.

[markwallace-microsoft]

Python Test Coverage Report

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	199	47	76%	148, 159, 163, 313–316, 423, 437–480
semantic_kernel/agents/channels
open_ai_assistant_channel.py	49	1	98%	42
semantic_kernel/agents/group_chat
agent_chat.py	116	1	99%	91
agent_group_chat.py	67	2	97%	113, 135
broadcast_queue.py	72	1	99%	35
semantic_kernel/agents/open_ai
assistant_content_generation.py	68	2	97%	81–82
open_ai_assistant_base.py	358	3	99%	243, 321–322
semantic_kernel/connectors/ai
chat_completion_client_base.py	116	2	98%	382, 392
completion_usage.py	8	1	88%	17
semantic_kernel/connectors/ai/anthropic/services
anthropic_chat_completion.py	124	3	98%	120, 137, 173
semantic_kernel/connectors/ai/azure_ai_inference/services
azure_ai_inference_chat_completion.py	119	7	94%	120, 146–149, 159, 180, 202
azure_ai_inference_text_embedding.py	41	1	98%	87
semantic_kernel/connectors/ai/embeddings
embedding_generator_base.py	8	1	88%	50
semantic_kernel/connectors/ai/google
shared_utils.py	26	1	96%	56
semantic_kernel/connectors/ai/google/google_ai/services
google_ai_chat_completion.py	119	4	97%	127, 153, 176, 178
google_ai_text_completion.py	63	2	97%	98, 121
utils.py	65	3	95%	140, 160–165
semantic_kernel/connectors/ai/google/vertex_ai/services
utils.py	66	3	95%	141, 161–166
vertex_ai_chat_completion.py	119	4	97%	121, 147, 170, 172
vertex_ai_text_completion.py	62	2	97%	95, 116
semantic_kernel/connectors/ai/hugging_face/services
hf_text_completion.py	60	3	95%	103, 112, 127
hf_text_embedding.py	32	5	84%	79–83
semantic_kernel/connectors/ai/mistral_ai/services
mistral_ai_chat_completion.py	118	7	94%	118–121, 307–310
semantic_kernel/connectors/ai/ollama/services
ollama_chat_completion.py	60	5	92%	95–98, 108, 143
ollama_text_completion.py	55	5	91%	87–90, 100, 128
semantic_kernel/connectors/ai/open_ai/prompt_execution_settings
open_ai_prompt_execution_settings.py	94	1	99%	112
semantic_kernel/connectors/ai/open_ai/services
azure_chat_completion.py	107	5	95%	118, 123, 157, 166, 169
azure_text_completion.py	28	2	93%	82, 87
azure_text_embedding.py	30	2	93%	84, 89
open_ai_chat_completion_base.py	127	5	96%	71, 121, 141, 177, 287
open_ai_handler.py	63	3	95%	86, 95–96
open_ai_text_completion_base.py	80	2	98%	56, 161
semantic_kernel/connectors/ai/open_ai/settings
azure_open_ai_settings.py	22	4	82%	97–100
semantic_kernel/connectors/memory/azure_ai_search
azure_ai_search_collection.py	87	2	98%	150, 152
semantic_kernel/connectors/memory/redis
redis_collection.py	160	2	99%	146, 316
utils.py	45	11	76%	145–146, 164, 166, 173–188
semantic_kernel/connectors/openapi_plugin
openapi_manager.py	58	2	97%	110–111
openapi_parser.py	88	2	98%	71, 128
openapi_runner.py	105	2	98%	181–182
semantic_kernel/connectors/openapi_plugin/models
rest_api_operation.py	129	1	99%	242
semantic_kernel/contents
function_call_content.py	97	1	99%	201
streaming_content_mixin.py	38	2	95%	37, 63
semantic_kernel/core_plugins/sessions_python_tool
sessions_python_plugin.py	134	8	94%	69, 82–91, 99
sessions_python_settings.py	39	4	90%	84–87
semantic_kernel/data
vector_store_record_collection.py	249	19	92%	410, 470–474, 482–486, 526–529, 536–539
vector_store_record_utils.py	26	2	92%	50, 52
semantic_kernel/functions
kernel_function_decorator.py	98	1	99%	102
kernel_function_from_method.py	96	1	99%	153
kernel_function_from_prompt.py	154	7	95%	165–166, 180, 201, 219, 239, 322
kernel_function_log_messages.py	36	6	83%	37–43
kernel_plugin.py	187	2	99%	472, 475
semantic_kernel/planners
plan.py	234	45	81%	54, 163–165, 197, 214–227, 264, 269, 277–278, 288–291, 308, 313, 329, 332–337, 355, 360, 363, 365, 372, 386–388, 393–397
semantic_kernel/planners/function_calling_stepwise_planner
function_calling_stepwise_planner.py	116	4	97%	145, 189–190, 198
semantic_kernel/planners/sequential_planner
sequential_planner.py	64	6	91%	71, 75, 109, 125, 134–135
sequential_planner_extensions.py	50	9	82%	31–32, 56, 110–124
sequential_planner_parser.py	77	12	84%	66–74, 93, 117–120
semantic_kernel/schema
kernel_json_schema_builder.py	129	9	93%	53, 90, 186, 194, 205, 213, 228, 232–233
semantic_kernel/services
ai_service_client_base.py	22	1	95%	64
semantic_kernel/template_engine/blocks
code_block.py	77	1	99%	119
named_arg_block.py	43	1	98%	98
semantic_kernel/utils/authentication
entra_id_authentication.py	15	9	40%	25–38
semantic_kernel/utils/telemetry
user_agent.py	16	2	88%	18–19
semantic_kernel/utils/telemetry/model_diagnostics
decorators.py	171	4	98%	372–375
TOTAL	11070	318	97%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
2450	4 💤	0 ❌	0 🔥	57.659s ⏱️

[moonbox3]

LGTM