Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tool call information missing in Model Diagnostics for Mistral streaming #6316

Closed
TaoChenOSU opened this issue May 17, 2024 · 3 comments
Closed
Assignees

Comments

@TaoChenOSU
Copy link
Contributor

Describe the bug
Tool call information in model diagnostics when using Mistral streaming.

Expected behavior
Tool call information should be presented in model diagnostics when using Mistral streaming.

@markwallace-microsoft
Copy link
Member

@TaoChenOSU can we close this issue?

@TaoChenOSU
Copy link
Contributor Author

Will do another round of testing.

@TaoChenOSU
Copy link
Contributor Author

Tool calls from MistralAI will be contained entirely in a single chunk returned from their streaming API, thus it doesn't require buffering during streaming and concatenation after streaming to reassemble the complete tool call. That being said, the current implementation of the MistralAI streaming chat completion service in .Net SK doesn't need any modification to have tool calls appear in the prompt span events.
Image
Note: we are not sure if MistralAI supports parallel function calling and if it does how will multiple tool calls get streamed back to the client.

@github-project-automation github-project-automation bot moved this to Sprint: Done in Semantic Kernel Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

2 participants