Python: Introduce the new function calling abstraction, FunctionChoiceBehavior #6910

moonbox3 · 2024-06-22T19:46:08Z

Motivation and Context

The current FunctionCallBehavior has allowed us to utilize auto function calling for OpenAI type models. As we proceed to support more AI connectors, that differ from OpenAI models, we need to be able to handle functions for all models that support function calling.

Description

In this PR:

We introduce a new function calling abstraction called FunctionChoiceBehavior which has three types: Auto, Required, and None.
One is able to configure the function_choice_behavior along with fully qualified function names (e.g., plugin1.function1), maximum_auto_invoke_attempts or auto_invoke_kernel_functions in yaml and JSON prompts. We have a new concept example showing how to do this for for yaml and JSON prompts.
- If the fully qualified names are specified in the config file, they take precedence over the filters, if specified at a later point.
To make sure this isn't a breaking change, we still handle the previous FunctionCallBehavior, but so we can make decisions on the new FunctionChoiceBehavior we map the FunctionCallBehavior to FunctionChoiceBehavior. Each time FunctionCallBehavior is updated, FunctionChoiceBehavior will be updated, too.
- The _process_tool_call() method in the open_ai_chat_completion_base needs to maintain the argument name as function_call_behavior as we cannot introduce a breaking change.
All concept samples have been converted to use FunctionChoiceBehavior.
New unit tests have been added for FunctionChoiceBehavior and we still currently support FunctionCallBehavior tests to make sure we haven't broken the backwards compatibility.
Added deprecated typing decorators to classes/methods to alert users that it would be best to transition to FunctionChoiceBehavior even though FunctionCallBehavior is still supported.
The FunctionCallingStepwisePlanner was updated to use the new FunctionChoiceBehavior.
Closes Python: Full support for abstracted function calling content #6496, Python: revisit when function call behavior is finalized, and other function calling models are added #6458
Addresses Python - Required Functions do not work correctly #6626

Contribution Checklist

The code builds clean without any errors or warnings
The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
All unit tests pass, and I have added new tests where possible
I didn't break anyone 😄

markwallace-microsoft · 2024-06-22T19:48:48Z

Python 3.10 Test Coverage Report •

File	Stmts	Miss	Cover	Missing
semantic_kernel
kernel.py	155	72	54%	142, 149–158, 196–204, 275–311, 339, 341–345, 361–405, 409–420
semantic_kernel/connectors/ai
function_calling_utils.py	11	3	73%	28–30
function_choice_behavior.py	93	4	96%	61, 121–128
prompt_execution_settings.py	50	1	98%	46
semantic_kernel/connectors/ai/open_ai/services
open_ai_chat_completion_base.py	168	70	58%	100, 104, 124, 149–153, 178, 186, 188, 192, 210–215, 233–261, 264–275, 290–297, 308–316, 332–339, 360, 368, 374–377, 389–392, 423
semantic_kernel/contents
function_call_content.py	82	7	91%	49, 56, 58, 69, 71, 73, 122
semantic_kernel/functions
kernel_function_from_prompt.py	162	7	96%	165–166, 187, 207, 232, 252, 335
semantic_kernel/planners/function_calling_stepwise_planner
function_calling_stepwise_planner.py	116	43	63%	133–223, 258–263
TOTAL	6668	764	89%

Python 3.10 Unit Test Overview

Tests	Skipped	Failures	Errors	Time
1567	1 💤	0 ❌	0 🔥	24.536s ⏱️

…in the function schema or not.

eavanvalkenburg

couple of comments, mostly small things, biggest one is wondering if we can simplify the function choice behavior class a bit further?

python/semantic_kernel/connectors/utils/function_call_format.py

python/tests/samples/test_concepts.py

..._kernel/connectors/ai/open_ai/prompt_execution_settings/open_ai_prompt_execution_settings.py

python/semantic_kernel/connectors/ai/function_choice_behavior.py

python/samples/concepts/plugins/plugin_with_param_not_included_in_function_schema.py

python/semantic_kernel/connectors/ai/function_choice_behavior.py

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

eavanvalkenburg

a bunch of comments...

python/semantic_kernel/connectors/ai/function_choice_behavior.py

python/semantic_kernel/functions/kernel_function_metadata.py

python/semantic_kernel/functions/kernel_parameter_metadata.py

...antic_kernel/planners/function_calling_stepwise_planner/function_calling_stepwise_planner.py

python/samples/concepts/auto_function_calling/function_calling_with_required_type.py

python/samples/concepts/auto_function_calling/functions_defined_in_json_prompt.py

…to invoke_function_call. Cleaned up other code/tests. Improved samples.

…ic-kernel into func_calling_abstraction

eavanvalkenburg

Some minor comments, overal looks great!

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py

python/semantic_kernel/kernel.py

…al for invoke_function_call.

…tegration tests for function choice behavior.

eavanvalkenburg

Love it, looks great!

…eBehavior (microsoft#6910) ### Motivation and Context The current `FunctionCallBehavior` has allowed us to utilize auto function calling for OpenAI type models. As we proceed to support more AI connectors, that differ from OpenAI models, we need to be able to handle functions for all models that support function calling.  ### Description In this PR: - We introduce a new function calling abstraction called `FunctionChoiceBehavior` which has three types: Auto, Required, and None. - One is able to configure the `function_choice_behavior` along with fully qualified function names (e.g., plugin1.function1), `maximum_auto_invoke_attempts` or `auto_invoke_kernel_functions` in yaml and JSON prompts. We have a new concept example showing how to do this for for yaml and JSON prompts. - If the fully qualified names are specified in the config file, they take precedence over the filters, if specified at a later point. - To make sure this isn't a breaking change, we still handle the previous `FunctionCallBehavior`, but so we can make decisions on the new `FunctionChoiceBehavior` we map the `FunctionCallBehavior` to `FunctionChoiceBehavior`. Each time `FunctionCallBehavior` is updated, `FunctionChoiceBehavior` will be updated, too. - The `_process_tool_call()` method in the `open_ai_chat_completion_base` needs to maintain the argument name as `function_call_behavior` as we cannot introduce a breaking change. - All concept samples have been converted to use `FunctionChoiceBehavior`. - New unit tests have been added for `FunctionChoiceBehavior` and we still currently support `FunctionCallBehavior` tests to make sure we haven't broken the backwards compatibility. - Added `deprecated` typing decorators to classes/methods to alert users that it would be best to transition to `FunctionChoiceBehavior` even though `FunctionCallBehavior` is still supported. - The `FunctionCallingStepwisePlanner` was updated to use the new `FunctionChoiceBehavior`. - Closes microsoft#6496, microsoft#6458 - Addresses microsoft#6626  ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄

moonbox3 added 19 commits June 4, 2024 14:48

wip for func calling abstractions

ae58089

wip func calling abstraction

cb9dd1d

Merge branch 'main' into func_calling_abstraction

ebf2882

Testing

3173b4a

Merge main to branch

f90aebb

Further work on func calling abstraction

1bf0596

Merge branch 'main' into func_calling_abstraction

9861d1c

Update comment on use of FunctionCallbehavior

08074b2

Merge branch 'main' into func_calling_abstraction

f93c5bb

Merge branch 'main' into func_calling_abstraction

24fe152

Further work on aligning to use Function Choice Behavior

a471aa6

Refining the abstractions

57c07fd

Improvements on function calling abstractions

f188f5d

Merge main to branch

8176f30

Update samples to use FunctionChoiceBehavior

fdd07d2

Fix concept example.

1363dd6

Update samples

4db9729

Improvements

2a88ab9

Fix typo

52c7849

moonbox3 requested a review from a team as a code owner June 22, 2024 19:46

markwallace-microsoft added the python Pull requests for the Python Semantic Kernel label Jun 22, 2024

This was linked to issues Jun 22, 2024

Python: revisit when function call behavior is finalized, and other function calling models are added #6458

Closed

Python - Required Functions do not work correctly #6626

Closed

Fix typos

4bf06d8

Allow a user to specify if a kernel function param is to be included …

c707d11

…in the function schema or not.

eavanvalkenburg reviewed Jun 24, 2024

View reviewed changes

moonbox3 requested a review from SergeyMenshykh June 24, 2024 15:03

moonbox3 mentioned this pull request Jun 24, 2024

Python: Filters for FunctionCallBehavior.EnableFunctions should be optional #6924

Closed

Address PR Feedback

2868b1a

alliscode approved these changes Jun 26, 2024

View reviewed changes

moonbox3 enabled auto-merge June 26, 2024 20:52

moonbox3 commented Jun 26, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Show resolved Hide resolved

moonbox3 disabled auto-merge June 26, 2024 23:38

moonbox3 and others added 2 commits June 26, 2024 20:24

Place into a common utils file. Update tests.

55d59bb

Merge branch 'main' into func_calling_abstraction

7cac93e

moonbox3 removed the request for review from SergeyMenshykh June 27, 2024 01:19

eavanvalkenburg reviewed Jun 27, 2024

View reviewed changes

moonbox3 added 6 commits June 27, 2024 12:36

Refactor where process_function_call lives to the kernel and renamed …

39148f1

…to invoke_function_call. Cleaned up other code/tests. Improved samples.

Add sample comment

06d936c

Merge branch 'main' into func_calling_abstraction

f3a308c

Merge branch 'func_calling_abstraction' of github.com:moonbox3/semant…

31e0861

…ic-kernel into func_calling_abstraction

Fix typos

abdf70c

Fix typos

780d309

eavanvalkenburg reviewed Jun 27, 2024

View reviewed changes

python/semantic_kernel/connectors/ai/open_ai/services/open_ai_chat_completion_base.py Outdated Show resolved Hide resolved

python/semantic_kernel/kernel.py Outdated Show resolved Hide resolved

python/semantic_kernel/kernel.py Outdated Show resolved Hide resolved

Make function_call_count, request_index, and function_behavior option…

fd3a759

…al for invoke_function_call.

eavanvalkenburg approved these changes Jun 27, 2024

View reviewed changes

Add ability to specify function choice behavior as string or . Add in…

5abe1ff

…tegration tests for function choice behavior.

eavanvalkenburg approved these changes Jun 27, 2024

View reviewed changes

moonbox3 enabled auto-merge June 27, 2024 20:20

moonbox3 added this pull request to the merge queue Jun 27, 2024

moonbox3 removed this pull request from the merge queue due to a manual request Jun 27, 2024

moonbox3 added this pull request to the merge queue Jun 27, 2024

Merged via the queue into microsoft:main with commit 8381e50 Jun 27, 2024
23 checks passed

moonbox3 deleted the func_calling_abstraction branch June 27, 2024 20:53

moonbox3 mentioned this pull request Jul 3, 2024

Python: Support Specifying Tool Call Behavior in YAML Prompts #6767

Closed

moonbox3 linked an issue Jul 3, 2024 that may be closed by this pull request

Python: Support Specifying Tool Call Behavior in YAML Prompts #6767

Closed

eavanvalkenburg mentioned this pull request Jul 5, 2024

Python: can enabling auto function calling become a one-liner? #6925

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python: Introduce the new function calling abstraction, FunctionChoiceBehavior #6910

Python: Introduce the new function calling abstraction, FunctionChoiceBehavior #6910

moonbox3 commented Jun 22, 2024 •

edited

Loading

markwallace-microsoft commented Jun 22, 2024 •

edited

Loading

eavanvalkenburg left a comment

eavanvalkenburg left a comment

eavanvalkenburg left a comment

eavanvalkenburg left a comment

Python: Introduce the new function calling abstraction, FunctionChoiceBehavior #6910

Python: Introduce the new function calling abstraction, FunctionChoiceBehavior #6910

Conversation

moonbox3 commented Jun 22, 2024 • edited Loading

Motivation and Context

Description

Contribution Checklist

markwallace-microsoft commented Jun 22, 2024 • edited Loading

Python 3.10 Unit Test Overview

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg left a comment

Choose a reason for hiding this comment

eavanvalkenburg left a comment

Choose a reason for hiding this comment

moonbox3 commented Jun 22, 2024 •

edited

Loading

markwallace-microsoft commented Jun 22, 2024 •

edited

Loading