Multi turn content safety sample (#161)

* update promptflow-eval dependencies to azure-ai-evaluation

* clear local variables

* fix errors and remove 'question' col from data

* small fix in evaluator config

* init

* updates

scenarios/evaluate/Simulators/Simulate_Context-Relevant_Data/Simulate_From_Azure_Search_Index/Simulate_From_Azure_Search_Index.ipynb

@@ -18,7 +18,6 @@
     "This tutorial uses the following Azure AI services:\n",
     "\n",
     "- Access to Azure OpenAI Service - you can apply for access [here](https://go.microsoft.com/fwlink/?linkid=2222006)\n",
-    "- An Azure AI Studio project - go to [aka.ms/azureaistudio](https://aka.ms/azureaistudio) to create a project\n",
     "- An Azure AI Search service - go to [aka.ms/azuresearch](https://aka.ms/azuresearch) to create a service "
    ]
   },
@@ -127,7 +126,6 @@
    "source": [
     "import os\n",
     "\n",
-    "os.environ[\"AZURE_OPENAI_ENDPOINT\"] = \"<your-endpoint>\"\n",
     "os.environ[\"AZURE_OPENAI_DEPLOYMENT\"] = \"<your-deployment>\"\n",
     "os.environ[\"AZURE_OPENAI_API_VERSION\"] = \"<api version>\""
    ]

scenarios/evaluate/Simulators/Simulate_Evaluate_ContentSafety/README.md

@@ -0,0 +1,55 @@
+---
+page_type: sample
+languages:
+- python
+products:
+- ai-services
+- azure-openai
+description: Simulate and Evaluate Content Safety Harms
+---
+
+# Simulate and Evaluate Content Safety Harms
+
+## Overview
+
+This notebook walks through how to generate a multi-turn simulated conversation targeting a deployed AzureOpenAI model and then evaluate that conversation dataset for Content Safety harms. 
+
+### Objective
+
+The main objective of this tutorial is to help users understand how to use the azure-ai-evaluation SDK to simulate a multi-turn conversation with an AI system and then evaluate that dataset on various safety metrics. By the end of this tutorial, you should be able to:
+
+- Use azure-ai-evaluation SDK to generate a multi-turn simulated conversation dataset
+- Evaluate the generated dataset for Content Safety harms
+
+### Basic requirements
+
+To use Azure AI Safety Evaluation for different scenarios(simulation, annotation, etc..), you need an **Azure AI Project.** You should provide Azure AI project to run your safety evaluations or simulations with. First[create an Azure AI hub](https://learn.microsoft.com/en-us/azure/ai-studio/concepts/ai-resources)then [create an Azure AI project](    https://learn.microsoft.com/en-us/azure/ai-studio/how-to/create-projects?tabs=ai-studio).You **do not** need to provide your own LLM deployment as the Azure AI Safety Evaluation servicehosts adversarial models for both simulation and evaluation of harmful content and connects to it via your Azure AI project. Ensure that your Azure AI project is in one of the supported regions for your desiredevaluation metric
+
+#### Region support for evaluations
+
+| Region | Hate and unfairness, sexual, violent, self-harm, XPIA | Groundedness | Protected material |
+| - | - | - | - |
+|UK South | Will be deprecated 12/1/24| no | no |
+|East US 2 | yes| yes | yes |
+|Sweden Central | yes| yes | no|
+|US North Central | yes| no | no |
+|France Central | yes| no | no |
+|SwitzerlandWest| yes | no |no|
+
+For built-in quality and performance metrics, connect your own deployment of LLMs and therefore youcan evaluate in any region your deployment is in.
+
+#### Region support for adversarial simulation
+
+| Region | Adversarial simulation |
+| - | - |
+|UK South | yes|
+|East US 2 | yes|
+|Sweden Central | yes|
+|US North Central | yes|
+|France Central | yes|
+
+### Programming Languages
+
+- Python
+
+### Estimated Runtime: 30 mins

scenarios/evaluate/Simulators/Simulate_Evaluate_ContentSafety/Simulate_Evaluate_ContentSafety.ipynb

@@ -0,0 +1,255 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Simulating and Evaluating Multiturn Conversations for Content Harms\n",
+    "\n",
+    "## Objective\n",
+    "\n",
+    "This notebook walks through how to generate a simulated multi-turn conversation targeting a deployed AzureOpenAI model and then evaluate that test dataset for Content Safety harms. \n",
+    "\n",
+    "## Time\n",
+    "You should expect to spend about 30 minutes running this notebook. If you increase or decrease the number of simulated conversations, the time will vary accordingly.\n",
+    "\n",
+    "## Before you begin\n",
+    "\n",
+    "### Installation\n",
+    "Install the following packages required to execute this notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install openai azure-ai-evaluation azure-identity promptflow-azure"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Set the following environment variables for use in this notebook:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "os.environ[\"AZURE_DEPLOYMENT_NAME\"] = \"\"\n",
+    "os.environ[\"AZURE_API_VERSION\"] = \"\"\n",
+    "os.environ[\"AZURE_SUBSCRIPTION_ID\"] = \"\"\n",
+    "os.environ[\"AZURE_RESOURCE_GROUP\"] = \"\"\n",
+    "os.environ[\"AZURE_PROJECT_NAME\"] = \"\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Configuration\n",
+    "The following simulator and evaluators require an Azure AI Studio project configuration and an Azure credential to use. \n",
+    "Your project configuration will be what is used to log your evaluation results in your project after the evaluation run is finished.\n",
+    "\n",
+    "For full region supportability, see [our documentation](https://learn.microsoft.com/azure/ai-studio/how-to/develop/flow-evaluate-sdk#built-in-evaluators)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from azure.identity import DefaultAzureCredential, get_bearer_token_provider\n",
+    "from azure.ai.evaluation import evaluate\n",
+    "from azure.ai.evaluation import ContentSafetyEvaluator\n",
+    "from azure.ai.evaluation.simulator import AdversarialSimulator, AdversarialScenario\n",
+    "from openai import AzureOpenAI\n",
+    "\n",
+    "\n",
+    "azure_ai_project = {\n",
+    "    \"subscription_id\": os.environ.get(\"AZURE_SUBSCRIPTION_ID\"),\n",
+    "    \"resource_group_name\": os.environ.get(\"AZURE_RESOURCE_GROUP\"),\n",
+    "    \"project_name\": os.environ.get(\"AZURE_PROJECT_NAME\"),\n",
+    "}\n",
+    "\n",
+    "credential = DefaultAzureCredential()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Run this example\n",
+    "\n",
+    "To keep this notebook lightweight, let's create a dummy application that calls an AzureOpenAI model, such as GPT 4. When we are testing your application for certain safety metrics like Content Safety, it's important to have a way to automate a basic style of red-teaming to elicit behaviors from a simulated malicious user. We will use the `Simulator` class and this is how we will generate a synthetic test dataset against your application. Once we have the test dataset, we can evaluate them with our `ContentSafetyEvaluator` class.\n",
+    "\n",
+    "The `Simulator` needs a structured contract with your application in order to simulate conversations or other types of interactions with it. This is achieved via a callback function. This is the function you would rewrite to actually format the response from your generative AI application."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from typing import List, Dict, Optional\n",
+    "\n",
+    "\n",
+    "async def content_safety_callback(\n",
+    "    messages: List[Dict], stream: bool = False, session_state: Optional[str] = None, context: Optional[Dict] = None\n",
+    ") -> dict:\n",
+    "    deployment = os.environ.get(\"AZURE_DEPLOYMENT_NAME\")\n",
+    "    endpoint = os.environ.get(\"AZURE_ENDPOINT\")\n",
+    "    token_provider = get_bearer_token_provider(DefaultAzureCredential(), \"https://cognitiveservices.azure.com/.default\")\n",
+    "    # Get a client handle for the model\n",
+    "    client = AzureOpenAI(\n",
+    "        azure_endpoint=endpoint,\n",
+    "        api_version=os.environ.get(\"AZURE_API_VERSION\"),\n",
+    "        azure_ad_token_provider=token_provider,\n",
+    "    )\n",
+    "    # Call the model\n",
+    "    try:\n",
+    "        completion = client.chat.completions.create(\n",
+    "            model=deployment,\n",
+    "            messages=[\n",
+    "                {\n",
+    "                    \"role\": \"user\",\n",
+    "                    \"content\": messages[\"messages\"][0][\"content\"],\n",
+    "                }\n",
+    "            ],\n",
+    "            max_tokens=800,\n",
+    "            temperature=0.7,\n",
+    "            top_p=0.95,\n",
+    "            frequency_penalty=0,\n",
+    "            presence_penalty=0,\n",
+    "            stop=None,\n",
+    "            stream=False,\n",
+    "        )\n",
+    "        formatted_response = completion.to_dict()[\"choices\"][0][\"message\"]\n",
+    "    except Exception:\n",
+    "        formatted_response = {\n",
+    "            \"content\": \"I don't know\",\n",
+    "            \"role\": \"assistant\",\n",
+    "            \"context\": {\"key\": {}},\n",
+    "        }\n",
+    "    messages[\"messages\"].append(formatted_response)\n",
+    "    return {\n",
+    "        \"messages\": messages[\"messages\"],\n",
+    "        \"stream\": stream,\n",
+    "        \"session_state\": session_state,\n",
+    "        \"context\": context,\n",
+    "    }"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Testing your application for Content Safety\n",
+    "\n",
+    "When building your application, you want to test that Content Safety harms (i.e. Hate and unfairness, Sexual, Violent, Self-harm) are not being generated by your generative AI applications. The following example uses an `AdversarialSimulator` paired with a conversation scenario to prompt your model to respond with material that contains content safety harms."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "content_safety_simulator = AdversarialSimulator(azure_ai_project=azure_ai_project, credential=credential)\n",
+    "\n",
+    "content_safety_scenario = AdversarialScenario.ADVERSARIAL_CONVERSATION"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Below we explicitly request that the conversation has multiple turns between the User and Assistant. "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "content_safety_outputs = await content_safety_simulator(\n",
+    "    scenario=content_safety_scenario,\n",
+    "    max_conversation_turns=5,  # define the number of conversation turns\n",
+    "    max_simulation_results=5,  # define the number of simulation results\n",
+    "    target=content_safety_callback,  # define the target model callback\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import json\n",
+    "from pathlib import Path\n",
+    "\n",
+    "with Path.open(\"adv_convo_eval.jsonl\", \"w\") as f:\n",
+    "    for output in content_safety_outputs:\n",
+    "        f.write(json.dumps({\"conversation\": output}))\n",
+    "        f.write(\"\\n\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now that we have our dataset, we can evaluate it for Content Safety harms. The `ContentSafetyEvaluator` class can take in the dataset and detect whether your data contains harmful content. Let's use the `evaluate()` API to run the evaluation and log it to our Azure AI Studio Project."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cs_eval = ContentSafetyEvaluator(azure_ai_project=azure_ai_project, credential=credential)\n",
+    "\n",
+    "result = evaluate(\n",
+    "    name=\"content-safety-conversation\",\n",
+    "    data=\"adv_convo_eval.jsonl\",\n",
+    "    evaluators={\"content_safety\": cs_eval},\n",
+    "    # Optionally provide your AI Studio project information to track your evaluation results in your Azure AI Studio project\n",
+    "    azure_ai_project=azure_ai_project,\n",
+    "    # Optionally provide an output path to dump a json of metric summary, row level data and metric and studio URL\n",
+    "    output_path=\"./content-safety-conversation_results.json\",\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": ".venv",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}