Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: openai/openai-realtime-console
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: main
Choose a base ref
...
head repository: qaware/openai-realtime-taxi-service-demo
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref
Can’t automatically merge. Don’t worry, you can still create the pull request.
  • 4 commits
  • 5 files changed
  • 1 contributor

Commits on Oct 19, 2024

  1. Create static.yml

    s-macke authored Oct 19, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    38d0609 View commit details
  2. Copy the full SHA
    3225e10 View commit details

Commits on Nov 2, 2024

  1. Copy the full SHA
    f798498 View commit details

Commits on Nov 3, 2024

  1. Update README.md

    s-macke authored Nov 3, 2024

    Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature.
    Copy the full SHA
    56987d2 View commit details
Showing with 156 additions and 58 deletions.
  1. +52 −0 .github/workflows/static.yml
  2. +5 −0 README.md
  3. +1 −0 package.json
  4. +47 −46 src/pages/ConsolePage.tsx
  5. +51 −12 src/utils/conversation_config.js
52 changes: 52 additions & 0 deletions .github/workflows/static.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Simple workflow for deploying static content to GitHub Pages
name: Deploy static content to Pages

on:
# Runs on pushes targeting the default branch
push:
branches: ["main"]

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# Sets permissions of the GITHUB_TOKEN to allow deployment to GitHub Pages
permissions:
contents: read
pages: write
id-token: write

# Allow only one concurrent deployment, skipping runs queued between the run in-progress and latest queued.
# However, do NOT cancel in-progress runs as we want to allow these production deployments to complete.
concurrency:
group: "pages"
cancel-in-progress: false

jobs:
# Single deploy job since we're just deploying
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Node
uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- run: npm run build

- name: Setup Pages
uses: actions/configure-pages@v5

- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: './build'

- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# Realtime Demo of a Taxi Service Call Center around Munich in Germany

### [Realtime Demo](https://blog.qaware.de/openai-realtime-taxi-service-demo/)
- OpenAI API Key required

# OpenAI Realtime Console

The OpenAI Realtime Console is intended as an inspector and interactive API reference
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
{
"name": "openai-realtime-console",
"version": "0.0.0",
"homepage": "./",
"type": "module",
"private": true,
"dependencies": {
93 changes: 47 additions & 46 deletions src/pages/ConsolePage.tsx
Original file line number Diff line number Diff line change
@@ -27,6 +27,10 @@ import { Map } from '../components/Map';
import './ConsolePage.scss';
import { isJsxOpeningLikeElement } from 'typescript';

function sleep(ms: number) {
return new Promise(resolve => setTimeout(resolve, ms));
}

/**
* Type for result from get_weather() function call
*/
@@ -376,6 +380,9 @@ export function ConsolePage() {
const wavStreamPlayer = wavStreamPlayerRef.current;
const client = clientRef.current;

// Set voice
// alloy, echo, fable, onyx, nova, and shimmer
client.updateSession({ voice: 'alloy' });
// Set instructions
client.updateSession({ instructions: instructions });
// Set transcription, otherwise we don't get user transcriptions back
@@ -384,74 +391,68 @@ export function ConsolePage() {
// Add tools
client.addTool(
{
name: 'set_memory',
description: 'Saves important data about the user into memory.',
name: 'reserviere_taxi',
description: 'Reserviert ein Taxi für den Anrufer.',
parameters: {
type: 'object',
properties: {
key: {
name: {
type: 'string',
description:
'The key of the memory value. Always use lowercase and underscores, no other characters.',
'Der Name des Kunden',
},
value: {
start_address: {
type: 'string',
description: 'Value can be anything represented as a string',
description:
'Die Startadresse der Taxifahrt',
},
},
required: ['key', 'value'],
finish_address: {
type: 'string',
description: 'Die Zieladresse der Taxifahrt',
},
persons: {
type: 'string',
description: 'Die Anzahl an Personen',
},
gepaeck: {
type: 'string',
description: 'Die Anzahl an Gepäckstücken',
}, },
required: ['start_address', 'finish_address', 'persons'],
},
},
async ({ key, value }: { [key: string]: any }) => {
setMemoryKv((memoryKv) => {
const newKv = { ...memoryKv };
newKv[key] = value;
return newKv;
});
return { ok: true };
async ({name, start_address, finish_address, persons, gepaeck }: { [key: string]: any }) => {
console.log("reserviere_taxi", name, start_address, finish_address, persons, gepaeck)
await sleep(2000);
return { ok: true, ankunftszeit: "30 Minuten", preis: "Etwa 25€" };
}
);

// Add tools
client.addTool(
{
name: 'get_weather',
description:
'Retrieves the weather for a given lat, lng coordinate pair. Specify a label for the location.',
name: 'fahrpreis_schaetzen',
description: 'Schaetzt den Fahrpreis.',
parameters: {
type: 'object',
properties: {
lat: {
type: 'number',
description: 'Latitude',
},
lng: {
type: 'number',
description: 'Longitude',
start_address: {
type: 'string',
description:
'Die Startadresse der Taxifahrt',
},
location: {
finish_address: {
type: 'string',
description: 'Name of the location',
description: 'Die Zieladresse der Taxifahrt',
},
},
required: ['lat', 'lng', 'location'],
required: ['start_address', 'finish_address'],
},
},
async ({ lat, lng, location }: { [key: string]: any }) => {
setMarker({ lat, lng, location });
setCoords({ lat, lng, location });
const result = await fetch(
`https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${lng}&current=temperature_2m,wind_speed_10m`
);
const json = await result.json();
const temperature = {
value: json.current.temperature_2m as number,
units: json.current_units.temperature_2m as string,
};
const wind_speed = {
value: json.current.wind_speed_10m as number,
units: json.current_units.wind_speed_10m as string,
};
setMarker({ lat, lng, location, temperature, wind_speed });
return json;
async ({ start_address, finish_address}: { [key: string]: any }) => {
console.log("fahrpreis_schaetzen", start_address, finish_address)
await sleep(2000);
return { ok: true, preis: "Etwa 35€" };
}
);

@@ -639,7 +640,7 @@ export function ConsolePage() {
(conversationItem.formatted.audio?.length
? '(awaiting transcript)'
: conversationItem.formatted.text ||
'(item sent)')}
'(item sent)')}
</div>
)}
{!conversationItem.formatted.tool &&
63 changes: 51 additions & 12 deletions src/utils/conversation_config.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,55 @@
export const instructions = `System settings:
Tool use: enabled.
Instructions:
- You are an artificial intelligence agent responsible for helping test realtime voice capabilities
- Please make sure to respond with a helpful voice via audio
- Be kind, helpful, and curteous
- It is okay to ask the user questions
- Use tools and functions you have available liberally, it is part of the training apparatus
- Be open to exploration and conversation
- Remember: this is just for fun and testing!
Personality:
- Be upbeat and genuine
- Try speaking quickly as if excited
Instruktionen:
- Du bist eine künstliche Intelligenz verantwortlich für den Telefondienst des Taxidienstes Isar
- Die Anrufer sind Kunden, die ein Taxi in der Stadt München benötigen oder die Fragen zu den Dienstleistungen des Unternehmens haben.
- Die Sprache ist Deutsch
- Du beginnst das Gespräch mit "Hier Taxidienst Isar, wie kann ich Ihnen helfen?"
- Bitte antworte in Deutsch mit einer hilfreichen Stimme über Audio
- Es ist in Ordnung, den Benutzer Fragen zu stellen
- Verwende die verfügbaren Tools und Funktionen großzügig
- Wiederhole die Anweisungen des Anrufers und bestätige, dass du sie verstanden hast
- Halte Dich an die Inforationen, die Dir zur Verfügung stehen
Unternehmen:
- Taxidienst Isar
- Vorstand: Max Mustermann
- Standort: München
- Adresse: Isarstraße 1, 80331 München
- Telefonnummer: 089 123456
- Öffnungszeiten: 24/7
- Webseite: www.taxi-isar.de
- E-Mail: kundenservice@tax-isar.de
Offizieller Taxitarif:
- Mindestfahrenpreis: 5,70 €
- Kilometerpreis: 2,50 €
- Wartezeit: 30 € pro Stunde
- Nachttarif (20:00 - 06:00 Uhr): 20% Aufschlag
- Feiertagszuschlag: 20% Aufschlag
Fahrpreis Schätzen:
- Frage nach der Abholadresse
- Frage nach der Zieladresse
- Benutze das Tool um den Fahrpreis zu schätzen
- Gib den geschätzten Fahrpreis an
Buchung eines Taxis:
- Frage nach dem Namen
- Frage nach der Abholadresse
- Frage nach der Zieladresse
- Frage nach der Anzahl der Personen
- Frage nach der Anzahl an Kindersitzen, die benötigt werden
- Frage nach Gepäck
- Vor der Bestellung wiederhole die Informationen und Frage nach einer Bestätigung der Buchung
- Benutze das Tool um ein Taxi zu bestellen
- Bestätige die Buchung und gib die geschätzte Ankunftszeit und Preis an.
Persönlichkeit:
- Sei freundlich und hilfsbereit und höflich
- Sei professionell und zuverlässig
`;