Issues running the dotnet OpenAIRealtime sample

[vicperdana]

Hi, I am getting this error when running the OpenAIRealtime sample, I've deployed AzOpenAI gpt-4o-realtime-preview and configured my dotnet user-secrets to point to the model/deployment. Is this something that anyone experienced before?

System.Net.WebSockets.WebSocketException (2): The remote party closed the WebSocket connection without completing the close handshake. ---> System.IO.IOException: Unable to write data to the transport connection: Broken pipe. ---> System.Net.Sockets.SocketException (32): Broken pipe --- End of inner exception stack trace --- at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) at System.Net.Security.SslStream.<WriteSingleChunk>g__CompleteWriteAsync|157_1[TIOAdapter](ValueTask writeTask, Byte[] bufferToReturn) at System.Net.Security.SslStream.WriteAsyncInternal[TIOAdapter](ReadOnlyMemory 1 buffer, CancellationToken cancellationToken) at System.Net.Http.HttpConnection.RawConnectionStream.WaitWithConnectionCancellationAsync(ValueTask task, HttpConnection connection, CancellationToken cancellationToken) at System.Net.WebSockets.ManagedWebSocket.WaitForWriteTaskAsync(ValueTask writeTask, Boolean shouldFlush) at System.Net.WebSockets.ManagedWebSocket.WaitForWriteTaskAsync(ValueTask writeTask, Boolean shouldFlush) at OpenAI.RealtimeConversation.RealtimeConversationSession.SendCommandAsync(BinaryData data, RequestOptions options) at OpenAI.RealtimeConversation.RealtimeConversationSession.SendInputAudioAsync(Stream audio, CancellationToken cancellationToken) at OpenAIRealtime.Program.Main(String[] args) in /Users/vicperdana/coderepo/ai/sk/semantic-kernel/dotnet/samples/Demos/OpenAIRealtime/Program.cs:line 78

[dmytrostruk]

@vicperdana Thanks for reporting this issue! I just checked and the example is working as expected for me with Azure deployment. A couple of notes:

1. Ensure that this code is used to initialize RealtimeConversationClient:

var client = new AzureOpenAIClient(
    endpoint: new Uri(azureOpenAIOptions.Endpoint),
    credential: new ApiKeyCredential(azureOpenAIOptions.ApiKey));

return client.GetRealtimeConversationClient(azureOpenAIOptions.DeploymentName);

2. You need to use different credentials for realtime model, which you can find in Azure AI Foundry under deployment information:
   

3. Ensure that valid credentials are used in runtime - sometimes it can be fetched from environment variables instead.

4. Ensure that you use deployment name and not model name.

5. Just as a note, I used East US region for my testing.

Please let me know if that helps, thanks!

[vicperdana]

Thanks @dmytrostruk I can run the sample successfully. Just a tip for others: make sure you don't have both OpenAI and Azure Open AI secrets as it uses OAI secrets first!

Also, just a minor thing, if you're using MacOS, you need to change this line from \ to / in this line
string inputAudioPath = FindFile("Assets/realtime_whats_the_weather_pcm16_24khz_mono.wav");