openai/openai-dotnet

Public

mirrored from https://github.com/openai/openai-dotnetAvailable

CodeCommitsIssuesPull requestsActionsInsightsSecurity
achandmsft-patch-2

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

README.md

1098lines · modecode

1# OpenAI .NET API library
2
3[![NuGet stable version](https://img.shields.io/nuget/v/openai.svg)](https://www.nuget.org/packages/OpenAI) [![NuGet preview version](https://img.shields.io/nuget/vpre/openai.svg)](https://www.nuget.org/packages/OpenAI/absoluteLatest)
4
5The OpenAI .NET library provides convenient access to the OpenAI REST API from .NET applications.
6
7It is generated from our [OpenAPI specification](https://github.com/openai/openai-openapi) in collaboration with Microsoft.
8
9## Table of Contents
10
11- [Getting started](#getting-started)
12 - [Prerequisites](#prerequisites)
13 - [Install the NuGet package](#install-the-nuget-package)
14- [Using the client library](#using-the-client-library)
15 - [Namespace organization](#namespace-organization)
16 - [Using the async API](#using-the-async-api)
17 - [Using the `OpenAIClient` class](#using-the-openaiclient-class)
18- [How to use dependency injection](#how-to-use-dependency-injection)
19- [How to use chat completions with streaming](#how-to-use-chat-completions-with-streaming)
20- [How to use chat completions with tools and function calling](#how-to-use-chat-completions-with-tools-and-function-calling)
21- [How to use chat completions with structured outputs](#how-to-use-chat-completions-with-structured-outputs)
22- [How to use chat completions with audio](#how-to-use-chat-completions-with-audio)
23- [How to use responses with streaming and reasoning](#how-to-use-responses-with-streaming-and-reasoning)
24- [How to use responses with file search](#how-to-use-responses-with-file-search)
25- [How to use responses with web search](#how-to-use-responses-with-web-search)
26- [How to generate text embeddings](#how-to-generate-text-embeddings)
27- [How to generate images](#how-to-generate-images)
28- [How to transcribe audio](#how-to-transcribe-audio)
29- [How to use assistants with retrieval augmented generation (RAG)](#how-to-use-assistants-with-retrieval-augmented-generation-rag)
30- [How to use assistants with streaming and vision](#how-to-use-assistants-with-streaming-and-vision)
31- [How to work with Azure OpenAI](#how-to-work-with-azure-openai)
32- [Advanced scenarios](#advanced-scenarios)
33 - [Using protocol methods](#using-protocol-methods)
34 - [Mock a client for testing](#mock-a-client-for-testing)
35 - [Automatically retrying errors](#automatically-retrying-errors)
36 - [Observability](#observability)
37
38## Getting started
39
40### Prerequisites
41
42To call the OpenAI REST API, you will need an API key. To obtain one, first [create a new OpenAI account](https://platform.openai.com/signup) or [log in](https://platform.openai.com/login). Next, navigate to the [API key page](https://platform.openai.com/account/api-keys) and select "Create new secret key", optionally naming the key. Make sure to save your API key somewhere safe and do not share it with anyone.
43
44### Install the NuGet package
45
46Add the client library to your .NET project by installing the [NuGet](https://www.nuget.org/) package via your IDE or by running the following command in the .NET CLI:
47
48```cli
49dotnet add package OpenAI
50```
51
52If you would like to try the latest preview version, remember to append the `--prerelease` command option.
53
54Note that the code examples included below were written using [.NET 8](https://dotnet.microsoft.com/download/dotnet/8.0). The OpenAI .NET library is compatible with all .NET Standard 2.0 applications, but the syntax used in some of the code examples in this document may depend on newer language features.
55
56## Using the client library
57
58The full API of this library can be found in the [OpenAI.netstandard2.0.cs](https://github.com/openai/openai-dotnet/blob/main/api/OpenAI.netstandard2.0.cs) file, and there are many [code examples](https://github.com/openai/openai-dotnet/tree/main/examples) to help. For instance, the following snippet illustrates the basic use of the chat completions API:
59
60```csharp
61using OpenAI.Chat;
62
63ChatClient client = new(model: "gpt-4o", apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
64
65ChatCompletion completion = client.CompleteChat("Say 'this is a test.'");
66
67Console.WriteLine($"[ASSISTANT]: {completion.Content[0].Text}");
68```
69
70While you can pass your API key directly as a string, it is highly recommended that you keep it in a secure location and instead access it via an environment variable or configuration file as shown above to avoid storing it in source control.
71
72### Using a custom base URL and API key
73
74If you need to connect to an alternative API endpoint (for example, a proxy or self-hosted OpenAI-compatible LLM), you can specify a custom base URL and API key using the `ApiKeyCredential` and `OpenAIClientOptions`:
75
76```csharp
77using OpenAI;
78using OpenAI.Chat;
79
80ChatClient client = new(
81 model: "MODEL_NAME",
82 credential: new ApiKeyCredential(Environment.GetEnvironmentVariable("OPENAI_API_KEY")),
83 options: new OpenAIClientOptions()
84 {
85 Endpoint = new Uri("BASE_URL")
86 }
87);
88```
89
90Replace `CHAT_MODEL` with your model name and `BASE_URL` with your endpoint URI. This is useful when working with OpenAI-compatible APIs or custom deployments.
91
92### Namespace organization
93
94The library is organized into namespaces by feature areas in the OpenAI REST API. Each namespace contains a corresponding client class.
95
96| Namespace | Client class |
97| ------------------------------|------------------------------|
98| `OpenAI.Assistants` | `AssistantClient` |
99| `OpenAI.Audio` | `AudioClient` |
100| `OpenAI.Batch` | `BatchClient` |
101| `OpenAI.Chat` | `ChatClient` |
102| `OpenAI.Embeddings` | `EmbeddingClient` |
103| `OpenAI.Evals` | `EvaluationClient` |
104| `OpenAI.FineTuning` | `FineTuningClient` |
105| `OpenAI.Files` | `OpenAIFileClient` |
106| `OpenAI.Images` | `ImageClient` |
107| `OpenAI.Models` | `OpenAIModelClient` |
108| `OpenAI.Moderations` | `ModerationClient` |
109| `OpenAI.Realtime` | `RealtimeClient` |
110| `OpenAI.Responses` | `OpenAIResponseClient` |
111| `OpenAI.VectorStores` | `VectorStoreClient` |
112
113### Using the async API
114
115Every client method that performs a synchronous API call has an asynchronous variant in the same client class. For instance, the asynchronous variant of the `ChatClient`'s `CompleteChat` method is `CompleteChatAsync`. To rewrite the call above using the asynchronous counterpart, simply `await` the call to the corresponding async variant:
116
117```csharp
118ChatCompletion completion = await client.CompleteChatAsync("Say 'this is a test.'");
119```
120
121### Using the `OpenAIClient` class
122
123In addition to the namespaces mentioned above, there is also the parent `OpenAI` namespace itself:
124
125```csharp
126using OpenAI;
127```
128
129This namespace contains the `OpenAIClient` class, which offers certain conveniences when you need to work with multiple feature area clients. Specifically, you can use an instance of this class to create instances of the other clients and have them share the same implementation details, which might be more efficient.
130
131You can create an `OpenAIClient` by specifying the API key that all clients will use for authentication:
132
133```csharp
134OpenAIClient client = new(Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
135```
136
137Next, to create an instance of an `AudioClient`, for example, you can call the `OpenAIClient`'s `GetAudioClient` method by passing the OpenAI model that the `AudioClient` will use, just as if you were using the `AudioClient` constructor directly. If necessary, you can create additional clients of the same type to target different models.
138
139```csharp
140AudioClient ttsClient = client.GetAudioClient("tts-1");
141AudioClient whisperClient = client.GetAudioClient("whisper-1");
142```
143
144## How to use dependency injection
145
146The OpenAI clients are **thread-safe** and can be safely registered as **singletons** in ASP.NET Core's Dependency Injection container. This maximizes resource efficiency and HTTP connection reuse.
147
148Register the `ChatClient` as a singleton in your `Program.cs`:
149
150```csharp
151builder.Services.AddSingleton<ChatClient>(serviceProvider =>
152{
153 var apiKey = Environment.GetEnvironmentVariable("OPENAI_API_KEY");
154
155 return new ChatClient(apiKey);
156});
157```
158
159Then inject and use the client in your controllers or services:
160
161```csharp
162[ApiController]
163[Route("api/[controller]")]
164public class ChatController : ControllerBase
165{
166 private readonly ChatClient _chatClient;
167
168 public ChatController(ChatClient chatClient)
169 {
170 _chatClient = chatClient;
171 }
172
173 [HttpPost("complete")]
174 public async Task<IActionResult> CompleteChat([FromBody] string message)
175 {
176 ChatCompletion completion = await _chatClient.CompleteChatAsync(message);
177
178 return Ok(new { response = completion.Content[0].Text });
179 }
180}
181```
182
183## How to use chat completions with streaming
184
185When you request a chat completion, the default behavior is for the server to generate it in its entirety before sending it back in a single response. Consequently, long chat completions can require waiting for several seconds before hearing back from the server. To mitigate this, the OpenAI REST API supports the ability to stream partial results back as they are being generated, allowing you to start processing the beginning of the completion before it is finished.
186
187The client library offers a convenient approach to working with streaming chat completions. If you wanted to re-write the example from the previous section using streaming, rather than calling the `ChatClient`'s `CompleteChat` method, you would call its `CompleteChatStreaming` method instead:
188
189```csharp
190CollectionResult<StreamingChatCompletionUpdate> completionUpdates = client.CompleteChatStreaming("Say 'this is a test.'");
191```
192
193Notice that the returned value is a `CollectionResult<StreamingChatCompletionUpdate>` instance, which can be enumerated to process the streaming response chunks as they arrive:
194
195```csharp
196Console.Write($"[ASSISTANT]: ");
197foreach (StreamingChatCompletionUpdate completionUpdate in completionUpdates)
198{
199 if (completionUpdate.ContentUpdate.Count > 0)
200 {
201 Console.Write(completionUpdate.ContentUpdate[0].Text);
202 }
203}
204```
205
206Alternatively, you can do this asynchronously by calling the `CompleteChatStreamingAsync` method to get an `AsyncCollectionResult<StreamingChatCompletionUpdate>` and enumerate it using `await foreach`:
207
208```csharp
209AsyncCollectionResult<StreamingChatCompletionUpdate> completionUpdates = client.CompleteChatStreamingAsync("Say 'this is a test.'");
210
211Console.Write($"[ASSISTANT]: ");
212await foreach (StreamingChatCompletionUpdate completionUpdate in completionUpdates)
213{
214 if (completionUpdate.ContentUpdate.Count > 0)
215 {
216 Console.Write(completionUpdate.ContentUpdate[0].Text);
217 }
218}
219```
220
221## How to use chat completions with tools and function calling
222
223In this example, you have two functions. The first function can retrieve a user's current geographic location (e.g., by polling the location service APIs of the user's device), while the second function can query the weather in a given location (e.g., by making an API call to some third-party weather service). You want the model to be able to call these functions if it deems it necessary to have this information in order to respond to a user request as part of generating a chat completion. For illustrative purposes, consider the following:
224
225```csharp
226private static string GetCurrentLocation()
227{
228 // Call the location API here.
229 return "San Francisco";
230}
231
232private static string GetCurrentWeather(string location, string unit = "celsius")
233{
234 // Call the weather API here.
235 return $"31 {unit}";
236}
237```
238
239Start by creating two `ChatTool` instances using the static `CreateFunctionTool` method to describe each function:
240
241```csharp
242private static readonly ChatTool getCurrentLocationTool = ChatTool.CreateFunctionTool(
243 functionName: nameof(GetCurrentLocation),
244 functionDescription: "Get the user's current location"
245);
246
247private static readonly ChatTool getCurrentWeatherTool = ChatTool.CreateFunctionTool(
248 functionName: nameof(GetCurrentWeather),
249 functionDescription: "Get the current weather in a given location",
250 functionParameters: BinaryData.FromBytes("""
251 {
252 "type": "object",
253 "properties": {
254 "location": {
255 "type": "string",
256 "description": "The city and state, e.g. Boston, MA"
257 },
258 "unit": {
259 "type": "string",
260 "enum": [ "celsius", "fahrenheit" ],
261 "description": "The temperature unit to use. Infer this from the specified location."
262 }
263 },
264 "required": [ "location" ]
265 }
266 """u8.ToArray())
267);
268```
269
270Next, create a `ChatCompletionOptions` instance and add both to its `Tools` property. You will pass the `ChatCompletionOptions` as an argument in your calls to the `ChatClient`'s `CompleteChat` method.
271
272```csharp
273List<ChatMessage> messages =
274[
275 new UserChatMessage("What's the weather like today?"),
276];
277
278ChatCompletionOptions options = new()
279{
280 Tools = { getCurrentLocationTool, getCurrentWeatherTool },
281};
282```
283
284When the resulting `ChatCompletion` has a `FinishReason` property equal to `ChatFinishReason.ToolCalls`, it means that the model has determined that one or more tools must be called before the assistant can respond appropriately. In those cases, you must first call the function specified in the `ChatCompletion`'s `ToolCalls` and then call the `ChatClient`'s `CompleteChat` method again while passing the function's result as an additional `ChatRequestToolMessage`. Repeat this process as needed.
285
286```csharp
287bool requiresAction;
288
289do
290{
291 requiresAction = false;
292 ChatCompletion completion = client.CompleteChat(messages, options);
293
294 switch (completion.FinishReason)
295 {
296 case ChatFinishReason.Stop:
297 {
298 // Add the assistant message to the conversation history.
299 messages.Add(new AssistantChatMessage(completion));
300 break;
301 }
302
303 case ChatFinishReason.ToolCalls:
304 {
305 // First, add the assistant message with tool calls to the conversation history.
306 messages.Add(new AssistantChatMessage(completion));
307
308 // Then, add a new tool message for each tool call that is resolved.
309 foreach (ChatToolCall toolCall in completion.ToolCalls)
310 {
311 switch (toolCall.FunctionName)
312 {
313 case nameof(GetCurrentLocation):
314 {
315 string toolResult = GetCurrentLocation();
316 messages.Add(new ToolChatMessage(toolCall.Id, toolResult));
317 break;
318 }
319
320 case nameof(GetCurrentWeather):
321 {
322 // The arguments that the model wants to use to call the function are specified as a
323 // stringified JSON object based on the schema defined in the tool definition. Note that
324 // the model may hallucinate arguments too. Consequently, it is important to do the
325 // appropriate parsing and validation before calling the function.
326 using JsonDocument argumentsJson = JsonDocument.Parse(toolCall.FunctionArguments);
327 bool hasLocation = argumentsJson.RootElement.TryGetProperty("location", out JsonElement location);
328 bool hasUnit = argumentsJson.RootElement.TryGetProperty("unit", out JsonElement unit);
329
330 if (!hasLocation)
331 {
332 throw new ArgumentNullException(nameof(location), "The location argument is required.");
333 }
334
335 string toolResult = hasUnit
336 ? GetCurrentWeather(location.GetString(), unit.GetString())
337 : GetCurrentWeather(location.GetString());
338 messages.Add(new ToolChatMessage(toolCall.Id, toolResult));
339 break;
340 }
341
342 default:
343 {
344 // Handle other unexpected calls.
345 throw new NotImplementedException();
346 }
347 }
348 }
349
350 requiresAction = true;
351 break;
352 }
353
354 case ChatFinishReason.Length:
355 throw new NotImplementedException("Incomplete model output due to MaxTokens parameter or token limit exceeded.");
356
357 case ChatFinishReason.ContentFilter:
358 throw new NotImplementedException("Omitted content due to a content filter flag.");
359
360 case ChatFinishReason.FunctionCall:
361 throw new NotImplementedException("Deprecated in favor of tool calls.");
362
363 default:
364 throw new NotImplementedException(completion.FinishReason.ToString());
365 }
366} while (requiresAction);
367```
368
369## How to use chat completions with structured outputs
370
371Beginning with the `gpt-4o-mini`, `gpt-4o-mini-2024-07-18`, and `gpt-4o-2024-08-06` model snapshots, structured outputs are available for both top-level response content and tool calls in the chat completion and assistants APIs. For information about the feature, see [the Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs/introduction).
372
373To use structured outputs to constrain chat completion content, set an appropriate `ChatResponseFormat` as in the following example:
374
375```csharp
376List<ChatMessage> messages =
377[
378 new UserChatMessage("How can I solve 8x + 7 = -23?"),
379];
380
381ChatCompletionOptions options = new()
382{
383 ResponseFormat = ChatResponseFormat.CreateJsonSchemaFormat(
384 jsonSchemaFormatName: "math_reasoning",
385 jsonSchema: BinaryData.FromBytes("""
386 {
387 "type": "object",
388 "properties": {
389 "steps": {
390 "type": "array",
391 "items": {
392 "type": "object",
393 "properties": {
394 "explanation": { "type": "string" },
395 "output": { "type": "string" }
396 },
397 "required": ["explanation", "output"],
398 "additionalProperties": false
399 }
400 },
401 "final_answer": { "type": "string" }
402 },
403 "required": ["steps", "final_answer"],
404 "additionalProperties": false
405 }
406 """u8.ToArray()),
407 jsonSchemaIsStrict: true)
408};
409
410ChatCompletion completion = client.CompleteChat(messages, options);
411
412using JsonDocument structuredJson = JsonDocument.Parse(completion.Content[0].Text);
413
414Console.WriteLine($"Final answer: {structuredJson.RootElement.GetProperty("final_answer")}");
415Console.WriteLine("Reasoning steps:");
416
417foreach (JsonElement stepElement in structuredJson.RootElement.GetProperty("steps").EnumerateArray())
418{
419 Console.WriteLine($" - Explanation: {stepElement.GetProperty("explanation")}");
420 Console.WriteLine($" Output: {stepElement.GetProperty("output")}");
421}
422```
423
424## How to use chat completions with audio
425
426Starting with the `gpt-4o-audio-preview` model, chat completions can process audio input and output.
427
428This example demonstrates:
429 1. Configuring the client with the supported `gpt-4o-audio-preview` model
430 1. Supplying user audio input on a chat completion request
431 1. Requesting model audio output from the chat completion operation
432 1. Retrieving audio output from a `ChatCompletion` instance
433 1. Using past audio output as `ChatMessage` conversation history
434
435```csharp
436// Chat audio input and output is only supported on specific models, beginning with gpt-4o-audio-preview
437ChatClient client = new("gpt-4o-audio-preview", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
438
439// Input audio is provided to a request by adding an audio content part to a user message
440string audioFilePath = Path.Combine("Assets", "realtime_whats_the_weather_pcm16_24khz_mono.wav");
441byte[] audioFileRawBytes = File.ReadAllBytes(audioFilePath);
442BinaryData audioData = BinaryData.FromBytes(audioFileRawBytes);
443List<ChatMessage> messages =
444 [
445 new UserChatMessage(ChatMessageContentPart.CreateInputAudioPart(audioData, ChatInputAudioFormat.Wav)),
446 ];
447
448// Output audio is requested by configuring ChatCompletionOptions to include the appropriate
449// ResponseModalities values and corresponding AudioOptions.
450ChatCompletionOptions options = new()
451{
452 ResponseModalities = ChatResponseModalities.Text | ChatResponseModalities.Audio,
453 AudioOptions = new(ChatOutputAudioVoice.Alloy, ChatOutputAudioFormat.Mp3),
454};
455
456ChatCompletion completion = client.CompleteChat(messages, options);
457
458void PrintAudioContent()
459{
460 if (completion.OutputAudio is ChatOutputAudio outputAudio)
461 {
462 Console.WriteLine($"Response audio transcript: {outputAudio.Transcript}");
463 string outputFilePath = $"{outputAudio.Id}.mp3";
464 using (FileStream outputFileStream = File.OpenWrite(outputFilePath))
465 {
466 outputFileStream.Write(outputAudio.AudioBytes);
467 }
468 Console.WriteLine($"Response audio written to file: {outputFilePath}");
469 Console.WriteLine($"Valid on followup requests until: {outputAudio.ExpiresAt}");
470 }
471}
472
473PrintAudioContent();
474
475// To refer to past audio output, create an assistant message from the earlier ChatCompletion, use the earlier
476// response content part, or use ChatMessageContentPart.CreateAudioPart(string) to manually instantiate a part.
477
478messages.Add(new AssistantChatMessage(completion));
479messages.Add("Can you say that like a pirate?");
480
481completion = client.CompleteChat(messages, options);
482
483PrintAudioContent();
484```
485
486Streaming is highly parallel: `StreamingChatCompletionUpdate` instances can include a `OutputAudioUpdate` that may
487contain any of:
488
489- The `Id` of the streamed audio content, which can be referenced by subsequent `AssistantChatMessage` instances via `ChatAudioReference` once the streaming response is complete; this may appear across multiple `StreamingChatCompletionUpdate` instances but will always be the same value when present
490- The `ExpiresAt` value that describes when the `Id` will no longer be valid for use with `ChatAudioReference` in subsequent requests; this typically appears once and only once, in the final `StreamingOutputAudioUpdate`
491- Incremental `TranscriptUpdate` and/or `AudioBytesUpdate` values, which can incrementally consumed and, when concatenated, form the complete audio transcript and audio output for the overall response; many of these typically appear
492
493## How to use responses with streaming and reasoning
494
495```csharp
496OpenAIResponseClient client = new(
497 model: "o3-mini",
498 apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
499
500OpenAIResponse response = await client.CreateResponseAsync(
501 userInputText: "What's the optimal strategy to win at poker?",
502 new ResponseCreationOptions()
503 {
504 ReasoningOptions = new ResponseReasoningOptions()
505 {
506 ReasoningEffortLevel = ResponseReasoningEffortLevel.High,
507 },
508 });
509
510await foreach (StreamingResponseUpdate update
511 in client.CreateResponseStreamingAsync(
512 userInputText: "What's the optimal strategy to win at poker?",
513 new ResponseCreationOptions()
514 {
515 ReasoningOptions = new ResponseReasoningOptions()
516 {
517 ReasoningEffortLevel = ResponseReasoningEffortLevel.High,
518 },
519 }))
520{
521 if (update is StreamingResponseItemUpdate itemUpdate
522 && itemUpdate.Item is ReasoningResponseItem reasoningItem)
523 {
524 Console.WriteLine($"[Reasoning] ({reasoningItem.Status})");
525 }
526 else if (update is StreamingResponseContentPartDeltaUpdate deltaUpdate)
527 {
528 Console.Write(deltaUpdate.Text);
529 }
530}
531```
532
533## How to use responses with file search
534
535```csharp
536OpenAIResponseClient client = new(
537 model: "gpt-4o-mini",
538 apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
539
540ResponseTool fileSearchTool
541 = ResponseTool.CreateFileSearchTool(
542 vectorStoreIds: [ExistingVectorStoreForTest.Id]);
543OpenAIResponse response = await client.CreateResponseAsync(
544 userInputText: "According to available files, what's the secret number?",
545 new ResponseCreationOptions()
546 {
547 Tools = { fileSearchTool }
548 });
549
550foreach (ResponseItem outputItem in response.OutputItems)
551{
552 if (outputItem is FileSearchCallResponseItem fileSearchCall)
553 {
554 Console.WriteLine($"[file_search] ({fileSearchCall.Status}): {fileSearchCall.Id}");
555 foreach (string query in fileSearchCall.Queries)
556 {
557 Console.WriteLine($" - {query}");
558 }
559 }
560 else if (outputItem is MessageResponseItem message)
561 {
562 Console.WriteLine($"[{message.Role}] {message.Content.FirstOrDefault()?.Text}");
563 }
564}
565```
566
567## How to use responses with web search
568
569```csharp
570OpenAIResponseClient client = new(
571 model: "gpt-4o-mini",
572 apiKey: Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
573
574OpenAIResponse response = await client.CreateResponseAsync(
575 userInputText: "What's a happy news headline from today?",
576 new ResponseCreationOptions()
577 {
578 Tools = { ResponseTool.CreateWebSearchTool() },
579 });
580
581foreach (ResponseItem item in response.OutputItems)
582{
583 if (item is WebSearchCallResponseItem webSearchCall)
584 {
585 Console.WriteLine($"[Web search invoked]({webSearchCall.Status}) {webSearchCall.Id}");
586 }
587 else if (item is MessageResponseItem message)
588 {
589 Console.WriteLine($"[{message.Role}] {message.Content?.FirstOrDefault()?.Text}");
590 }
591}
592```
593
594## How to generate text embeddings
595
596In this example, you want to create a trip-planning website that allows customers to write a prompt describing the kind of hotel that they are looking for and then offers hotel recommendations that closely match this description. To achieve this, it is possible to use text embeddings to measure the relatedness of text strings. In summary, you can get embeddings of the hotel descriptions, store them in a vector database, and use them to build a search index that you can query using the embedding of a given customer's prompt.
597
598To generate a text embedding, use `EmbeddingClient` from the `OpenAI.Embeddings` namespace:
599
600```csharp
601using OpenAI.Embeddings;
602
603EmbeddingClient client = new("text-embedding-3-small", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
604
605string description = "Best hotel in town if you like luxury hotels. They have an amazing infinity pool, a spa,"
606 + " and a really helpful concierge. The location is perfect -- right downtown, close to all the tourist"
607 + " attractions. We highly recommend this hotel.";
608
609OpenAIEmbedding embedding = client.GenerateEmbedding(description);
610ReadOnlyMemory<float> vector = embedding.ToFloats();
611```
612
613Notice that the resulting embedding is a list (also called a vector) of floating point numbers represented as an instance of `ReadOnlyMemory<float>`. By default, the length of the embedding vector will be 1536 when using the `text-embedding-3-small` model or 3072 when using the `text-embedding-3-large` model. Generally, larger embeddings perform better, but using them also tends to cost more in terms of compute, memory, and storage. You can reduce the dimensions of the embedding by creating an instance of the `EmbeddingGenerationOptions` class, setting the `Dimensions` property, and passing it as an argument in your call to the `GenerateEmbedding` method:
614
615```csharp
616EmbeddingGenerationOptions options = new() { Dimensions = 512 };
617
618OpenAIEmbedding embedding = client.GenerateEmbedding(description, options);
619```
620
621## How to generate images
622
623In this example, you want to build an app to help interior designers prototype new ideas based on the latest design trends. As part of the creative process, an interior designer can use this app to generate images for inspiration simply by describing the scene in their head as a prompt. As expected, high-quality, strikingly dramatic images with finer details deliver the best results for this application.
624
625To generate an image, use `ImageClient` from the `OpenAI.Images` namespace:
626
627```csharp
628using OpenAI.Images;
629
630ImageClient client = new("dall-e-3", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
631```
632
633Generating an image always requires a `prompt` that describes what should be generated. To further tailor the image generation to your specific needs, you can create an instance of the `ImageGenerationOptions` class and set the `Quality`, `Size`, and `Style` properties accordingly. Note that you can also set the `ResponseFormat` property of `ImageGenerationOptions` to `GeneratedImageFormat.Bytes` in order to receive the resulting PNG as `BinaryData` (instead of the default remote `Uri`) if this is convenient for your use case.
634
635```csharp
636string prompt = "The concept for a living room that blends Scandinavian simplicity with Japanese minimalism for"
637 + " a serene and cozy atmosphere. It's a space that invites relaxation and mindfulness, with natural light"
638 + " and fresh air. Using neutral tones, including colors like white, beige, gray, and black, that create a"
639 + " sense of harmony. Featuring sleek wood furniture with clean lines and subtle curves to add warmth and"
640 + " elegance. Plants and flowers in ceramic pots adding color and life to a space. They can serve as focal"
641 + " points, creating a connection with nature. Soft textiles and cushions in organic fabrics adding comfort"
642 + " and softness to a space. They can serve as accents, adding contrast and texture.";
643
644ImageGenerationOptions options = new()
645{
646 Quality = GeneratedImageQuality.High,
647 Size = GeneratedImageSize.W1792xH1024,
648 Style = GeneratedImageStyle.Vivid,
649 ResponseFormat = GeneratedImageFormat.Bytes
650};
651```
652
653Finally, call the `ImageClient`'s `GenerateImage` method by passing the prompt and the `ImageGenerationOptions` instance as arguments:
654
655```csharp
656GeneratedImage image = client.GenerateImage(prompt, options);
657BinaryData bytes = image.ImageBytes;
658```
659
660For illustrative purposes, you could then save the generated image to local storage:
661
662```csharp
663using FileStream stream = File.OpenWrite($"{Guid.NewGuid()}.png");
664bytes.ToStream().CopyTo(stream);
665```
666
667## How to transcribe audio
668
669In this example, an audio file is transcribed using the Whisper speech-to-text model, including both word- and audio-segment-level timestamp information.
670
671```csharp
672using OpenAI.Audio;
673
674AudioClient client = new("whisper-1", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
675
676string audioFilePath = Path.Combine("Assets", "audio_houseplant_care.mp3");
677
678AudioTranscriptionOptions options = new()
679{
680 ResponseFormat = AudioTranscriptionFormat.Verbose,
681 TimestampGranularities = AudioTimestampGranularities.Word | AudioTimestampGranularities.Segment,
682};
683
684AudioTranscription transcription = client.TranscribeAudio(audioFilePath, options);
685
686Console.WriteLine("Transcription:");
687Console.WriteLine($"{transcription.Text}");
688
689Console.WriteLine();
690Console.WriteLine($"Words:");
691foreach (TranscribedWord word in transcription.Words)
692{
693 Console.WriteLine($" {word.Word,15} : {word.StartTime.TotalMilliseconds,5:0} - {word.EndTime.TotalMilliseconds,5:0}");
694}
695
696Console.WriteLine();
697Console.WriteLine($"Segments:");
698foreach (TranscribedSegment segment in transcription.Segments)
699{
700 Console.WriteLine($" {segment.Text,90} : {segment.StartTime.TotalMilliseconds,5:0} - {segment.EndTime.TotalMilliseconds,5:0}");
701}
702```
703
704## How to use assistants with retrieval augmented generation (RAG)
705
706In this example, you have a JSON document with the monthly sales information of different products, and you want to build an assistant capable of analyzing it and answering questions about it.
707
708To achieve this, use both `OpenAIFileClient` from the `OpenAI.Files` namespace and `AssistantClient` from the `OpenAI.Assistants` namespace.
709
710Important: The Assistants REST API is currently in beta. As such, the details are subject to change, and correspondingly the `AssistantClient` is attributed as `[Experimental]`. To use it, you must suppress the `OPENAI001` warning first.
711
712```csharp
713using OpenAI.Assistants;
714using OpenAI.Files;
715
716OpenAIClient openAIClient = new(Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
717OpenAIFileClient fileClient = openAIClient.GetOpenAIFileClient();
718AssistantClient assistantClient = openAIClient.GetAssistantClient();
719```
720
721Here is an example of what the JSON document might look like:
722
723```csharp
724using Stream document = BinaryData.FromBytes("""
725 {
726 "description": "This document contains the sale history data for Contoso products.",
727 "sales": [
728 {
729 "month": "January",
730 "by_product": {
731 "113043": 15,
732 "113045": 12,
733 "113049": 2
734 }
735 },
736 {
737 "month": "February",
738 "by_product": {
739 "113045": 22
740 }
741 },
742 {
743 "month": "March",
744 "by_product": {
745 "113045": 16,
746 "113055": 5
747 }
748 }
749 ]
750 }
751 """u8.ToArray()).ToStream();
752```
753
754Upload this document to OpenAI using the `OpenAIFileClient`'s `UploadFile` method, ensuring that you use `FileUploadPurpose.Assistants` to allow your assistant to access it later:
755
756```csharp
757OpenAIFile salesFile = fileClient.UploadFile(
758 document,
759 "monthly_sales.json",
760 FileUploadPurpose.Assistants);
761```
762
763Create a new assistant using an instance of the `AssistantCreationOptions` class to customize it. Here, we use:
764
765- A friendly `Name` for the assistant, as will display in the Playground
766- Tool definition instances for the tools that the assistant should have access to; here, we use `FileSearchToolDefinition` to process the sales document we just uploaded and `CodeInterpreterToolDefinition` so we can analyze and visualize the numeric data
767- Resources for the assistant to use with its tools, here using the `VectorStoreCreationHelper` type to automatically make a new vector store that indexes the sales file; alternatively, you could use `VectorStoreClient` to manage the vector store separately
768
769```csharp
770AssistantCreationOptions assistantOptions = new()
771{
772 Name = "Example: Contoso sales RAG",
773 Instructions =
774 "You are an assistant that looks up sales data and helps visualize the information based"
775 + " on user queries. When asked to generate a graph, chart, or other visualization, use"
776 + " the code interpreter tool to do so.",
777 Tools =
778 {
779 new FileSearchToolDefinition(),
780 new CodeInterpreterToolDefinition(),
781 },
782 ToolResources = new()
783 {
784 FileSearch = new()
785 {
786 NewVectorStores =
787 {
788 new VectorStoreCreationHelper([salesFile.Id]),
789 }
790 }
791 },
792};
793
794Assistant assistant = assistantClient.CreateAssistant("gpt-4o", assistantOptions);
795```
796
797Next, create a new thread. For illustrative purposes, you could include an initial user message asking about the sales information of a given product and then use the `AssistantClient`'s `CreateThreadAndRun` method to get it started:
798
799```csharp
800ThreadCreationOptions threadOptions = new()
801{
802 InitialMessages = { "How well did product 113045 sell in February? Graph its trend over time." }
803};
804
805ThreadRun threadRun = assistantClient.CreateThreadAndRun(assistant.Id, threadOptions);
806```
807
808Poll the status of the run until it is no longer queued or in progress:
809
810```csharp
811do
812{
813 Thread.Sleep(TimeSpan.FromSeconds(1));
814 threadRun = assistantClient.GetRun(threadRun.ThreadId, threadRun.Id);
815} while (!threadRun.Status.IsTerminal);
816```
817
818If everything went well, the terminal status of the run will be `RunStatus.Completed`.
819
820Finally, you can use the `AssistantClient`'s `GetMessages` method to retrieve the messages associated with this thread, which now include the responses from the assistant to the initial user message.
821
822For illustrative purposes, you could print the messages to the console and also save any images produced by the assistant to local storage:
823
824```csharp
825CollectionResult<ThreadMessage> messages
826 = assistantClient.GetMessages(threadRun.ThreadId, new MessageCollectionOptions() { Order = MessageCollectionOrder.Ascending });
827
828foreach (ThreadMessage message in messages)
829{
830 Console.Write($"[{message.Role.ToString().ToUpper()}]: ");
831 foreach (MessageContent contentItem in message.Content)
832 {
833 if (!string.IsNullOrEmpty(contentItem.Text))
834 {
835 Console.WriteLine($"{contentItem.Text}");
836
837 if (contentItem.TextAnnotations.Count > 0)
838 {
839 Console.WriteLine();
840 }
841
842 // Include annotations, if any.
843 foreach (TextAnnotation annotation in contentItem.TextAnnotations)
844 {
845 if (!string.IsNullOrEmpty(annotation.InputFileId))
846 {
847 Console.WriteLine($"* File citation, file ID: {annotation.InputFileId}");
848 }
849 if (!string.IsNullOrEmpty(annotation.OutputFileId))
850 {
851 Console.WriteLine($"* File output, new file ID: {annotation.OutputFileId}");
852 }
853 }
854 }
855 if (!string.IsNullOrEmpty(contentItem.ImageFileId))
856 {
857 OpenAIFile imageInfo = fileClient.GetFile(contentItem.ImageFileId);
858 BinaryData imageBytes = fileClient.DownloadFile(contentItem.ImageFileId);
859 using FileStream stream = File.OpenWrite($"{imageInfo.Filename}.png");
860 imageBytes.ToStream().CopyTo(stream);
861
862 Console.WriteLine($"<image: {imageInfo.Filename}.png>");
863 }
864 }
865 Console.WriteLine();
866}
867```
868
869And it would yield something like this:
870
871```text
872[USER]: How well did product 113045 sell in February? Graph its trend over time.
873
874[ASSISTANT]: Product 113045 sold 22 units in February【4:0†monthly_sales.json】.
875
876Now, I will generate a graph to show its sales trend over time.
877
878* File citation, file ID: file-hGOiwGNftMgOsjbynBpMCPFn
879
880[ASSISTANT]: <image: 015d8e43-17fe-47de-af40-280f25452280.png>
881The sales trend for Product 113045 over the past three months shows that:
882
883- In January, 12 units were sold.
884- In February, 22 units were sold, indicating significant growth.
885- In March, sales dropped slightly to 16 units.
886
887The graph above visualizes this trend, showing a peak in sales during February.
888```
889
890## How to use assistants with streaming and vision
891
892This example shows how to use the v2 Assistants API to provide image data to an assistant and then stream the run's response.
893
894As before, you will use a `OpenAIFileClient` and an `AssistantClient`:
895
896```csharp
897OpenAIClient openAIClient = new(Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
898OpenAIFileClient fileClient = openAIClient.GetOpenAIFileClient();
899AssistantClient assistantClient = openAIClient.GetAssistantClient();
900```
901
902For this example, we will use both image data from a local file as well as an image located at a URL. For the local data, we upload the file with the `Vision` upload purpose, which would also allow it to be downloaded and retrieved later.
903
904```csharp
905OpenAIFile pictureOfAppleFile = fileClient.UploadFile(
906 Path.Combine("Assets", "images_apple.png"),
907 FileUploadPurpose.Vision);
908
909Uri linkToPictureOfOrange = new("https://raw.githubusercontent.com/openai/openai-dotnet/refs/heads/main/examples/Assets/images_orange.png");
910```
911
912Next, create a new assistant with a vision-capable model like `gpt-4o` and a thread with the image information referenced:
913
914```csharp
915Assistant assistant = assistantClient.CreateAssistant(
916 "gpt-4o",
917 new AssistantCreationOptions()
918 {
919 Instructions = "When asked a question, attempt to answer very concisely. "
920 + "Prefer one-sentence answers whenever feasible."
921 });
922
923AssistantThread thread = assistantClient.CreateThread(new ThreadCreationOptions()
924{
925 InitialMessages =
926 {
927 new ThreadInitializationMessage(
928 MessageRole.User,
929 [
930 "Hello, assistant! Please compare these two images for me:",
931 MessageContent.FromImageFileId(pictureOfAppleFile.Id),
932 MessageContent.FromImageUri(linkToPictureOfOrange),
933 ]),
934 }
935});
936```
937
938With the assistant and thread prepared, use the `CreateRunStreaming` method to get an enumerable `CollectionResult<StreamingUpdate>`. You can then iterate over this collection with `foreach`. For async calling patterns, use `CreateRunStreamingAsync` and iterate over the `AsyncCollectionResult<StreamingUpdate>` with `await foreach`, instead. Note that streaming variants also exist for `CreateThreadAndRunStreaming` and `SubmitToolOutputsToRunStreaming`.
939
940```csharp
941CollectionResult<StreamingUpdate> streamingUpdates = assistantClient.CreateRunStreaming(
942 thread.Id,
943 assistant.Id,
944 new RunCreationOptions()
945 {
946 AdditionalInstructions = "When possible, try to sneak in puns if you're asked to compare things.",
947 });
948```
949
950Finally, to handle the `StreamingUpdates` as they arrive, you can use the `UpdateKind` property on the base `StreamingUpdate` and/or downcast to a specifically desired update type, like `MessageContentUpdate` for `thread.message.delta` events or `RequiredActionUpdate` for streaming tool calls.
951
952```csharp
953foreach (StreamingUpdate streamingUpdate in streamingUpdates)
954{
955 if (streamingUpdate.UpdateKind == StreamingUpdateReason.RunCreated)
956 {
957 Console.WriteLine($"--- Run started! ---");
958 }
959 if (streamingUpdate is MessageContentUpdate contentUpdate)
960 {
961 Console.Write(contentUpdate.Text);
962 }
963}
964```
965
966This will yield streamed output from the run like the following:
967
968```text
969--- Run started! ---
970The first image depicts a multicolored apple with a blend of red and green hues, while the second image shows an orange with a bright, textured orange peel; one might say it’s comparing apples to oranges!
971```
972
973## How to work with Azure OpenAI
974
975For Azure OpenAI scenarios use the [Azure SDK](https://github.com/Azure/azure-sdk-for-net) and more specifically the [Azure OpenAI client library for .NET](https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/openai/Azure.AI.OpenAI/README.md).
976
977The Azure OpenAI client library for .NET is a companion to this library and all common capabilities between OpenAI and Azure OpenAI share the same scenario clients, methods, and request/response types. It is designed to make Azure specific scenarios straightforward, with extensions for Azure-specific concepts like Responsible AI content filter results and On Your Data integration.
978
979```c#
980AzureOpenAIClient azureClient = new(
981 new Uri("https://your-azure-openai-resource.com"),
982 new DefaultAzureCredential());
983ChatClient chatClient = azureClient.GetChatClient("my-gpt-35-turbo-deployment");
984
985ChatCompletion completion = chatClient.CompleteChat(
986 [
987 // System messages represent instructions or other guidance about how the assistant should behave
988 new SystemChatMessage("You are a helpful assistant that talks like a pirate."),
989 // User messages represent user input, whether historical or the most recen tinput
990 new UserChatMessage("Hi, can you help me?"),
991 // Assistant messages in a request represent conversation history for responses
992 new AssistantChatMessage("Arrr! Of course, me hearty! What can I do for ye?"),
993 new UserChatMessage("What's the best way to train a parrot?"),
994 ]);
995
996Console.WriteLine($"{completion.Role}: {completion.Content[0].Text}");
997```
998
999## Advanced scenarios
1000
1001### Using protocol methods
1002
1003In addition to the client methods that use strongly-typed request and response objects, the .NET library also provides _protocol methods_ that enable more direct access to the REST API. Protocol methods are "binary in, binary out" accepting `BinaryContent` as request bodies and providing `BinaryData` as response bodies.
1004
1005For example, to use the protocol method variant of the `ChatClient`'s `CompleteChat` method, pass the request body as `BinaryContent`:
1006
1007```csharp
1008ChatClient client = new("gpt-4o", Environment.GetEnvironmentVariable("OPENAI_API_KEY"));
1009
1010BinaryData input = BinaryData.FromBytes("""
1011 {
1012 "model": "gpt-4o",
1013 "messages": [
1014 {
1015 "role": "user",
1016 "content": "Say 'this is a test.'"
1017 }
1018 ]
1019 }
1020 """u8.ToArray());
1021
1022using BinaryContent content = BinaryContent.Create(input);
1023ClientResult result = client.CompleteChat(content);
1024BinaryData output = result.GetRawResponse().Content;
1025
1026using JsonDocument outputAsJson = JsonDocument.Parse(output.ToString());
1027string message = outputAsJson.RootElement
1028 .GetProperty("choices"u8)[0]
1029 .GetProperty("message"u8)
1030 .GetProperty("content"u8)
1031 .GetString();
1032
1033Console.WriteLine($"[ASSISTANT]: {message}");
1034```
1035
1036Notice how you can then call the resulting `ClientResult`'s `GetRawResponse` method and retrieve the response body as `BinaryData` via the `PipelineResponse`'s `Content` property.
1037
1038### Mock a client for testing
1039
1040The OpenAI .NET library has been designed to support mocking, providing key features such as:
1041
1042- Client methods made virtual to allow overriding.
1043- Model factories to assist in instantiating API output models that lack public constructors.
1044
1045To illustrate how mocking works, suppose you want to validate the behavior of the following method using the [Moq](https://github.com/devlooped/moq) library. Given the path to an audio file, it determines whether it contains a specified secret word:
1046
1047```csharp
1048public bool ContainsSecretWord(AudioClient client, string audioFilePath, string secretWord)
1049{
1050 AudioTranscription transcription = client.TranscribeAudio(audioFilePath);
1051 return transcription.Text.Contains(secretWord);
1052}
1053```
1054
1055Create mocks of `AudioClient` and `ClientResult<AudioTranscription>`, set up methods and properties that will be invoked, then test the behavior of the `ContainsSecretWord` method. Since the `AudioTranscription` class does not provide public constructors, it must be instantiated by the `OpenAIAudioModelFactory` static class:
1056
1057```csharp
1058// Instantiate mocks and the AudioTranscription object.
1059
1060Mock<AudioClient> mockClient = new();
1061Mock<ClientResult<AudioTranscription>> mockResult = new(null, Mock.Of<PipelineResponse>());
1062AudioTranscription transcription = OpenAIAudioModelFactory.AudioTranscription(text: "I swear I saw an apple flying yesterday!");
1063
1064// Set up mocks' properties and methods.
1065
1066mockResult
1067 .SetupGet(result => result.Value)
1068 .Returns(transcription);
1069
1070mockClient.Setup(client => client.TranscribeAudio(
1071 It.IsAny<string>(),
1072 It.IsAny<AudioTranscriptionOptions>()))
1073 .Returns(mockResult.Object);
1074
1075// Perform validation.
1076
1077AudioClient client = mockClient.Object;
1078bool containsSecretWord = ContainsSecretWord(client, "<audioFilePath>", "apple");
1079
1080Assert.That(containsSecretWord, Is.True);
1081```
1082
1083All namespaces have their corresponding model factory to support mocking with the exception of the `OpenAI.Assistants` and `OpenAI.VectorStores` namespaces, for which model factories are coming soon.
1084
1085### Automatically retrying errors
1086
1087By default, the client classes will automatically retry the following errors up to three additional times using exponential backoff:
1088
1089- 408 Request Timeout
1090- 429 Too Many Requests
1091- 500 Internal Server Error
1092- 502 Bad Gateway
1093- 503 Service Unavailable
1094- 504 Gateway Timeout
1095
1096### Observability
1097
1098OpenAI .NET library supports experimental distributed tracing and metrics with OpenTelemetry. Check out [Observability with OpenTelemetry](./docs/observability.md) for more details.