openai/chatkit-python
Publicmirrored fromhttps://github.com/openai/chatkit-pythonAvailable
docs/guides/prepare-your-app-for-production.md
255lines · modecode
| 1 | # Prepare your app for production |
| 2 | |
| 3 | This guide covers the operational work you should do before rolling out a ChatKit‑powered experience in production: |
| 4 | |
| 5 | - Set up **localization** so prompts, system messages, and tool output match the user’s locale. |
| 6 | - Configure **monitoring and logging** so you can debug issues and correlate ChatKit traffic with your backend traces. |
| 7 | - Review **security and authentication** for your ChatKit endpoint. |
| 8 | - Register and use **domain keys** to lock ChatKit down to your approved hostnames. |
| 9 | |
| 10 | Use it as a checklist alongside your own product’s launch process. |
| 11 | |
| 12 | ## Localize prompts, UI copy, and tool output |
| 13 | |
| 14 | By the time you go live, you should have a clear story for which locales you support and how locale flows from the client into your backend and model prompts. |
| 15 | |
| 16 | ChatKit always picks a **single active locale** and: |
| 17 | |
| 18 | - Uses the **browser locale by default**. |
| 19 | - Lets you **override the locale on the client** (for example, from your own locale picker) by passing the `locale` option when you initialize ChatKit. |
| 20 | |
| 21 | For every request to your ChatKit backend, the client sends an `Accept-Language` header with that single locale value. You can rely on this header to drive your own localization logic on the server. |
| 22 | |
| 23 | At a minimum: |
| 24 | |
| 25 | - **Decide which locales you support** (for example `["en", "fr", "de"]`) and what the default/fallback is. |
| 26 | - **Localize tool output and error messages** so the assistant’s replies feel consistent with the rest of your product. |
| 27 | |
| 28 | For example, you might include locale in your per‑request context: |
| 29 | |
| 30 | ```python |
| 31 | from dataclasses import dataclass |
| 32 | |
| 33 | |
| 34 | @dataclass |
| 35 | class RequestContext: |
| 36 | user_id: str |
| 37 | locale: str |
| 38 | ``` |
| 39 | |
| 40 | Then, when you build prompts or tool output, read `context.locale` and render language‑appropriate text using your localization system. For example, with `gettext`: |
| 41 | |
| 42 | ```python |
| 43 | from pathlib import Path |
| 44 | import gettext |
| 45 | |
| 46 | from agents import RunContextWrapper, function_tool |
| 47 | from chatkit.agents import AgentContext |
| 48 | |
| 49 | |
| 50 | LOCALE_DIR = Path(__file__).with_suffix("").parent / "locales" |
| 51 | _translations: dict[str, gettext.NullTranslations] = {} |
| 52 | |
| 53 | |
| 54 | def get_translations(locale: str) -> gettext.NullTranslations: |
| 55 | """Return a gettext translation object for the given locale.""" |
| 56 | if locale not in _translations: |
| 57 | _translations[locale] = gettext.translation( |
| 58 | "messages", # your .po/.mo domain |
| 59 | localedir=LOCALE_DIR, |
| 60 | languages=[locale], |
| 61 | fallback=True, |
| 62 | ) |
| 63 | return _translations[locale] |
| 64 | |
| 65 | |
| 66 | @function_tool() |
| 67 | async def load_document( |
| 68 | ctx: RunContextWrapper[AgentContext], |
| 69 | document_id: str, |
| 70 | ): |
| 71 | locale = ctx.context.request_context.locale |
| 72 | _ = get_translations(locale).gettext |
| 73 | await ctx.context.stream_progress( |
| 74 | icon="document", |
| 75 | text=_("Loading document…"), |
| 76 | ) |
| 77 | doc = await get_document_by_id(document_id) |
| 78 | if not doc: |
| 79 | raise ValueError(_("We couldn’t find that document.")) |
| 80 | return doc |
| 81 | ``` |
| 82 | |
| 83 | When you call the model (for example via the OpenAI Responses API), include the user’s locale either directly in the prompt or as part of a system message so the model responds in the right language. |
| 84 | |
| 85 | ## Monitor logs and errors |
| 86 | |
| 87 | You should be able to answer questions like “what went wrong for this user at this time?” and “are ChatKit requests healthy right now?” before you roll out broadly. |
| 88 | |
| 89 | ### Capture client logs |
| 90 | |
| 91 | On the **client side**, subscribe to ChatKit’s log and error events and forward them into your own telemetry system, tagged with: |
| 92 | |
| 93 | - User identifier (or stable anonymous id). |
| 94 | - Session or request id. |
| 95 | - The current thread id. |
| 96 | |
| 97 | In React, use the `onLog` and `onError` options (mirroring the patterns in the ChatKit JS [Monitor logs](https://openai.github.io/chatkit-js/guides/monitor-logs/) guide): |
| 98 | |
| 99 | ```tsx |
| 100 | import { ChatKit, useChatKit } from "@openai/chatkit-react"; |
| 101 | |
| 102 | export function SupportChat({ |
| 103 | clientToken, |
| 104 | userId, |
| 105 | }: { |
| 106 | clientToken: string; |
| 107 | userId: string; |
| 108 | }) { |
| 109 | const { control } = useChatKit({ |
| 110 | api: { clientToken }, |
| 111 | onLog: ({ name, data }) => |
| 112 | sendToTelemetry({ |
| 113 | name, |
| 114 | // Avoid forwarding raw message text or tool arguments directly. |
| 115 | data: scrubSensitiveFields(data), |
| 116 | userId, |
| 117 | }), |
| 118 | onError: ({ error }) => |
| 119 | sendToTelemetry({ |
| 120 | name: "chatkit.error", |
| 121 | error: scrubSensitiveFields(error), |
| 122 | userId, |
| 123 | }), |
| 124 | }); |
| 125 | |
| 126 | return <ChatKit control={control} className="h-[600px]" />; |
| 127 | } |
| 128 | ``` |
| 129 | |
| 130 | With the web component, listen for `chatkit.log` and `chatkit.error` events: |
| 131 | |
| 132 | ```ts |
| 133 | const chatkit = document.getElementById("my-chat") as OpenAIChatKit; |
| 134 | |
| 135 | chatkit.addEventListener("chatkit.log", ({ detail }) => { |
| 136 | sendToTelemetry({ |
| 137 | name: detail.name, |
| 138 | data: scrubSensitiveFields(detail.data), |
| 139 | userId, |
| 140 | }); |
| 141 | }); |
| 142 | |
| 143 | chatkit.addEventListener("chatkit.error", (event) => { |
| 144 | sendToTelemetry({ |
| 145 | name: "chatkit.error", |
| 146 | error: scrubSensitiveFields(event.detail.error), |
| 147 | userId, |
| 148 | }); |
| 149 | }); |
| 150 | ``` |
| 151 | |
| 152 | These events can include **PII and message contents**, so avoid blanket-forwarding entire payloads; instead, extract and forward only the fields you need (for example, error codes, item ids, thread ids, and high‑level event names) and/or scrub sensitive fields before sending them to your logging provider. |
| 153 | |
| 154 | Separately from your own telemetry, the ChatKit iframe sends **its own outbound telemetry** to OpenAI‑controlled endpoints (Datadog and `chatgpt.com`) for monitoring and debugging. These internal logs **do not contain PII or message input/output content** and are used only to monitor the health of the ChatKit experience. |
| 155 | |
| 156 | ### Monitor your ChatKit endpoint |
| 157 | |
| 158 | On the **backend**, you should still capture basic logs around your ChatKit endpoint so you can correlate client telemetry with server behavior: |
| 159 | |
| 160 | - Incoming HTTP request (path, method, user id, thread id). |
| 161 | - Calls to `ChatKitServer.process` and your `Store` implementation. |
| 162 | - Outbound calls to OpenAI or other model providers. |
| 163 | - Any errors raised from tools or your own business logic. |
| 164 | |
| 165 | |
| 166 | ## Security and authentication |
| 167 | |
| 168 | Production deployments should treat your ChatKit endpoint as a privileged backend: |
| 169 | |
| 170 | - **Authenticate every request** to your `/chatkit` endpoint (for example, with your existing session cookies, bearer tokens, or signed JWTs). |
| 171 | - **Authorize access to threads and attachments** based on your own user and tenant model. |
| 172 | - **Protect secrets** such as OpenAI API keys and internal service credentials in environment variables or a secret manager—never in source control. |
| 173 | - **Validate inputs** before calling tools or downstream systems. |
| 174 | |
| 175 | You should also be explicit about how you handle **prompt injection**: |
| 176 | |
| 177 | - Treat all user text, attachments, and tool output as untrusted input. |
| 178 | - Avoid building any `role="system"` model inputs from values that might come from the user (including fields like subject lines, titles, or descriptions). |
| 179 | - Keep system messages static or derived only from trusted configuration so users cannot silently change your instructions to the model. |
| 180 | |
| 181 | ### Authenticate your ChatKit endpoint |
| 182 | |
| 183 | The Python SDK expects your own app to handle authentication; `ChatKitServer` works with whatever `RequestContext` you choose. A common pattern is to: |
| 184 | |
| 185 | 1. Authenticate the incoming HTTP request using your web framework (session middleware, OAuth bearer tokens, etc.). |
| 186 | 2. Build a `RequestContext` that includes the authenticated user id, org/tenant, and any roles or scopes. |
| 187 | 3. Pass that context into `server.process`. |
| 188 | |
| 189 | For example: |
| 190 | |
| 191 | ```python |
| 192 | from fastapi import Depends, FastAPI, HTTPException, Request, Response |
| 193 | from fastapi.responses import StreamingResponse |
| 194 | |
| 195 | from chatkit.server import ChatKitServer, StreamingResult |
| 196 | |
| 197 | |
| 198 | def get_current_user(request: Request) -> str: |
| 199 | # Replace this with your real auth: session cookies, JWTs, etc. |
| 200 | user_id = request.headers.get("x-user-id") |
| 201 | if not user_id: |
| 202 | raise HTTPException(status_code=401, detail="Unauthorized") |
| 203 | return user_id |
| 204 | |
| 205 | |
| 206 | app = FastAPI() |
| 207 | store = MyStore(...) |
| 208 | server = MyChatKitServer(store) |
| 209 | |
| 210 | |
| 211 | @app.post("/chatkit") |
| 212 | async def chatkit_endpoint( |
| 213 | request: Request, |
| 214 | user_id: str = Depends(get_current_user), |
| 215 | ): |
| 216 | context = RequestContext(user_id=user_id, locale="en") |
| 217 | result = await server.process(await request.body(), context) |
| 218 | if isinstance(result, StreamingResult): |
| 219 | return StreamingResponse(result, media_type="text/event-stream") |
| 220 | return Response(content=result.json, media_type="application/json") |
| 221 | ``` |
| 222 | |
| 223 | Inside your `Store` and tools, enforce per‑user or per‑tenant access by checking `context.user_id` (and any other identifiers you include) before returning or mutating data. |
| 224 | |
| 225 | ### Handle PII and data retention |
| 226 | |
| 227 | Because ChatKit threads and items can contain user text, attachments, and tool output: |
| 228 | |
| 229 | - **Decide what you persist** and for how long. Implement retention policies in your `Store` (for example, delete threads older than N days). |
| 230 | - **Avoid storing unnecessary PII** in thread metadata or tool return values. |
| 231 | - **Encrypt data at rest** using your database’s built‑in features or application‑level encryption where needed. |
| 232 | |
| 233 | ## Domain keys |
| 234 | |
| 235 | Domain keys lock ChatKit down to the hostnames you control. When you embed ChatKit in a web app, the client and iframe can use a domain key to prove that the page is allowed to load ChatKit. |
| 236 | |
| 237 | 1. Visit the OpenAI domain allowlist page at `https://platform.openai.com/settings/organization/security/domain-allowlist`. |
| 238 | 2. Register each hostname that will host your ChatKit UI (for example, `app.example.com`, `support.example.com`). |
| 239 | 3. Copy the generated **domain key** for each entry. |
| 240 | |
| 241 | Your client configuration should include that `domainKey` alongside the URL to your ChatKit Python backend. |
| 242 | |
| 243 | ```ts |
| 244 | const options = { |
| 245 | api: { |
| 246 | url: "https://your-domain.com/api/chatkit", |
| 247 | // Copy this value from the domain allowlist entry. |
| 248 | domainKey: "your-domain-key", |
| 249 | }, |
| 250 | }; |
| 251 | ``` |
| 252 | |
| 253 | The ChatKit iframe will make an outbound request to `https://api.openai.com` to verify the domain key on load. If the key is missing or invalid, ChatKit will refuse to load, preventing unauthorized hostnames from embedding your ChatKit experience. |
| 254 | |
| 255 | When you go live, make sure all of your production hostnames are registered in the domain allowlist. |