GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

openai/chatkit-python

Public

mirrored fromhttps://github.com/openai/chatkit-pythonAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
v1.6.4

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

chatkit-python/docs/guides

docs/guides/pass-extra-app-context-to-your-model.md

166lines · modecode

RawDownload
Latest commit unavailable.
1# Pass extra app context to your model
2
3Sometimes the model needs information that is not part of the thread: the current route, user plan, selected document, feature flags, or host-app state. This guide shows several patterns for passing that extra context into your inference pipeline.
4
5At a high level:
6
7- Send app/user context from the client to your ChatKit server on every request.
8- Fetch context on demand with tools (including client tools).
9- Inject extra context as an additional model input item when you build the request.
10
11## Send app context with each request
12
13### Attach headers from `useChatKit`
14
15Use a custom `fetch` (or equivalent) when configuring `useChatKit` to attach app/user context via headers to every request:
16
17```tsx
18const chatkit = useChatKit({
19 api: {
20 // ...
21 fetch: (input, init) =>
22 fetch(input, {
23 ...init,
24 headers: {
25 // Make sure to pipe through headers sent by ChatKit
26 ...(init?.headers || {}),
27 "X-Org-Id": currentOrgId,
28 "X-Plan": currentPlan,
29 },
30 }),
31 },
32});
33```
34
35On the server, read these headers before calling `ChatKitServer.process` and add them to your request context:
36
37```python
38from dataclasses import dataclass
39
40
41@dataclass
42class RequestContext:
43 user_id: str
44 org_id: str
45 plan: str
46```
47
48Use this context in your `respond` method, tools, and store methods to keep the model and your business logic aware of the current app state.
49
50## Fetch context on demand with tools
51
52Sometimes you only need extra context for certain requests—fetch it on demand with tools instead of sending it for every turn.
53
54### Server tools that fetch app context
55
56Define a server tool that looks up app state (for example, the user’s current workspace or preferences) and returns a JSON payload to the model:
57
58```python
59@function_tool(description_override="Fetch the user's workspace context.")
60async def get_workspace_context(ctx: RunContextWrapper[AgentContext]):
61 workspace = await load_workspace(ctx.context.request_context.org_id)
62 return {
63 "workspace_id": workspace.id,
64 "features": workspace.feature_flags,
65 }
66```
67
68Include this tool in your agent so the model can call it when it needs that information.
69
70### Client tools for browser/app-only state
71
72When the context only exists on the client (selection, viewport, local app state), use a client tool:
73
74```python
75@function_tool(description_override="Read the user's current canvas selection.")
76async def get_canvas_selection(ctx: RunContextWrapper[AgentContext]) -> None:
77 ctx.context.client_tool_call = ClientToolCall(
78 name="get_canvas_selection",
79 arguments={},
80 )
81```
82
83On the client, implement the callback:
84
85```ts
86const chatkit = useChatKit({
87 // ...
88 onClientTool: async ({name, params}) => {
89 if (name === "get_canvas_selection") {
90 const selection = myCanvas.getSelection();
91 return {
92 nodes: selection.map((node) => {
93 name: node.name,
94 description: node.description,
95 }),
96 };
97 }
98 },
99});
100```
101
102ChatKit will send the client tool result back to the server and continue the run with that data included as model input.
103
104## Inject extra context as model input item
105
106You can also inject context directly as an extra model input item when you build the request.
107
108### Add a dedicated context item
109
110Before running your agent, prepend a short, structured context item describing app/user state:
111
112```python
113from openai.types.responses import ResponseInputTextParam
114
115
116extra_context = ResponseInputTextParam(
117 type="input_text",
118 text=(
119 "<APP_CONTEXT>\n"
120 f"user_id: {context.user_id}\n"
121 f"org_id: {context.org_id}\n"
122 f"plan: {context.plan}\n"
123 "</APP_CONTEXT>"
124 ),
125)
126
127input_items = [extra_context, *input_items]
128```
129
130Pair this with a short system prompt telling the model how to interpret the `<APP_CONTEXT>` block.
131
132### Combine ids + tools
133
134A useful pattern is to combine a lightweight context item with a follow-up tool call:
135
1361. Add an input item that contains a stable id or handle:
137
138 ```python
139 extra_context = ResponseInputTextParam(
140 type="input_text",
141 text=f"<WORKSPACE_REF id={workspace.id} />",
142 )
143 input_items = [extra_context, *input_items]
144 ```
145
1462. Provide a tool (server or client) that can fetch the full details when needed:
147
148 ```python
149 @function_tool(description_override="Fetch full workspace details.")
150 async def fetch_workspace(ctx: RunContextWrapper[AgentContext], id: str):
151 workspace = await load_workspace(id)
152 return {
153 "id": workspace.id,
154 "features": workspace.feature_flags,
155 "limits": workspace.limits,
156 }
157 ```
158
1593. In your prompt, tell the model:
160
161 - The `<WORKSPACE_REF>` tag carries the id it should use.
162 - It should call `fetch_workspace` when it needs more detail instead of guessing.
163
164This keeps your model inputs compact while still giving the model a reliable way to pull detailed context on demand.
165
166
167