openai/chatkit-python
Publicmirrored from https://github.com/openai/chatkit-pythonAvailable
docs/actions.md
264lines · modeblame
f688d870victor-openai8 months ago | 1 | # ChatKit actions |
| 2 | | |
| 3 | Actions are a way for the ChatKit SDK frontend to trigger a streaming response without the user submitting a message. They can also be used to trigger side-effects outside ChatKit SDK. | |
| 4 | | |
| 5 | ## Triggering actions | |
| 6 | | |
| 7 | ### In response to user interaction with widgets | |
| 8 | | |
| 9 | Actions can be triggered by attaching an `ActionConfig` to any widget node that supports it. For example, you can respond to click events on Buttons. When a user clicks on this button, the action will be sent to your server where you can update the widget, run inference, stream new thread items, etc. | |
| 10 | | |
| 11 | ```python | |
| 12 | Button( | |
| 13 | label="Example", | |
| 14 | onClickAction=ActionConfig( | |
| 15 | type="example", | |
| 16 | payload={"id": 123}, | |
| 17 | ) | |
| 18 | ) | |
| 19 | ``` | |
| 20 | | |
| 21 | Actions can also be sent imperatively by your frontend with `sendAction()`. This is probably most useful when you need ChatKit to respond to interaction happening outside ChatKit, but it can also be used to chain actions when you need to respond on both the client and the server (more on that below). | |
| 22 | | |
| 23 | ```tsx | |
| 24 | await chatKit.sendAction({ | |
| 25 | type: "example", | |
| 26 | payload: { id: 123 }, | |
| 27 | }); | |
| 28 | ``` | |
| 29 | | |
| 30 | ## Handling actions | |
| 31 | | |
| 32 | ### On the server | |
| 33 | | |
| 34 | By default, actions are sent to your server. You can handle actions on your server by implementing the `action` method on `ChatKitServer`. | |
| 35 | | |
| 36 | ```python | |
| 37 | from collections.abc import AsyncIterator | |
| 38 | from datetime import datetime | |
| 39 | from typing import Any | |
| 40 | | |
| 41 | from chatkit.actions import Action | |
| 42 | from chatkit.server import ChatKitServer | |
| 43 | from chatkit.types import ( | |
| 44 | HiddenContextItem, | |
| 45 | ThreadItemDoneEvent, | |
| 46 | ThreadMetadata, | |
| 47 | ThreadStreamEvent, | |
| 48 | WidgetItem, | |
| 49 | ) | |
| 50 | | |
| 51 | RequestContext = dict[str, Any] | |
| 52 | | |
| 53 | | |
| 54 | class MyChatKitServer(ChatKitServer[RequestContext]): | |
| 55 | async def action( | |
| 56 | self, | |
| 57 | thread: ThreadMetadata, | |
| 58 | action: Action[str, Any], | |
| 59 | sender: WidgetItem | None, | |
| 60 | context: RequestContext, | |
| 61 | ) -> AsyncIterator[ThreadStreamEvent]: | |
| 62 | if action.type == "example": | |
| 63 | await do_thing(action.payload['id']) | |
| 64 | | |
| 65 | # often you'll want to add a HiddenContextItem so the model | |
| 66 | # can see that the user did something | |
| 67 | hidden = HiddenContextItem( | |
| 68 | id=self.store.generate_item_id("message", thread, context), | |
| 69 | thread_id=thread.id, | |
| 70 | created_at=datetime.now(), | |
| 71 | content=["<USER_ACTION>The user did a thing</USER_ACTION>"], | |
| 72 | ) | |
| 73 | await self.store.add_thread_item(thread.id, hidden, context) | |
| 74 | | |
| 75 | # then you might want to run inference to stream a response | |
| 76 | # back to the user. | |
| 77 | async for e in self.generate(context, thread): | |
| 78 | yield e | |
| 79 | | |
| 80 | if action.type == "another.example" | |
| 81 | # ... | |
| 82 | ``` | |
| 83 | | |
| 84 | **NOTE:** As with any client/server interaction, actions and their payloads are sent by the client and should be treated as untrusted data. | |
| 85 | | |
| 86 | ### Client | |
| 87 | | |
| 88 | Sometimes you’ll want to handle actions in your client integration. To do that you need to specify that the action should be sent to your client-side action handler by adding `handler="client` to the `ActionConfig`. | |
| 89 | | |
| 90 | ```python | |
| 91 | Button( | |
| 92 | label="Example", | |
| 93 | onClickAction=ActionConfig( | |
| 94 | type="example", | |
| 95 | payload={"id": 123}, | |
| 96 | handler="client" | |
| 97 | ) | |
| 98 | ) | |
| 99 | ``` | |
| 100 | | |
| 101 | Then, when the action is triggered, it will then be passed to a callback that you provide when instantiating ChatKit. | |
| 102 | | |
| 103 | ```ts | |
| 104 | async function handleWidgetAction(action: {type: string, Record<string, unknown>}) { | |
| 105 | if (action.type === "example") { | |
| 106 | const res = await doSomething(action) | |
| 107 | | |
| 108 | // You can fire off actions to your server from here as well. | |
| 109 | // e.g. if you want to stream new thread items or update a widget. | |
| 110 | await chatKit.sendAction({ | |
| 111 | type: "example_complete", | |
| 112 | payload: res | |
| 113 | }) | |
| 114 | } | |
| 115 | } | |
| 116 | | |
| 117 | chatKit.setOptions({ | |
| 118 | // other options... | |
| 119 | widgets: { onAction: handleWidgetAction } | |
| 120 | }) | |
| 121 | ``` | |
| 122 | | |
| 123 | ## Strongly typed actions | |
| 124 | | |
| 125 | By default `Action` and `ActionConfig` are not strongly typed. However, we do expose a `create` helper on `Action` making it easy to generate `ActionConfig`s from a set of strongly-typed actions. | |
| 126 | | |
| 127 | ```python | |
| 128 | | |
| 129 | class ExamplePayload(BaseModel) | |
| 130 | id: int | |
| 131 | | |
| 132 | ExampleAction = Action[Literal["example"], ExamplePayload] | |
| 133 | OtherAction = Action[Literal["other"], None] | |
| 134 | | |
| 135 | AppAction = Annotated[ | |
| 136 | ExampleAction | |
| 137 | | OtherAction, | |
| 138 | Field(discriminator="type"), | |
| 139 | ] | |
| 140 | | |
| 141 | ActionAdapter: TypeAdapter[AppAction] = TypeAdapter(AppAction) | |
| 142 | | |
| 143 | def parse_app_action(action: Action[str, Any]): AppAction | |
| 144 | return ActionAdapter.validate_python(action) | |
| 145 | | |
| 146 | # Usage in a widget | |
| 147 | # Action provides a create helper which makes it easy to generate | |
| 148 | # ActionConfigs from strongly typed actions. | |
| 149 | Button( | |
| 150 | label="Example", | |
| 151 | onClickAction=ExampleAction.create(ExamplePayload(id=123)) | |
| 152 | ) | |
| 153 | | |
| 154 | # usage in action handler | |
| 155 | class MyChatKitServer(ChatKitServer[RequestContext]) | |
| 156 | async def action( | |
| 157 | self, | |
| 158 | thread: ThreadMetadata, | |
| 159 | action: Action[str, Any], | |
| 160 | sender: WidgetItem | None, | |
| 161 | context: RequestContext, | |
| 162 | ) -> AsyncIterator[Event]: | |
| 163 | # add custom error handling if needed | |
| 164 | app_action = parse_app_action(action) | |
| 165 | if (app_action.type == "example"): | |
| 166 | await do_thing(app_action.payload.id) | |
| 167 | ``` | |
| 168 | | |
| 169 | ## Use widgets and actions to create custom forms | |
| 170 | | |
| 171 | When widget nodes that take user input are mounted inside a `Form`, the values from those fields will be included in the `payload` of all actions that originate from within the `Form`. | |
| 172 | | |
| 173 | Form values are keyed in the `payload` by their `name` e.g. | |
| 174 | | |
| 175 | - `Select(name="title")` → `action.payload.title` | |
| 176 | - `Select(name="todo.title")` → `action.payload.todo.title` | |
| 177 | | |
| 178 | ```python | |
| 179 | Form( | |
| 180 | direction="col", | |
| 181 | onSubmitAction=ActionConfig( | |
| 182 | type="update_todo", | |
| 183 | payload={"id": todo.id} | |
| 184 | ), | |
| 185 | children=[ | |
| 186 | Title(value="Edit Todo"), | |
| 187 | | |
| 188 | Text(value="Title", color="secondary", size="sm"), | |
| 189 | Text( | |
| 190 | value=todo.title, | |
| 191 | editable=EditableProps(name="title", required=True), | |
| 192 | ) | |
| 193 | | |
| 194 | Text(value="Description", color="secondary", size="sm"), | |
| 195 | Text( | |
| 196 | value=todo.description, | |
| 197 | editable=EditableProps(name="description"), | |
| 198 | ), | |
| 199 | | |
| 200 | Button(label="Save", submit=true) | |
| 201 | ] | |
| 202 | ) | |
| 203 | | |
| 204 | class MyChatKitServer(ChatKitServer[RequestContext]) | |
| 205 | async def action( | |
| 206 | self, | |
| 207 | thread: ThreadMetadata, | |
| 208 | action: Action[str, Any], | |
| 209 | sender: WidgetItem | None, | |
| 210 | context: RequestContext, | |
| 211 | ) -> AsyncIterator[Event]: | |
| 212 | if (action.type == "update_todo"): | |
| 213 | id = action.payload['id'] | |
| 214 | # Any action that originates from within the Form will | |
| 215 | # include title and description | |
| 216 | title = action.payload['title'] | |
| 217 | description = action.payload['description'] | |
| 218 | | |
| 219 | # ... | |
| 220 | | |
| 221 | ``` | |
| 222 | | |
| 223 | ### Validation | |
| 224 | | |
| 225 | `Form` uses basic native form validation; enforcing `required` and `pattern` on fields where they are configured and blocking submission when the form has any invalid field. | |
| 226 | | |
| 227 | We may add new validation modes with better UX, more expressive validation, custom error display, etc in the future. Until then, widgets are not a great medium for complex forms with tricky validation. If you have this need, a better pattern would be to use client side action handling to trigger a modal, show a custom form there, then pass the result back into ChatKit with `sendAction`. | |
| 228 | | |
| 229 | ### Treating `Card` as a `Form` | |
| 230 | | |
| 231 | You can pass `asForm=True` to `Card` and it will behave as a `Form`, running validation and passing collected fields to the Card’s `confirm` action. | |
| 232 | | |
| 233 | ### Payload key collisions | |
| 234 | | |
| 235 | If there is a naming collision with some other existing pre-defined key on your payload, the form value will be ignored. This is probably a bug, so we’ll emit an `error` event when we see this. | |
| 236 | | |
| 237 | ## Customize how actions interact with loading states in widgets | |
| 238 | | |
| 239 | Use `ActionConfig.loadingBehavior` to control how actions trigger different loading states in a widget. | |
| 240 | | |
| 241 | ```python | |
| 242 | Button( | |
| 243 | label="This make take a while...", | |
| 244 | onClickAction=ActionConfig( | |
| 245 | type="long_running_action_that_should_block_other_ui_interactions", | |
| 246 | loadingBehavior="container" | |
| 247 | ) | |
| 248 | ) | |
| 249 | ``` | |
| 250 | | |
| 251 | | Value | Behavior | | |
| 252 | | ----------- | ------------------------------------------------------------------------------------------------------------------------------- | | |
| 253 | | `auto` | The action will adapt to how it’s being used. (_default_) | | |
| 254 | | `self` | The action triggers loading state on the widget node that the action was bound to. | | |
| 255 | | `container` | The action triggers loading state on the entire widget container. This causes the widget to fade out slightly and become inert. | | |
| 256 | | `none` | No loading state | | |
| 257 | | |
| 258 | ### Using `auto` behavior | |
| 259 | | |
| 260 | Generally, we recommend using `auto`, which is the default. `auto` triggers loading states based on where the action is bound, for example: | |
| 261 | | |
| 262 | - `Button.onClickAction` → `self` | |
| 263 | - `Select.onChangeAction` → `none` | |
| 264 | - `Card.confirm.action` → `container` |