GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

openai/chatkit-python

Public

mirrored from https://github.com/openai/chatkit-pythonAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
v1.6.3

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

chatkit-python/docs/guides

docs/guides/add-annotations.md

161lines · modeblame

RawDownload
Latest commit unavailable.
2e1209a2Jiwon Kim6 months ago1# Add annotations in assistant messages
2
3ChatKit renders clickable inline citations when assistant text includes `annotations` and rolls every reference into a collapsed **Sources** list beneath each message. You can let the model emit annotations directly or attach sources yourself before streaming the message.
4
5## Use model-emitted citations
6
7When you stream a Responses run through `stream_agent_response`, ChatKit automatically converts any `file_citation`, `container_file_citation`, and `url_citation` annotations returned by the OpenAI API into ChatKit `Annotation` objects and attaches them to streamed message content.
8
9Provide the model with citable evidence via tools to receive citation annotations, most commonly:
10
11- `FileSearchTool` for uploaded documents (emits `file_citation` / `container_file_citation`)
12- `WebSearchTool` for live URLs (emits `url_citation`)
13
14No additional server-side wiring is required beyond calling `stream_agent_response`. If the model emits citation annotations from tool usage, ChatKit will forward them automatically as `Annotation` objects on the corresponding content parts.
15
2f23b675Jiwon Kim5 months ago16### Customize how citations are converted
17
18Sometimes the default rendering for model-emitted citations is not very helpful. For example, file citations may not include enough metadata for ChatKit to show a meaningful default title or description. You can pass a custom [`ResponseStreamConverter`](../../api/chatkit/agents/#chatkit.agents.ResponseStreamConverter) and override:
19
20- `file_citation_to_annotation`
21- `container_file_citation_to_annotation`
22- `url_citation_to_annotation`
23
24Here is a minimal example that enriches file citations with a more helpful title/description using an internal mapping:
25
26```python
27from chatkit.agents import ResponseStreamConverter, stream_agent_response
28from chatkit.types import Annotation, FileSource
29
30
31class MyResponseStreamConverter(ResponseStreamConverter):
32def __init__(self, file_lookup: dict[str, dict[str, str]]):
33super().__init__()
34self._file_lookup = file_lookup
35
36async def file_citation_to_annotation(self, citation):
37info = self._file_lookup.get(citation.file_id, {})
38return Annotation(
39source=FileSource(
40filename=info.get("filename", citation.file_id),
41title=info.get("title"),
42description=info.get("description"),
43),
44index=citation.index,
45)
46
47
48converter = MyResponseStreamConverter(
49file_lookup={
50"file_123": {
51"filename": "q1_report.pdf",
52"title": "Q1 Report",
53"description": "Quarterly performance summary",
54}
55}
56)
57
58stream_agent_response(..., converter=converter)
59```
60
61You can also return an `EntitySource` instead of a `FileSource` to control the inline label, handle clicks, and customize the popover preview. For more on entity annotations (including `interactive` click/preview hooks), see [Annotating with custom entities](#annotating-with-custom-entities) below.
62
2e1209a2Jiwon Kim6 months ago63
64## Attach sources manually
65
66If you build assistant messages yourself, include annotations on each `AssistantMessageContent` item.
67
68```python
69from datetime import datetime
70from chatkit.types import (
71Annotation,
72AssistantMessageContent,
73AssistantMessageItem,
74FileSource,
75ThreadItemDoneEvent,
76URLSource,
77)
78
79text = "Quarterly revenue grew 12% year over year."
80annotations = [
81Annotation(
82source=FileSource(filename="q1_report.pdf", title="Q1 Report"),
83index=len(text) - 1, # attach near the end of the sentence
84),
85Annotation(
86source=URLSource(
87url="https://example.com/press-release",
88title="Press release",
89),
90index=len(text) - 1,
91),
92]
93
94yield ThreadItemDoneEvent(
95item=AssistantMessageItem(
96id=self.store.generate_item_id("message", thread, context),
97thread_id=thread.id,
98created_at=datetime.now(),
99content=[AssistantMessageContent(text=text, annotations=annotations)],
100)
101)
102```
103
104`index` is the character position to place the footnote marker; re-use the same index when multiple citations support the same claim so the footnote numbers stay grouped.
105
106## Annotating with custom entities
107
6646f83aJiwon Kim6 months ago108You can attach `EntitySource` items as annotations to show entity references inline in assistant text and in the **Sources** list below the message.
109
110Entity annotations support a few UI-focused fields:
111
112- `icon`: Controls the icon shown for the entity in the default inline/hover UI.
113- `label`: Customizes what's shown in the default entity hover header (when you are not rendering a custom preview).
114- `inline_label`: Shows a label inline instead of an icon.
115- `interactive=True`: Wires the annotation to client-side callbacks (`ChatKitOptions.entities.onClick` and `ChatKitOptions.entities.onRequestPreview`).
2e1209a2Jiwon Kim6 months ago116
117```python
118from datetime import datetime
119from chatkit.types import (
120Annotation,
121AssistantMessageContent,
122AssistantMessageItem,
123EntitySource,
124ThreadItemDoneEvent,
125)
126
6646f83aJiwon Kim6 months ago127text = "Here are the ACME account details for reference."
128
2e1209a2Jiwon Kim6 months ago129annotations = [
130Annotation(
131source=EntitySource(
132id="customer_123",
133title="ACME Corp",
134description="Enterprise plan · 500 seats",
135icon="suitcase",
6646f83aJiwon Kim6 months ago136label="Customer",
137interactive=True,
138# Free-form data object passed to your client-side entity callbacks
2e1209a2Jiwon Kim6 months ago139data={"href": "https://crm.example.com/customers/123"},
6646f83aJiwon Kim6 months ago140),
141# `index` controls where the inline marker is placed in the text.
142index=text.index("ACME") + len("ACME"),
2e1209a2Jiwon Kim6 months ago143)
144]
145
146yield ThreadItemDoneEvent(
147item=AssistantMessageItem(
148id=self.store.generate_item_id("message", thread, context),
149thread_id=thread.id,
150created_at=datetime.now(),
151content=[
152AssistantMessageContent(
6646f83aJiwon Kim6 months ago153text=text,
2e1209a2Jiwon Kim6 months ago154annotations=annotations,
155)
156],
157)
158)
159```
160
6646f83aJiwon Kim6 months ago161Provide richer previews and navigation by handling [`entities.onRequestPreview`](https://openai.github.io/chatkit-js/api/openai/chatkit/type-aliases/entitiesoption/#onrequestpreview) and [`entities.onClick`](https://openai.github.io/chatkit-js/api/openai/chatkit/type-aliases/entitiesoption/#onclick) in ChatKit.js. These callbacks are only invoked for entity annotations with `interactive=True`; use the `data` payload to pass entity information and deep link into your app.