GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

openai/openai-python

Public

mirrored from https://github.com/openai/openai-pythonAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
v0.28.1

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

chatml.md

96lines · modeblame

RawDownload
Latest commit unavailable.
120d225bLogan Kilpatrick2 years ago1> [!IMPORTANT]
2> This page is not currently maintained and is intended to provide general insight into the ChatML format, not current up-to-date information.
3
11656398Greg Brockman3 years ago4(This document is a preview of the underlying format consumed by
120d225bLogan Kilpatrick2 years ago5GPT models. As a developer, you can use our [higher-level
11656398Greg Brockman3 years ago6API](https://platform.openai.com/docs/guides/chat) and won't need to
81e624e8Kevin Huang3 years ago7interact directly with this format today — but expect to have the
11656398Greg Brockman3 years ago8option in the future!)
9
75c90a71Logan Kilpatrick3 years ago10Traditionally, GPT models consumed unstructured text. ChatGPT models
11instead expect a structured format, called Chat Markup Language
12(ChatML for short).
81ad94e7Kanta Yamaoka (山岡幹太)3 years ago13ChatML documents consist of a sequence of messages. Each message
75c90a71Logan Kilpatrick3 years ago14contains a header (which today consists of who said it, but in the
15future will contain other metadata) and contents (which today is a
16text payload, but in the future will contain other datatypes).
17We are still evolving ChatML, but the current version (ChatML v0) can
18be represented with our upcoming "list of dicts" JSON format as
19follows:
20```
21[
22{"token": "<|im_start|>"},
23"system\nYou are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.\nKnowledge cutoff: 2021-09-01\nCurrent date: 2023-03-01",
24{"token": "<|im_end|>"}, "\n", {"token": "<|im_start|>"},
25"user\nHow are you",
26{"token": "<|im_end|>"}, "\n", {"token": "<|im_start|>"},
27"assistant\nI am doing well!",
28{"token": "<|im_end|>"}, "\n", {"token": "<|im_start|>"},
29"user\nHow are you now?",
30{"token": "<|im_end|>"}, "\n"
31]
32```
33You could also represent it in the classic "unsafe raw string"
15c6aebcOwain Davies3 years ago34format. However, this format inherently allows injections from user
35input containing special-token syntax, similar to SQL injections:
75c90a71Logan Kilpatrick3 years ago36```
37<|im_start|>system
38You are ChatGPT, a large language model trained by OpenAI. Answer as concisely as possible.
39Knowledge cutoff: 2021-09-01
40Current date: 2023-03-01<|im_end|>
41<|im_start|>user
42How are you<|im_end|>
43<|im_start|>assistant
44I am doing well!<|im_end|>
45<|im_start|>user
46How are you now?<|im_end|>
47```
48## Non-chat use-cases
49ChatML can be applied to classic GPT use-cases that are not
50traditionally thought of as chat. For example, instruction following
51(where a user requests for the AI to complete an instruction) can be
52implemented as a ChatML query like the following:
53```
54[
55{"token": "<|im_start|>"},
56"user\nList off some good ideas:",
57{"token": "<|im_end|>"}, "\n", {"token": "<|im_start|>"},
58"assistant"
59]
60```
61We do not currently allow autocompleting of partial messages,
62```
63[
64{"token": "<|im_start|>"},
237448dcliuliu3 years ago65"system\nPlease autocomplete the user's message.",
75c90a71Logan Kilpatrick3 years ago66{"token": "<|im_end|>"}, "\n", {"token": "<|im_start|>"},
67"user\nThis morning I decided to eat a giant"
68]
69```
70Note that ChatML makes explicit to the model the source of each piece
71of text, and particularly shows the boundary between human and AI
72text. This gives an opportunity to mitigate and eventually solve
73injections, as the model can tell which instructions come from the
74developer, the user, or its own input.
75## Few-shot prompting
76In general, we recommend adding few-shot examples using separate
77`system` messages with a `name` field of `example_user` or
78`example_assistant`. For example, here is a 1-shot prompt:
79```
80<|im_start|>system
81Translate from English to French
82<|im_end|>
83<|im_start|>system name=example_user
84How are you?
85<|im_end|>
86<|im_start|>system name=example_assistant
87Comment allez-vous?
88<|im_end|>
89<|im_start|>user
90{{user input here}}<|im_end|>
91```
92If adding instructions in the `system` message doesn't work, you can
237448dcliuliu3 years ago93also try putting them into a `user` message. (In the near future, we
75c90a71Logan Kilpatrick3 years ago94will train our models to be much more steerable via the system
95message. But to date, we have trained only on a few system messages,
d008c3b7David Thayer3 years ago96so the models pay much more attention to user examples.)