GitVita
New repositoryImport repositoryNew issue
Your repositories Your dashboard Sign in

openai/openai-python

Public

mirrored from https://github.com/openai/openai-pythonAvailable

Watch0Fork0Star0
CodeCommitsIssuesPull requestsActionsInsightsSecurity
v0.11.3

Branches

Tags

  • No tags available.
0Branches0Tags
Go to file
Add file
Code

Clone

HTTPS

Download ZIP

openai-python/examples/finetuning

examples/finetuning/finetuning-classification.ipynb

740lines · modeblame

RawDownload
Latest commit unavailable.
c79fefcaBoris Power4 years ago1{
2"cells": [
3{
4"cell_type": "markdown",
5"source": [
6"# Fine tuning classification example\n",
7"\n",
8"We will fine-tune an ada classifier to distinguish between the two sports: Baseball and Hockey."
9],
10"metadata": {}
11},
12{
13"cell_type": "code",
7febb755Boris Power4 years ago14"execution_count": 1,
c79fefcaBoris Power4 years ago15"source": [
16"from sklearn.datasets import fetch_20newsgroups\n",
17"import pandas as pd\n",
18"import openai\n",
19"\n",
20"categories = ['rec.sport.baseball', 'rec.sport.hockey']\n",
21"sports_dataset = fetch_20newsgroups(subset='train', shuffle=True, random_state=42, categories=categories)"
22],
23"outputs": [],
24"metadata": {}
25},
26{
27"cell_type": "markdown",
28"source": [
29" ## Data exploration\n",
30" The newsgroup dataset can be loaded using sklearn. First we will look at the data itself:"
31],
32"metadata": {}
33},
34{
35"cell_type": "code",
7febb755Boris Power4 years ago36"execution_count": 2,
c79fefcaBoris Power4 years ago37"source": [
38"print(sports_dataset['data'][0])"
39],
40"outputs": [
41{
42"output_type": "stream",
43"name": "stdout",
44"text": [
45"From: dougb@comm.mot.com (Doug Bank)\n",
46"Subject: Re: Info needed for Cleveland tickets\n",
47"Reply-To: dougb@ecs.comm.mot.com\n",
48"Organization: Motorola Land Mobile Products Sector\n",
49"Distribution: usa\n",
50"Nntp-Posting-Host: 145.1.146.35\n",
51"Lines: 17\n",
52"\n",
53"In article <1993Apr1.234031.4950@leland.Stanford.EDU>, bohnert@leland.Stanford.EDU (matthew bohnert) writes:\n",
54"\n",
55"|> I'm going to be in Cleveland Thursday, April 15 to Sunday, April 18.\n",
56"|> Does anybody know if the Tribe will be in town on those dates, and\n",
57"|> if so, who're they playing and if tickets are available?\n",
58"\n",
59"The tribe will be in town from April 16 to the 19th.\n",
60"There are ALWAYS tickets available! (Though they are playing Toronto,\n",
61"and many Toronto fans make the trip to Cleveland as it is easier to\n",
62"get tickets in Cleveland than in Toronto. Either way, I seriously\n",
63"doubt they will sell out until the end of the season.)\n",
64"\n",
65"-- \n",
66"Doug Bank Private Systems Division\n",
67"dougb@ecs.comm.mot.com Motorola Communications Sector\n",
68"dougb@nwu.edu Schaumburg, Illinois\n",
69"dougb@casbah.acns.nwu.edu 708-576-8207 \n",
70"\n"
71]
72}
73],
74"metadata": {}
75},
76{
77"cell_type": "code",
7febb755Boris Power4 years ago78"execution_count": 3,
c79fefcaBoris Power4 years ago79"source": [
80"sports_dataset.target_names[sports_dataset['target'][0]]\n"
81],
82"outputs": [
83{
84"output_type": "execute_result",
85"data": {
86"text/plain": [
87"'rec.sport.baseball'"
88]
89},
90"metadata": {},
7febb755Boris Power4 years ago91"execution_count": 3
c79fefcaBoris Power4 years ago92}
93],
94"metadata": {}
95},
96{
97"cell_type": "code",
7febb755Boris Power4 years ago98"execution_count": 4,
c79fefcaBoris Power4 years ago99"source": [
100"len_all, len_baseball, len_hockey = len(sports_dataset.data), len([e for e in sports_dataset.target if e == 0]), len([e for e in sports_dataset.target if e == 1])\n",
101"print(f\"Total examples: {len_all}, Baseball examples: {len_baseball}, Hockey examples: {len_hockey}\")"
102],
103"outputs": [
104{
105"output_type": "stream",
106"name": "stdout",
107"text": [
108"Total examples: 1197, Baseball examples: 597, Hockey examples: 600\n"
109]
110}
111],
112"metadata": {}
113},
114{
115"cell_type": "markdown",
116"source": [
117"One sample from the baseball category can be seen above. It is an email to a mailing list. We can observe that we have 1197 examples in total, which are evenly split between the two sports."
118],
119"metadata": {}
120},
121{
122"cell_type": "markdown",
123"source": [
124"## Data Preparation\n",
125"We transform the dataset into a pandas dataframe, with a column for prompt and completion. The prompt contains the email from the mailing list, and the completion is a name of the sport, either hockey or baseball. For demonstration purposes only and speed of fine-tuning we take only 300 examples. In a real use case the more examples the better the performance."
126],
127"metadata": {}
128},
129{
130"cell_type": "code",
7febb755Boris Power4 years ago131"execution_count": 5,
c79fefcaBoris Power4 years ago132"source": [
133"import pandas as pd\n",
134"\n",
135"labels = [sports_dataset.target_names[x].split('.')[-1] for x in sports_dataset['target']]\n",
136"texts = [text.strip() for text in sports_dataset['data']]\n",
137"df = pd.DataFrame(zip(texts, labels), columns = ['prompt','completion']) #[:300]\n",
138"df.head()"
139],
140"outputs": [
141{
142"output_type": "execute_result",
143"data": {
144"text/plain": [
145" prompt completion\n",
146"0 From: dougb@comm.mot.com (Doug Bank)\\nSubject:... baseball\n",
147"1 From: gld@cunixb.cc.columbia.edu (Gary L Dare)... hockey\n",
148"2 From: rudy@netcom.com (Rudy Wade)\\nSubject: Re... baseball\n",
149"3 From: monack@helium.gas.uug.arizona.edu (david... hockey\n",
150"4 Subject: Let it be Known\\nFrom: <ISSBTL@BYUVM.... baseball"
151],
152"text/html": [
153"<div>\n",
154"<style scoped>\n",
155" .dataframe tbody tr th:only-of-type {\n",
156" vertical-align: middle;\n",
157" }\n",
158"\n",
159" .dataframe tbody tr th {\n",
160" vertical-align: top;\n",
161" }\n",
162"\n",
163" .dataframe thead th {\n",
164" text-align: right;\n",
165" }\n",
166"</style>\n",
167"<table border=\"1\" class=\"dataframe\">\n",
168" <thead>\n",
169" <tr style=\"text-align: right;\">\n",
170" <th></th>\n",
171" <th>prompt</th>\n",
172" <th>completion</th>\n",
173" </tr>\n",
174" </thead>\n",
175" <tbody>\n",
176" <tr>\n",
177" <th>0</th>\n",
178" <td>From: dougb@comm.mot.com (Doug Bank)\\nSubject:...</td>\n",
179" <td>baseball</td>\n",
180" </tr>\n",
181" <tr>\n",
182" <th>1</th>\n",
183" <td>From: gld@cunixb.cc.columbia.edu (Gary L Dare)...</td>\n",
184" <td>hockey</td>\n",
185" </tr>\n",
186" <tr>\n",
187" <th>2</th>\n",
188" <td>From: rudy@netcom.com (Rudy Wade)\\nSubject: Re...</td>\n",
189" <td>baseball</td>\n",
190" </tr>\n",
191" <tr>\n",
192" <th>3</th>\n",
193" <td>From: monack@helium.gas.uug.arizona.edu (david...</td>\n",
194" <td>hockey</td>\n",
195" </tr>\n",
196" <tr>\n",
197" <th>4</th>\n",
198" <td>Subject: Let it be Known\\nFrom: &lt;ISSBTL@BYUVM....</td>\n",
199" <td>baseball</td>\n",
200" </tr>\n",
201" </tbody>\n",
202"</table>\n",
203"</div>"
204]
205},
206"metadata": {},
7febb755Boris Power4 years ago207"execution_count": 5
c79fefcaBoris Power4 years ago208}
209],
210"metadata": {}
211},
212{
213"cell_type": "markdown",
214"source": [
215"Both baseball and hockey are single tokens. We save the dataset as a jsonl file."
216],
217"metadata": {}
218},
219{
220"cell_type": "code",
7febb755Boris Power4 years ago221"execution_count": 6,
c79fefcaBoris Power4 years ago222"source": [
7febb755Boris Power4 years ago223"df.to_json(\"sport2.jsonl\", orient='records', lines=True)"
c79fefcaBoris Power4 years ago224],
225"outputs": [],
226"metadata": {}
227},
228{
229"cell_type": "markdown",
230"source": [
231"### Data Preparation tool\n",
232"We can now use a data preparation tool which will suggest a few improvements to our dataset before fine-tuning. Before launching the tool we update the openai library to ensure we're using the latest data preparation tool. We additionally specify `-q` which auto-accepts all suggestions."
233],
234"metadata": {}
235},
236{
237"cell_type": "code",
7febb755Boris Power4 years ago238"execution_count": 7,
c79fefcaBoris Power4 years ago239"source": [
240"!pip install --upgrade openai"
241],
242"outputs": [],
243"metadata": {}
244},
245{
246"cell_type": "code",
7febb755Boris Power4 years ago247"execution_count": 8,
c79fefcaBoris Power4 years ago248"source": [
7febb755Boris Power4 years ago249"!openai tools fine_tunes.prepare_data -f sport2.jsonl -q"
c79fefcaBoris Power4 years ago250],
251"outputs": [
252{
253"output_type": "stream",
254"name": "stdout",
255"text": [
256"Analyzing...\n",
257"\n",
258"- Your file contains 1197 prompt-completion pairs\n",
259"- Based on your data it seems like you're trying to fine-tune a model for classification\n",
260"- For classification, we recommend you try one of the faster and cheaper models, such as `ada`. You should also set the `--no_packing` parameter when fine-tuning\n",
261"- For classification, you can estimate the expected model performance by keeping a held out dataset, which is not used for training\n",
7febb755Boris Power4 years ago262"- There are 11 examples that are very long. These are rows: [134, 200, 281, 320, 404, 595, 704, 838, 1113, 1139, 1174]\n",
263"For conditional generation, and for classification the examples shouldn't be longer than 2048 tokens.\n",
c79fefcaBoris Power4 years ago264"- Your data does not contain a common separator at the end of your prompts. Having a separator string appended to the end of the prompt makes it clearer to the fine-tuned model where the completion should begin. See https://beta.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more detail and examples. If you intend to do open-ended generation, then you should leave the prompts empty\n",
265"- The completion should start with a whitespace character (` `). This tends to produce better results due to the tokenization we use. See https://beta.openai.com/docs/guides/fine-tuning/preparing-your-dataset for more details\n",
266"\n",
267"Based on the analysis we will perform the following actions:\n",
7febb755Boris Power4 years ago268"- [Recommended] Remove 11 long examples [Y/n]: Y\n",
269"- [Recommended] Add a suffix separator `\\n\\n###\\n\\n` to all prompts [Y/n]: Y\n",
270"- [Recommended] Add a whitespace character to the beginning of the completion [Y/n]: Y\n",
271"- [Recommended] Would you like to split into training and validation set? [Y/n]: Y\n",
272"\n",
c79fefcaBoris Power4 years ago273"\n",
274"Your data will be written to a new JSONL file. Proceed [Y/n]: Y\n",
7febb755Boris Power4 years ago275"\n",
276"Wrote modified files to `sport2_prepared_train.jsonl` and `sport2_prepared_valid.jsonl`\n",
c79fefcaBoris Power4 years ago277"Feel free to take a look!\n",
278"\n",
279"Now use that file when fine-tuning:\n",
7febb755Boris Power4 years ago280"> openai api fine_tunes.create -t \"sport2_prepared_train.jsonl\" -v \"sport2_prepared_valid.jsonl\" --no_packing --compute_classification_metrics --classification_positive_class \" baseball\"\n",
c79fefcaBoris Power4 years ago281"\n",
282"After you’ve fine-tuned a model, remember that your prompt has to end with the indicator string `\\n\\n###\\n\\n` for the model to start generating completions, rather than continuing with the prompt.\n",
7febb755Boris Power4 years ago283"Once your model starts training, it'll approximately take 30.8 minutes to train a `curie` model, and less for `ada` and `babbage`. Queue will approximately take half an hour per job ahead of you.\n"
c79fefcaBoris Power4 years ago284]
285}
286],
287"metadata": {}
288},
289{
290"cell_type": "markdown",
291"source": [
292"The tool helpfully suggests a few improvements to the dataset and splits the dataset into training and validation set.\n",
293"\n",
294"A suffix between a prompt and a completion is necessary to tell the model that the input text has stopped, and that it now needs to predict the class. Since we use the same separator in each example, the model is able to learn that it is meant to predict either baseball or hockey following the separator.\n",
295"A whitespace prefix in completions is useful, as most word tokens are tokenized with a space prefix.\n",
296"The tool also recognized that this is likely a classification task, so it suggested to split the dataset into training and validation datasets. This will allow us to easily measure expected performance on new data."
297],
298"metadata": {}
299},
300{
301"cell_type": "markdown",
302"source": [
303"## Fine-tuning\n",
7febb755Boris Power4 years ago304"The tool suggests we run the following command to train the dataset. Since this is a classification task, we would like to know what the generalization performance on the provided validation set is for our classification use case. The tool suggests to add `--compute_classification_metrics --classification_positive_class \" baseball\"` in order to compute the classification metrics. Classification performs better with a hyperparameter `--no_packing`.\n",
305"\n",
306"We can simply copy the suggested command from the CLI tool. We specifically add `-m ada` to fine-tune a cheaper and faster ada model, which is usually comperable in performance to slower and more expensive models on classification use cases. "
c79fefcaBoris Power4 years ago307],
308"metadata": {}
309},
310{
311"cell_type": "code",
7febb755Boris Power4 years ago312"execution_count": 9,
c79fefcaBoris Power4 years ago313"source": [
7febb755Boris Power4 years ago314"!openai api fine_tunes.create -t \"sport2_prepared_train.jsonl\" -v \"sport2_prepared_valid.jsonl\" --no_packing --compute_classification_metrics --classification_positive_class \" baseball\" -m ada"
c79fefcaBoris Power4 years ago315],
316"outputs": [
317{
318"output_type": "stream",
319"name": "stdout",
320"text": [
7febb755Boris Power4 years ago321"Upload progress: 100%|████████████████████| 1.52M/1.52M [00:00<00:00, 1.81Mit/s]\n",
322"Uploaded file from sport2_prepared_train.jsonl: file-Dxx2xJqyjcwlhfDHpZdmCXlF\n",
323"Upload progress: 100%|███████████████████████| 388k/388k [00:00<00:00, 507kit/s]\n",
324"Uploaded file from sport2_prepared_valid.jsonl: file-Mvb8YAeLnGdneSAFcfiVcgcN\n",
325"Created fine-tune: ft-2zaA7qi0rxJduWQpdvOvmGn3\n",
c79fefcaBoris Power4 years ago326"Streaming events until fine-tuning is complete...\n",
327"\n",
328"(Ctrl-C will interrupt the stream, but not cancel the fine-tune)\n",
7febb755Boris Power4 years ago329"[2021-07-30 13:15:50] Created fine-tune: ft-2zaA7qi0rxJduWQpdvOvmGn3\n",
330"[2021-07-30 13:15:52] Fine-tune enqueued. Queue number: 0\n",
331"[2021-07-30 13:15:56] Fine-tune started\n",
332"[2021-07-30 13:18:55] Completed epoch 1/4\n",
333"[2021-07-30 13:20:47] Completed epoch 2/4\n",
334"[2021-07-30 13:22:40] Completed epoch 3/4\n",
335"[2021-07-30 13:24:31] Completed epoch 4/4\n",
336"[2021-07-30 13:26:22] Uploaded model: ada:ft-openai-2021-07-30-12-26-20\n",
337"[2021-07-30 13:26:27] Uploaded result file: file-6Ki9RqLQwkChGsr9CHcr1ncg\n",
338"[2021-07-30 13:26:28] Fine-tune succeeded\n",
c79fefcaBoris Power4 years ago339"\n",
340"Job complete! Status: succeeded 🎉\n",
341"Try out your fine-tuned model:\n",
342"\n",
7febb755Boris Power4 years ago343"openai api completions.create -m ada:ft-openai-2021-07-30-12-26-20 -p <YOUR_PROMPT>\n"
c79fefcaBoris Power4 years ago344]
345}
346],
347"metadata": {}
348},
349{
350"cell_type": "markdown",
351"source": [
7febb755Boris Power4 years ago352"The model is successfully trained in about ten minutes. We can see the model name is `ada:ft-openai-2021-07-30-12-26-20`, which we can use for doing inference."
c79fefcaBoris Power4 years ago353],
354"metadata": {}
355},
356{
357"cell_type": "markdown",
358"source": [
359"### [Advanced] Results and expected model performance\n",
360"We can now download the results file to observe the expected performance on a held out validation set."
361],
362"metadata": {}
363},
364{
365"cell_type": "code",
7febb755Boris Power4 years ago366"execution_count": 10,
c79fefcaBoris Power4 years ago367"source": [
7febb755Boris Power4 years ago368"!openai api fine_tunes.results -i ft-2zaA7qi0rxJduWQpdvOvmGn3 > result.csv"
c79fefcaBoris Power4 years ago369],
370"outputs": [],
371"metadata": {}
372},
373{
374"cell_type": "code",
7febb755Boris Power4 years ago375"execution_count": 11,
c79fefcaBoris Power4 years ago376"source": [
377"results = pd.read_csv('result.csv')\n",
378"results[results['classification/accuracy'].notnull()].tail(1)"
379],
380"outputs": [
381{
382"output_type": "execute_result",
383"data": {
384"text/plain": [
385" step elapsed_tokens elapsed_examples training_loss \\\n",
7febb755Boris Power4 years ago386"929 930 3027688 3720 0.044408 \n",
c79fefcaBoris Power4 years ago387"\n",
388" training_sequence_accuracy training_token_accuracy \\\n",
7febb755Boris Power4 years ago389"929 1.0 1.0 \n",
c79fefcaBoris Power4 years ago390"\n",
391" classification/accuracy classification/precision classification/recall \\\n",
7febb755Boris Power4 years ago392"929 0.991597 0.983471 1.0 \n",
c79fefcaBoris Power4 years ago393"\n",
394" classification/auroc classification/auprc classification/f1.0 \\\n",
7febb755Boris Power4 years ago395"929 1.0 1.0 0.991667 \n",
c79fefcaBoris Power4 years ago396"\n",
397" validation_loss validation_sequence_accuracy validation_token_accuracy \n",
7febb755Boris Power4 years ago398"929 NaN NaN NaN "
c79fefcaBoris Power4 years ago399],
400"text/html": [
401"<div>\n",
402"<style scoped>\n",
403" .dataframe tbody tr th:only-of-type {\n",
404" vertical-align: middle;\n",
405" }\n",
406"\n",
407" .dataframe tbody tr th {\n",
408" vertical-align: top;\n",
409" }\n",
410"\n",
411" .dataframe thead th {\n",
412" text-align: right;\n",
413" }\n",
414"</style>\n",
415"<table border=\"1\" class=\"dataframe\">\n",
416" <thead>\n",
417" <tr style=\"text-align: right;\">\n",
418" <th></th>\n",
419" <th>step</th>\n",
420" <th>elapsed_tokens</th>\n",
421" <th>elapsed_examples</th>\n",
422" <th>training_loss</th>\n",
423" <th>training_sequence_accuracy</th>\n",
424" <th>training_token_accuracy</th>\n",
425" <th>classification/accuracy</th>\n",
426" <th>classification/precision</th>\n",
427" <th>classification/recall</th>\n",
428" <th>classification/auroc</th>\n",
429" <th>classification/auprc</th>\n",
430" <th>classification/f1.0</th>\n",
431" <th>validation_loss</th>\n",
432" <th>validation_sequence_accuracy</th>\n",
433" <th>validation_token_accuracy</th>\n",
434" </tr>\n",
435" </thead>\n",
436" <tbody>\n",
437" <tr>\n",
7febb755Boris Power4 years ago438" <th>929</th>\n",
439" <td>930</td>\n",
440" <td>3027688</td>\n",
441" <td>3720</td>\n",
442" <td>0.044408</td>\n",
443" <td>1.0</td>\n",
444" <td>1.0</td>\n",
445" <td>0.991597</td>\n",
446" <td>0.983471</td>\n",
c79fefcaBoris Power4 years ago447" <td>1.0</td>\n",
448" <td>1.0</td>\n",
449" <td>1.0</td>\n",
450" <td>0.991667</td>\n",
451" <td>NaN</td>\n",
452" <td>NaN</td>\n",
453" <td>NaN</td>\n",
454" </tr>\n",
455" </tbody>\n",
456"</table>\n",
457"</div>"
458]
459},
460"metadata": {},
7febb755Boris Power4 years ago461"execution_count": 11
c79fefcaBoris Power4 years ago462}
463],
464"metadata": {}
465},
466{
467"cell_type": "markdown",
468"source": [
469"The accuracy reaches 99.6%. On the plot below we can see how accuracy on the validation set increases during the training run. "
470],
471"metadata": {}
472},
473{
474"cell_type": "code",
7febb755Boris Power4 years ago475"execution_count": 12,
c79fefcaBoris Power4 years ago476"source": [
477"results[results['classification/accuracy'].notnull()]['classification/accuracy'].plot()"
478],
479"outputs": [
480{
481"output_type": "execute_result",
482"data": {
483"text/plain": [
484"<AxesSubplot:>"
485]
486},
487"metadata": {},
7febb755Boris Power4 years ago488"execution_count": 12
c79fefcaBoris Power4 years ago489},
490{
491"output_type": "display_data",
492"data": {
493"text/plain": [
494"<Figure size 432x288 with 1 Axes>"
495],
7febb755Boris Power4 years ago496"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Z1A+gAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZKUlEQVR4nO3de5BU55nf8e8zdxhguMxwm0GALCSELggYyXKktWVdbAlZQhcEUmUr65R39c9qd+NskpKSlOOoKpVK1Va8u1WKE2XXu/FWYk2DLkYya2RL8trrkuTu4Spu0hgsnR4GZrjDAHPrJ3/MQdseDUwD3XO6T/8+VV30Oeel++kzhx9n3vf0e8zdERGR0lcRdQEiIpIfCnQRkZhQoIuIxIQCXUQkJhToIiIxURXVGzc2NvqCBQuiensRkZLU3t5+2N2bRtsWWaAvWLCAVCoV1duLiJQkM/v4QtvU5SIiEhMKdBGRmBgz0M3se2bWbWYfXGC7mdlfmlmHmW03s+X5L1NERMaSyxn63wL3X2T7A8Ci8PE08N0rL0tERC7VmIHu7j8Hjl6kySrg+z7sPWCqmc3JV4EiIpKbfPShNwNB1nI6XPcZZva0maXMLNXT05OHtxYRkfPGdVDU3V9091Z3b21qGvUyShERuUz5uA69E5iXtdwSrpMikMk433/3Nxzt7Y+6FBEJ3XP9LJbOm5r3181HoG8AnjGzl4DPAyfcvSsPryt58IuOw3z79V0AmEVcjIgAMHNKXTSBbmY/AO4CGs0sDfwnoBrA3f8nsBFYCXQAZ4B/mfcq5bIlkgHTJlbz3r+/h9qqyqjLEZECGjPQ3f2pMbY78Id5q0jy5mhvP2/uOsjv3j5fYS5SBvRN0Rh7dUsnA0PO2lvnjd1YREqeAj2m3J11qYClLQ0snj0l6nJEZBwo0GNqe/oEew6eYo3OzkXKhgI9ptpSAXXVFTy0dG7UpYjIOFGgx9DZ/iFe33qAlTfOYUpdddTliMg4UaDH0MYdXZzqG1R3i0iZUaDHUFsqYMGMiXx+4fSoSxGRcaRAj5n9h3v51f6jPNE6D9NXQ0XKigI9ZhKpgAqD1Staoi5FRMaZAj1GBocyvNye5svXzWTWlLqoyxGRcaZAj5F/+LCH7lN9GgwVKVMK9BhpSwY0Tqrh7sUzoy5FRCKgQI+JnlN9vL2nm8eWt1BdqR+rSDnSv/yYeGVzmsGMs6ZV3S0i5UqBHgPuTlsqYMX8aVwzc1LU5YhIRBToMdD+8TH29fSyVmfnImVNgR4DiVRAfU0lD948J+pSRCRCCvQSd7pvkDe2d/G1m+dSX5uPW8SKSKlSoJe4H20/wJn+Idbcqm+GipQ7BXqJa0sGfK6pnuVXTYu6FBGJmAK9hHV0n2LzJ8dZe6sm4hIRBXpJa0sGVFUYjy1Xd4uIKNBLVv9ghlc2d3LP9TNpnFQbdTkiUgQU6CXq7T3dHOntZ60m4hKRkAK9RCVSAbOm1PLFRU1RlyIiRUKBXoIOnjjHz/Z2s3pFC1WaiEtEQkqDEvTy5jQZhydWqLtFRP6JAr3EZDJOIhXw+YXTWdBYH3U5IlJEFOgl5v39R/n4yBkNhorIZyjQS0wiFTC5tooHbtREXCLy2xToJeTkuQE27uji4VvmMqGmMupyRKTI5BToZna/me01sw4ze3aU7fPN7C0z225mPzMzfXWxADZsPUDfYEbdLSIyqjED3cwqgReAB4AlwFNmtmREsz8Dvu/uNwPPA/8134XKcHfL4tmTuam5IepSRKQI5XKGfhvQ4e773L0feAlYNaLNEuDt8Pk7o2yXK7S76yTb0ydY06qJuERkdLkEejMQZC2nw3XZtgGPhc8fBSab2YyRL2RmT5tZysxSPT09l1Nv2WpLBtRUVvDospG7XkRkWL4GRf8N8CUz2wJ8CegEhkY2cvcX3b3V3VubmvSV9Vz1DQ7x2tZO7rthFtPqa6IuR0SKVC73LOsEskfhWsJ1n3L3A4Rn6GY2CXjc3Y/nqcay9+bOQxw/M6CbQIvIReVyhp4EFpnZQjOrAZ4ENmQ3MLNGMzv/Ws8B38tvmeUtkQponjqBO69pjLoUESliYwa6uw8CzwCbgN1Awt13mtnzZvZw2OwuYK+ZfQjMAv5LgeotO+ljZ/jHjsOsXtFCRYUGQ0XkwnK6Tby7bwQ2jlj3razn64H1+S1NANa3pwF4olWX9ovIxembokUsk3HWpdLc8blGWqZNjLocESlyCvQi9stfH6bz+FnW6JuhIpIDBXoRa0sGNEyo5itLZkVdioiUAAV6kTrW28+bOw/x6LJm6qo1EZeIjE2BXqR+uLWT/qEMa3TtuYjkSIFehNydtlSam5obWDJ3StTliEiJUKAXoQ86T7K766QGQ0XkkijQi1Bb6hNqqyp4eOncqEsRkRKiQC8y5waG+OHWAzxw42waJlRHXY6IlBAFepH5+w+6OHVuUN0tInLJFOhFpi0ZcNX0idy+8DPTyYuIXJQCvYh8fKSX9/YdZU2rJuISkUunQC8i61JpKgxWr1B3i4hcOgV6kRjKOOvb03zp2iZmN9RFXY6IlCAFepH4+Yc9HDx5Tt8MFZHLpkAvEm3JgBn1NdxzvSbiEpHLo0AvAodP9/HT3cMTcdVU6UciIpdH6VEEXt3cyWDGWatrz0XkCijQI+buJFIBy66ayqJZk6MuR0RKmAI9YluC43zUfZq1GgwVkSukQI9YIhkwobqSr2kiLhG5Qgr0CPX2DfL6tgM8ePMcJtVWRV2OiJQ4BXqEfrSji97+IQ2GikheKNAjlEgGXN1UT+v8aVGXIiIxoECPSEf3aVIfH2NN6zzMNBGXiFw5BXpE1rUHVFYYjy1vjroUEYkJBXoEBoYyvNzeyd2LZzJzsibiEpH8UKBH4J093Rw+3adrz0UkrxToEUikApom13LXdU1RlyIiMaJAH2fdJ8/xzt4eHl/eQlWldr+I5I8SZZyt35xmKOOsaW2JuhQRiRkF+jhyd9al0ty2YDpXN02KuhwRiZmcAt3M7jezvWbWYWbPjrL9KjN7x8y2mNl2M1uZ/1JLX/I3x9h/uJc1+maoiBTAmIFuZpXAC8ADwBLgKTNbMqLZfwQS7r4MeBL4H/kuNA7akgGTaqtYedPsqEsRkRjK5Qz9NqDD3fe5ez/wErBqRBsHpoTPG4AD+SsxHk6dG2Djji4eWjqXiTWaiEtE8i+XQG8GgqzldLgu27eB3zWzNLAR+KPRXsjMnjazlJmlenp6LqPc0vX6ti7ODmgiLhEpnHwNij4F/K27twArgb8zs8+8tru/6O6t7t7a1FRe12C3pQKunTWJpS0NUZciIjGVS6B3AtmnlS3humzfABIA7v4uUAc05qPAONh78BTbguOaiEtECiqXQE8Ci8xsoZnVMDzouWFEm0+AewDM7HqGA728+lQuoi0ZUF1pPLZc156LSOGMGejuPgg8A2wCdjN8NctOM3vezB4Om/0p8Admtg34AfB1d/dCFV1K+gczvLolzX1LZjG9vibqckQkxnK63MLdNzI82Jm97ltZz3cBd+S3tHj46e5DHDszwBpNxCUiBaZvihZYWzJgbkMdv7OovAaBRWT8KdAL6MDxs/z8ox5Wr2ihskKDoSJSWAr0AlrfnsYdVq9Qd4uIFJ4CvUAyGSeRCvhnn5vBVTMmRl2OiJQBBXqBvLvvCOljZ/XNUBEZNwr0AkmkAqbUVfHVGzQRl4iMDwV6AZw4M8Dff3CQR5Y1U1ddGXU5IlImFOgF8MNtnfQPZnTtuYiMKwV6AbQlA26YO4UbmzURl4iMHwV6nn3QeYKdB07q7FxExp0CPc8SqYCaqgoeuWXklPEiIoWlQM+jcwNDvLalk/tvmE3DxOqoyxGRMqNAz6NNOw9y8tygrj0XkUgo0PMokQqYN30CX7h6RtSliEgZUqDnSXD0DL/sOMITK+ZRoYm4RCQCCvQ8WZcKMIPVK3RXIhGJhgI9D4Yyzrr2NL+zqIm5UydEXY6IlCkFeh784qMeuk6cY62uPReRCCnQ8yCRCpg2sZp7l8yMuhQRKWMK9Ct0tLefn+w6xKPLWqit0kRcIhIdBfoVenVLJwNDrmvPRSRyCvQr4O4kkgFL503lutmToy5HRMqcAv0KbEufYO+hUxoMFZGioEC/Am3JgLrqCh5aOifqUkREFOiX60z/IK9vO8DKm+YwuU4TcYlI9BTol2njjoOc7htUd4uIFA0F+mVKpAIWNtZz28LpUZciIgIo0C/L/sO9/Gr/UZ5obcFME3GJSHFQoF+GRCqgssJYvVwTcYlI8VCgX6LBoQwvt6f58nVNzJxSF3U5IiKfUqBfop/t7aH7VJ9uAi0iRSenQDez+81sr5l1mNmzo2z/jpltDR8fmtnxvFdaJNpSAY2TavnyYk3EJSLFpWqsBmZWCbwA3AekgaSZbXD3XefbuPs3s9r/EbCsALVGrvvUOd7e083v37mQ6kr9ciMixSWXVLoN6HD3fe7eD7wErLpI+6eAH+SjuGLz6uZOhjLOE+puEZEilEugNwNB1nI6XPcZZjYfWAi8feWlFRd3py0V0Dp/GtfMnBR1OSIin5HvfoMngfXuPjTaRjN72sxSZpbq6enJ81sXVvvHx9jX08saTZMrIkUql0DvBLJTrCVcN5onuUh3i7u/6O6t7t7a1NSUe5VFoC0ZUF9TyYM3aSIuESlOuQR6ElhkZgvNrIbh0N4wspGZLQamAe/mt8Tone4b5Ec7unho6Vzqa8ccRxYRicSYge7ug8AzwCZgN5Bw951m9ryZPZzV9EngJXf3wpQanTe2HeBM/5AGQ0WkqOV0uunuG4GNI9Z9a8Tyt/NXVnFpSwVcM3MSy6+aGnUpIiIXpIupx/DRoVNs+eQ4a1vnaSIuESlqCvQxJFIBVRXGo8tHvVJTRKRoKNAvon8wwyubO7n3+lk0TqqNuhwRkYtSoF/E23sOcaS3n7W69lxESoAC/SLakgGzp9TxxWtL65p5ESlPCvQLOHjiHP/wYQ+Pr2imskKDoSJS/BToF7C+PSDjaN5zESkZCvRRZDJOIpXm9qunM39GfdTliIjkRIE+ivf3H+WTo2c0GCoiJUWBPopEKmByXRUP3KiJuESkdCjQRzhxdoCNO7pYdctc6qoroy5HRCRnCvQRNmw7QN9ghrWtV0VdiojIJVGgj5BIBiyePZkbm6dEXYqIyCVRoGfZdeAkOzpPsPZWTcQlIqVHgZ4lkQqoqazgkVs0EZeIlB4FeqhvcIjXtnbylRtmMa2+JupyREQumQI99ObOQxw/M6Brz0WkZCnQQ4lUQPPUCdzxucaoSxERuSwKdCB97Az/2HGYJ1pbqNBEXCJSohTowLpUGoDVK1oirkRE5PKVfaAPZZz17WnuvKaRlmkToy5HROSylX2g/7LjMJ3Hz2qaXBEpeWUf6IlUwNSJ1XzlhllRlyIickXKOtCP9fbz5s5DPHJLM7VVmohLREpbWQf6a1s76R/K6NpzEYmFsg10d6ctGXBzSwPXz9FEXCJS+so20Hd0nmDPwVMaDBWR2CjbQG9LBtRWVfDQ0rlRlyIikhdlGehn+4fYsPUAK2+aQ8OE6qjLERHJi7IM9B/v7OJU36C6W0QkVsoy0NuSAfNnTOT2q6dHXYqISN6UXaB/fKSX9/YdZU2r7kokIvGSU6Cb2f1mttfMOszs2Qu0WWNmu8xsp5n9v/yWmT+JVECFwePLNRGXiMRL1VgNzKwSeAG4D0gDSTPb4O67stosAp4D7nD3Y2Y2s1AFX4nBoQzr29Pcdd1MZjfURV2OiEhe5XKGfhvQ4e773L0feAlYNaLNHwAvuPsxAHfvzm+Z+fHzj3o4dLKPNa06OxeR+Mkl0JuBIGs5Ha7Ldi1wrZn90szeM7P7R3shM3vazFJmlurp6bm8iq9AWzJgRn0Ndy/WRFwiEj/5GhStAhYBdwFPAf/bzKaObOTuL7p7q7u3NjU15emtc3P4dB9v7e7mseXN1FSV3ViwiJSBXJKtE8i+YLslXJctDWxw9wF33w98yHDAF41XN3cymHFNxCUisZVLoCeBRWa20MxqgCeBDSPavMbw2Tlm1shwF8y+/JV5ZdydtlTA8qumcs3MyVGXIyJSEGMGursPAs8Am4DdQMLdd5rZ82b2cNhsE3DEzHYB7wD/1t2PFKroS7X5k+N0dJ/W2bmIxNqYly0CuPtGYOOIdd/Keu7Avw4fRSeRDJhYU8mDN2siLhGJr9iPDvb2DfLG9gM8eNMcJtXm9P+XiEhJin2g/2h7F739Q+puEZHYi32gJ1IBVzfVs2L+tKhLEREpqFgHekf3aVIfH2OtJuISkTIQ60BflwqoqjAe00RcIlIGYhvoA0MZXt6c5u7FM2maXBt1OSIiBRfbQH97TzeHT/drMFREykZsAz2RDJg5uZYvXTu+c8aIiEQlloF+6OQ53tnbzeMrWqiqjOVHFBH5jFim3cub02Qc3QRaRMpK7ALd3VmXSnPbwuksbKyPuhwRkXETu0D/1f6j7D/cy1qdnYtImYldoLelAibXVrHypjlRlyIiMq5iFegnzw2wcUcXD90ylwk1lVGXIyIyrmIV6K9vO8C5gYwGQ0WkLMUq0BPJgOtmTWZpS0PUpYiIjLvYBPqegyfZlj7Bmls1EZeIlKfYBHoimaa60nh0WXPUpYiIRCIWgd43OMSrW9J8ZclsptfXRF2OiEgkYhHoP93VzbEzA6zRRFwiUsZiEehtqYC5DXXceU1j1KWIiESm5AO98/hZfvFRD6tb51FZocFQESlfJR/o61Np3OGJFborkYiUt5IO9EzGWdcecMc1M5g3fWLU5YiIRKqkA/3dfUdIHzurb4aKiFDigd6WDGiYUM1Xb5gddSkiIpEr2UA/cWaAH+88yCO3zKWuWhNxiYiUbKC/trWT/sGMrj0XEQmVbKC3JQNubJ7CDXM1EZeICJRooH/QeYJdXSc1GCoikqUkAz2RCqipqmDVUk3EJSJyXk6Bbmb3m9leM+sws2dH2f51M+sxs63h4/fzX+qwcwNDvLalkwdunE3DxOpCvY2ISMmpGquBmVUCLwD3AWkgaWYb3H3XiKZt7v5MAWr8LZt2HuTkuUHdBFpEZIRcztBvAzrcfZ+79wMvAasKW9aF1ddUcd+SWdx+9YyoShARKUpjnqEDzUCQtZwGPj9Ku8fN7IvAh8A33T0Ypc0Vu3fJLO5dMqsQLy0iUtLyNSj6OrDA3W8GfgL8n9EamdnTZpYys1RPT0+e3lpERCC3QO8EsjusW8J1n3L3I+7eFy7+FbBitBdy9xfdvdXdW5uami6nXhERuYBcAj0JLDKzhWZWAzwJbMhuYGZzshYfBnbnr0QREcnFmH3o7j5oZs8Am4BK4HvuvtPMngdS7r4B+GMzexgYBI4CXy9gzSIiMgpz90jeuLW11VOpVCTvLSJSqsys3d1bR9tWkt8UFRGRz1Kgi4jEhAJdRCQmIutDN7Me4OPL/OuNwOE8llOqtB+0D87TfiiffTDf3Ue97juyQL8SZpa60KBAOdF+0D44T/tB+wDU5SIiEhsKdBGRmCjVQH8x6gKKhPaD9sF52g/aB6XZhy4iIp9VqmfoIiIyggJdRCQmSi7Qx7q/aVyY2Twze8fMdpnZTjP7k3D9dDP7iZl9FP45LVxvZvaX4X7ZbmbLo/0E+WNmlWa2xczeCJcXmtn74WdtC2cBxcxqw+WOcPuCSAvPIzObambrzWyPme02sy+U6bHwzfDfwwdm9gMzqyvH4+FCSirQs+5v+gCwBHjKzJZEW1XBDAJ/6u5LgNuBPww/67PAW+6+CHgrXIbhfbIofDwNfHf8Sy6YP+G3p2T+b8B33P0a4BjwjXD9N4Bj4frvhO3i4i+AH7v7YmApw/ujrI4FM2sG/hhodfcbGZ799UnK83gYnbuXzAP4ArApa/k54Lmo6xqnz/5Dhm/UvReYE66bA+wNn/8v4Kms9p+2K+UHwzdUeQu4G3gDMIa/DVg18phgeIrnL4TPq8J2FvVnyMM+aAD2j/wsZXgsnL8d5vTw5/sG8NVyOx4u9iipM3RGv79pc0S1jJvwV8VlwPvALHfvCjcdBM7fYDWu++bPgX8HZMLlGcBxdx8Ml7M/56f7INx+Imxf6hYCPcDfhF1Pf2Vm9ZTZseDuncCfAZ8AXQz/fNspv+Phgkot0MuOmU0CXgb+lbufzN7mw6cesb3u1My+BnS7e3vUtUSsClgOfNfdlwG9/FP3ChD/YwEgHCNYxfB/cHOBeuD+SIsqMqUW6GPe3zROzKya4TD/v+7+Srj60Plb/oV/dofr47hv7gAeNrPfAC8x3O3yF8BUMzt/t63sz/npPgi3NwBHxrPgAkkDaXd/P1xez3DAl9OxAHAvsN/de9x9AHiF4WOk3I6HCyq1QB/z/qZxYWYG/DWw293/e9amDcDvhc9/j+G+9fPr/0V4hcPtwImsX8dLkrs/5+4t7r6A4Z/12+7+z4F3gNVhs5H74Py+WR22L/mzVnc/CARmdl246h5gF2V0LIQ+AW43s4nhv4/z+6GsjoeLiroT/1IfwErgQ+DXwH+Iup4Cfs47Gf4VejuwNXysZLgP8C3gI+CnwPSwvTF8BdCvgR0MXwkQ+efI4/64C3gjfH418CugA1gH1Ibr68LljnD71VHXncfPfwuQCo+H14Bp5XgsAP8Z2AN8APwdUFuOx8OFHvrqv4hITJRal4uIiFyAAl1EJCYU6CIiMaFAFxGJCQW6iEhMKNBFRGJCgS4iEhP/HxPg2XO9XdJVAAAAAElFTkSuQmCC"
c79fefcaBoris Power4 years ago497},
498"metadata": {
499"needs_background": "light"
500}
501}
502],
503"metadata": {}
504},
505{
506"cell_type": "markdown",
507"source": [
508"## Using the model\n",
509"We can now call the model to get the predictions."
510],
511"metadata": {}
512},
513{
514"cell_type": "code",
7febb755Boris Power4 years ago515"execution_count": 13,
c79fefcaBoris Power4 years ago516"source": [
7febb755Boris Power4 years ago517"test = pd.read_json('sport2_prepared_valid.jsonl', lines=True)\n",
c79fefcaBoris Power4 years ago518"test.head()"
519],
520"outputs": [
521{
522"output_type": "execute_result",
523"data": {
524"text/plain": [
525" prompt completion\n",
526"0 From: gld@cunixb.cc.columbia.edu (Gary L Dare)... hockey\n",
527"1 From: smorris@venus.lerc.nasa.gov (Ron Morris ... hockey\n",
528"2 From: golchowy@alchemy.chem.utoronto.ca (Geral... hockey\n",
529"3 From: krattige@hpcc01.corp.hp.com (Kim Krattig... baseball\n",
530"4 From: warped@cs.montana.edu (Doug Dolven)\\nSub... baseball"
531],
532"text/html": [
533"<div>\n",
534"<style scoped>\n",
535" .dataframe tbody tr th:only-of-type {\n",
536" vertical-align: middle;\n",
537" }\n",
538"\n",
539" .dataframe tbody tr th {\n",
540" vertical-align: top;\n",
541" }\n",
542"\n",
543" .dataframe thead th {\n",
544" text-align: right;\n",
545" }\n",
546"</style>\n",
547"<table border=\"1\" class=\"dataframe\">\n",
548" <thead>\n",
549" <tr style=\"text-align: right;\">\n",
550" <th></th>\n",
551" <th>prompt</th>\n",
552" <th>completion</th>\n",
553" </tr>\n",
554" </thead>\n",
555" <tbody>\n",
556" <tr>\n",
557" <th>0</th>\n",
558" <td>From: gld@cunixb.cc.columbia.edu (Gary L Dare)...</td>\n",
559" <td>hockey</td>\n",
560" </tr>\n",
561" <tr>\n",
562" <th>1</th>\n",
563" <td>From: smorris@venus.lerc.nasa.gov (Ron Morris ...</td>\n",
564" <td>hockey</td>\n",
565" </tr>\n",
566" <tr>\n",
567" <th>2</th>\n",
568" <td>From: golchowy@alchemy.chem.utoronto.ca (Geral...</td>\n",
569" <td>hockey</td>\n",
570" </tr>\n",
571" <tr>\n",
572" <th>3</th>\n",
573" <td>From: krattige@hpcc01.corp.hp.com (Kim Krattig...</td>\n",
574" <td>baseball</td>\n",
575" </tr>\n",
576" <tr>\n",
577" <th>4</th>\n",
578" <td>From: warped@cs.montana.edu (Doug Dolven)\\nSub...</td>\n",
579" <td>baseball</td>\n",
580" </tr>\n",
581" </tbody>\n",
582"</table>\n",
583"</div>"
584]
585},
586"metadata": {},
7febb755Boris Power4 years ago587"execution_count": 13
c79fefcaBoris Power4 years ago588}
589],
590"metadata": {}
591},
7febb755Boris Power4 years ago592{
593"cell_type": "markdown",
594"source": [
595"We need to use the same separator following the prompt which we used during fine-tuning. In this case it is `\\n\\n###\\n\\n`. Since we're concerned with classification, we want the temperature to be as low as possible, and we only require one token completion to determine the prediction of the model."
596],
597"metadata": {}
598},
c79fefcaBoris Power4 years ago599{
600"cell_type": "code",
7febb755Boris Power4 years ago601"execution_count": 14,
c79fefcaBoris Power4 years ago602"source": [
7febb755Boris Power4 years ago603"ft_model = 'ada:ft-openai-2021-07-30-12-26-20'\n",
c79fefcaBoris Power4 years ago604"res = openai.Completion.create(model=ft_model, prompt=test['prompt'][0] + '\\n\\n###\\n\\n', max_tokens=1, temperature=0)\n",
605"res['choices'][0]['text']\n"
606],
607"outputs": [
608{
609"output_type": "execute_result",
610"data": {
611"text/plain": [
612"' hockey'"
613]
614},
615"metadata": {},
7febb755Boris Power4 years ago616"execution_count": 14
c79fefcaBoris Power4 years ago617}
618],
619"metadata": {}
620},
621{
622"cell_type": "markdown",
623"source": [
624"To get the log probabilities, we can specify logprobs parameter on the completion request"
625],
626"metadata": {}
627},
628{
629"cell_type": "code",
7febb755Boris Power4 years ago630"execution_count": 15,
c79fefcaBoris Power4 years ago631"source": [
632"res = openai.Completion.create(model=ft_model, prompt=test['prompt'][0] + '\\n\\n###\\n\\n', max_tokens=1, temperature=0, logprobs=2)\n",
633"res['choices'][0]['logprobs']['top_logprobs'][0]"
634],
635"outputs": [
636{
637"output_type": "execute_result",
638"data": {
639"text/plain": [
7febb755Boris Power4 years ago640"<OpenAIObject at 0x7fe114e435c8> JSON: {\n",
641" \" baseball\": -7.6311407,\n",
642" \" hockey\": -0.0006307676\n",
c79fefcaBoris Power4 years ago643"}"
644]
645},
646"metadata": {},
7febb755Boris Power4 years ago647"execution_count": 15
c79fefcaBoris Power4 years ago648}
649],
650"metadata": {}
651},
652{
653"cell_type": "markdown",
654"source": [
655"We can see that the model predicts hockey as a lot more likely than baseball, which is the correct prediction. By requesting log_probs, we can see the prediction (log) probability for each class."
656],
657"metadata": {}
658},
659{
660"cell_type": "markdown",
661"source": [
662"### Generalization\n",
663"Interestingly, our fine-tuned classifier is quite versatile. Despite being trained on emails to different mailing lists, it also successfully predicts tweets."
664],
665"metadata": {}
666},
667{
668"cell_type": "code",
7febb755Boris Power4 years ago669"execution_count": 16,
c79fefcaBoris Power4 years ago670"source": [
671"sample_hockey_tweet = \"\"\"Thank you to the \n",
672"@Canes\n",
673" and all you amazing Caniacs that have been so supportive! You guys are some of the best fans in the NHL without a doubt! Really excited to start this new chapter in my career with the \n",
674"@DetroitRedWings\n",
675" !!\"\"\"\n",
676"res = openai.Completion.create(model=ft_model, prompt=sample_hockey_tweet + '\\n\\n###\\n\\n', max_tokens=1, temperature=0, logprobs=2)\n",
677"res['choices'][0]['text']"
678],
679"outputs": [
680{
681"output_type": "execute_result",
682"data": {
683"text/plain": [
684"' hockey'"
685]
686},
687"metadata": {},
7febb755Boris Power4 years ago688"execution_count": 16
c79fefcaBoris Power4 years ago689}
690],
691"metadata": {}
692},
693{
694"cell_type": "code",
7febb755Boris Power4 years ago695"execution_count": 17,
c79fefcaBoris Power4 years ago696"source": [
697"sample_baseball_tweet=\"\"\"BREAKING: The Tampa Bay Rays are finalizing a deal to acquire slugger Nelson Cruz from the Minnesota Twins, sources tell ESPN.\"\"\"\n",
698"res = openai.Completion.create(model=ft_model, prompt=sample_baseball_tweet + '\\n\\n###\\n\\n', max_tokens=1, temperature=0, logprobs=2)\n",
699"res['choices'][0]['text']"
700],
701"outputs": [
702{
703"output_type": "execute_result",
704"data": {
705"text/plain": [
706"' baseball'"
707]
708},
709"metadata": {},
7febb755Boris Power4 years ago710"execution_count": 17
c79fefcaBoris Power4 years ago711}
712],
713"metadata": {}
714}
715],
716"metadata": {
717"orig_nbformat": 4,
718"language_info": {
719"name": "python",
720"version": "3.7.3",
721"mimetype": "text/x-python",
722"codemirror_mode": {
723"name": "ipython",
724"version": 3
725},
726"pygments_lexer": "ipython3",
727"nbconvert_exporter": "python",
728"file_extension": ".py"
729},
730"kernelspec": {
731"name": "python3",
732"display_name": "Python 3.7.3 64-bit ('base': conda)"
733},
734"interpreter": {
735"hash": "3b138a8faad971cc852f62bcf00f59ea0e31721743ea2c5a866ca26adf572e75"
736}
737},
738"nbformat": 4,
739"nbformat_minor": 2
740}