microsoft/AI-For-Beginners
Publicmirrored fromhttps://github.com/microsoft/AI-For-BeginnersAvailable
lessons/5-NLP/19-NER/README.md
82lines · modecode
| 1 | # Named Entity Recognition |
| 2 | |
| 3 | Up to now, we have mostly been concentrating on one NLP task - classification. However, there are also other NLP tasks that can be accomplished with neural networks. One of those tasks is **[Named Entity Recognition](https://wikipedia.org/wiki/Named-entity_recognition)** (NER), which deals with recognizing specific entities within text, such as places, person names, date-time intervals, chemical formulae and so on. |
| 4 | |
| 5 | ## [Pre-lecture quiz](https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/119) |
| 6 | |
| 7 | ## Example of Using NER |
| 8 | |
| 9 | Suppose you want to develop a natural language chat bot, similar to Amazon Alexa or Google Assistant. The way intelligent chat bots work is to *understand* what the user wants by doing text classification on the input sentence. The result of this classification is so-called **intent**, which determines what a chat bot should do. |
| 10 | |
| 11 | <img alt="Bot NER" src="images/bot-ner.png" width="50%"/> |
| 12 | |
| 13 | > Image by the author |
| 14 | |
| 15 | However, a user may provide some parameters as part of the phrase. For example, when asking for the weather, she may specify a location or date. A bot should be able to understand those entities, and fill in the parameter slots accordingly before performing the action. This is exactly where NER comes in. |
| 16 | |
| 17 | > ✅ Another example would be [analyzing scientific medical papers](https://soshnikov.com/science/analyzing-medical-papers-with-azure-and-text-analytics-for-health/). One of the main things we need to look for are specific medical terms, such as diseases and medical substances. While a small number of diseases can probably be extracted using substring search, more complex entities, such as chemical compounds and medication names, need a more complex approach. |
| 18 | |
| 19 | ## NER as Token Classification |
| 20 | |
| 21 | NER models are essentially **token classification models**, because for each of the input tokens we need to decide whether it belongs to an entity or not, and if it does - to which entity class. |
| 22 | |
| 23 | Consider the following paper title: |
| 24 | |
| 25 | **Tricuspid valve regurgitation** and **lithium carbonate** **toxicity** in a newborn infant. |
| 26 | |
| 27 | Entities here are: |
| 28 | |
| 29 | * Tricuspid valve regurgitation is a disease (`DIS`) |
| 30 | * Lithium carbonate is a chemical substance (`CHEM`) |
| 31 | * Toxicity is also a disease (`DIS`) |
| 32 | |
| 33 | Notice that one entity can span several tokens. And, as in this case, we need to distinguish between two consecutive entities. Thus, it is common to use two classes for each entity - one specifying the first token of the entity (often the `B-` prefix is used, for **b**eginning), and another - the continuation of an entity (`I-`, for **i**nner token). We also use `O` as a class to represent all **o**ther tokens. Such token tagging is called [BIO tagging](https://en.wikipedia.org/wiki/Inside%E2%80%93outside%E2%80%93beginning_(tagging)) (or IOB). When tagged, our title will look like this: |
| 34 | |
| 35 | Token | Tag |
| 36 | ------|----- |
| 37 | Tricuspid | B-DIS |
| 38 | valve | I-DIS |
| 39 | regurgitation | I-DIS |
| 40 | and | O |
| 41 | lithium | B-CHEM |
| 42 | carbonate | I-CHEM |
| 43 | toxicity | B-DIS |
| 44 | in | O |
| 45 | a | O |
| 46 | newborn | O |
| 47 | infant | O |
| 48 | . | O |
| 49 | |
| 50 | Since we need to build a one-to-one correspondence between tokens and classes, we can train a rightmost **many-to-many** neural network model from this picture: |
| 51 | |
| 52 |  |
| 53 | |
| 54 | > *Image from [this blog post](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) by [Andrej Karpathy](http://karpathy.github.io/). NER token classification models correspond to the right-most network architecture on this picture.* |
| 55 | |
| 56 | ## Training NER models |
| 57 | |
| 58 | Since a NER model is essentially a token classification model, we can use RNNs that we are already familiar with for this task. In this case, each block of recurrent network will return the token ID. The following example notebook shows how to train LSTM for token classification. |
| 59 | |
| 60 | ## ✍️ Example Notebooks: NER |
| 61 | |
| 62 | Continue your learning in the following notebook: |
| 63 | |
| 64 | * [NER with TensorFlow](NER-TF.ipynb) |
| 65 | |
| 66 | ## Conclusion |
| 67 | |
| 68 | A NER model is a **token classification model**, which means that it can be used to perform token classification. This is a very common task in NLP, helping to recognize specific entities within text including places, names, dates, and more. |
| 69 | |
| 70 | ## 🚀 Challenge |
| 71 | |
| 72 | Complete the assignment linked below to train a named entity recognition model for medical terms, then try it on a different dataset. |
| 73 | |
| 74 | ## [Post-lecture quiz](https://red-field-0a6ddfd03.1.azurestaticapps.net/quiz/219) |
| 75 | |
| 76 | ## Review & Self Study |
| 77 | |
| 78 | Read through the blog [The Unreasonable Effectiveness of Recurrent Neural Networks](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) and follow along with the Further Reading section in that article to deepen your knowledge. |
| 79 | |
| 80 | ## [Assignment](lab/README.md) |
| 81 | |
| 82 | In the assignment for this lesson, you will have to train a medical entity recognition model. You can start with training an LSTM model as described in this lesson, and proceed with using the BERT transformer model. Read [the instructions](lab/README.md) to get all the details. |
| 83 | |