{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"# Family Relationships Ontology\n",
"\n",
"This example is a part of [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners), and it has been inspired by [this blog post](https://habr.com/post/270857/).\n",
"\n",
"I always find it difficult to remember different relationships between people in a family. In this example, we will take an ontology that defines family relationships, and the actual genealogical tree, and show how we can then perform automatic inference to find all relatives.\n",
"\n",
"### Getting the Genealogical Tree\n",
"\n",
"As an example, we will take genealogical tree of [Romanov Tsar Family](https://en.wikipedia.org/wiki/House_of_Romanov). The most common format for describing family relationships is [GEDCOM](https://en.wikipedia.org/wiki/GEDCOM). We will take Romanov family tree in GEDCOM format:"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"0 HEAD\n",
"1 CHAR UTF8\n",
"1 GEDC\n",
"2 VERS 5.5\n",
"0 @0@ INDI\n",
"1 NAME Mihail Fedorovich /Romanov/\n",
"1 SEX M\n",
"1 BIRT\n",
"2 DATE 1613\n",
"1 DEAT \n",
"2 DATE 1645\n",
"1 FAMS @41@\n",
"0 @1@ INDI\n",
"1 NAME Evdokija Lukjanovna /Streshneva/\n",
"1 SEX F\n"
]
}
],
"source": [
"!head -15 data/tsars.ged"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To use GEDCOM file, we can use `python-gedcom` library:"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: python-gedcom in c:\\winapp\\miniconda3\\lib\\site-packages (1.0.0)\n"
]
}
],
"source": [
"import sys\n",
"!{sys.executable} -m pip install python-gedcom"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This library takes away some of the technical problems with file parsing, but it still gives us pretty low-level access to all individuals and families in the tree. Here is how we can parse the file, and show the list of all individuals:"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"trusted": true
},
"outputs": [],
"source": [
"from gedcom.parser import Parser\n",
"from gedcom.element.individual import IndividualElement\n",
"from gedcom.element.family import FamilyElement\n",
"g = Parser()\n",
"g.parse_file('data/tsars.ged')"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"scrolled": true,
"trusted": true
},
"outputs": [
{
"data": {
"text/plain": [
"[('@0@', ('Mihail Fedorovich', 'Romanov')),\n",
" ('@1@', ('Evdokija Lukjanovna', 'Streshneva')),\n",
" ('@2@', ('Aleksej Mihajlovich', 'Romanov')),\n",
" ('@3@', ('Marija Ilinichna', 'Miloslavskaja')),\n",
" ('@4@', ('Natalja Kirillovna', 'Naryshkina')),\n",
" ('@5@', ('Marfa Matveevna', 'Apraksina')),\n",
" ('@6@', ('Fedor Alekseevich', 'Romanov')),\n",
" ('@7@', ('Sofja Aleksevna', 'Romanova')),\n",
" ('@8@', ('Ivan V Alekseevich', 'Romanov')),\n",
" ('@9@', ('Praskovja Fedorovna', 'Saltykova')),\n",
" ('@10@', ('Ekaterina Ivanovna', 'Romanova')),\n",
" ('@11@', ('Anna Ivanovna', 'Romanova')),\n",
" ('@12@', ('Fridrih Vilgelm', 'Kurlandskij')),\n",
" ('@13@', ('Karl Leopold', 'Meklenburg-Shverinskij')),\n",
" ('@14@', ('Anna Leopoldovna', 'Meklenburg-Shverinskaja')),\n",
" ('@15@', ('Anton Ulrih', 'Braunshvejg-Volfenbjuttelskij')),\n",
" ('@16@', ('Ivan VI Antonovich', 'Braunshvejg-Volfenbjuttelskij')),\n",
" ('@17@', ('Petr I Alekseevich', 'Romanov')),\n",
" ('@18@', ('Evdokija Fedorovna', 'Lopuhina')),\n",
" ('@19@', ('Ekaterina I Alekseevna', 'Mihajlova')),\n",
" ('@20@', ('Aleksej Petrovich', 'Romanov')),\n",
" ('@21@', ('Sharlotta Kristina', 'Braunshvejg-Volfenbjuttelskaja')),\n",
" ('@22@', ('Petr II Alekseevich', 'Romanov')),\n",
" ('@23@', ('Anna Petrovna', 'Romanova')),\n",
" ('@24@', ('Elizaveta Petrovna', 'Romanova')),\n",
" ('@25@', ('Karl Fridrih', 'Golshtejn-Gottorpskij')),\n",
" ('@26@', ('Petr III Fedorovich', 'Romanov')),\n",
" ('@27@', ('Ekaterina II', 'Alekseevna')),\n",
" ('@28@', ('Pavel I Petrovich', 'Romanov')),\n",
" ('@29@', ('Natalja Alekseevna', 'Gessen-Darmshtadskaja')),\n",
" ('@30@', ('Marija Fedorovna', 'Vjurtembergskaja')),\n",
" ('@31@', ('Aleksandr I Pavlovich', 'Romanov')),\n",
" ('@32@', ('Elizaveta Alekseevna', 'Baden-Durlahskaja')),\n",
" ('@33@', ('Nikolaj I Pavlovich', 'Romanov')),\n",
" ('@34@', ('Aleksandra Fedorovna', 'Prusskaja')),\n",
" ('@35@', ('Aleksandr II Nikolaevich', 'Romanov')),\n",
" ('@36@', ('Marija Aleksandrovna', 'Gessenskaja')),\n",
" ('@37@', ('Aleksandr III Aleksandrovich', 'Romanov')),\n",
" ('@38@', ('Marija Fedorovna', 'Datskaja')),\n",
" ('@39@', ('Nikolaj II Aleksandrovich', 'Romanov')),\n",
" ('@40@', ('Aleksandra Fedorovna', 'Gessenskaja'))]"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"d = g.get_element_dictionary()\n",
"[ (k,v.get_name()) for k,v in d.items() if isinstance(v,IndividualElement)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here is how we can get information about families. Note that is gives us a list of **identifiers**, and we need to convert them to names if we want more clarity:"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[('@41@', ['@0@', '@1@', '@2@']),\n",
" ('@42@', ['@2@', '@3@', '@6@', '@7@', '@8@']),\n",
" ('@43@', ['@8@', '@9@', '@10@', '@11@']),\n",
" ('@44@', ['@13@', '@10@', '@14@']),\n",
" ('@45@', ['@15@', '@14@', '@16@']),\n",
" ('@46@', ['@2@', '@4@', '@17@']),\n",
" ('@47@', ['@17@', '@18@', '@20@']),\n",
" ('@48@', ['@20@', '@21@', '@22@']),\n",
" ('@49@', ['@17@', '@19@', '@23@', '@24@']),\n",
" ('@50@', ['@25@', '@23@', '@26@']),\n",
" ('@51@', ['@26@', '@27@', '@28@']),\n",
" ('@52@', ['@28@', '@30@', '@31@', '@33@']),\n",
" ('@53@', ['@33@', '@34@', '@35@']),\n",
" ('@54@', ['@35@', '@36@', '@37@']),\n",
" ('@55@', ['@37@', '@38@', '@39@'])]"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"d = g.get_element_dictionary()\n",
"[ (k,[x.get_value() for x in v.get_child_elements()]) for k,v in d.items() if isinstance(v,FamilyElement)]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting Family Ontology\n",
"\n",
"Next, let's have a look at [family ontology](https://raw.githubusercontent.com/blokhin/genealogical-trees/master/data/header.ttl) defined as a set of Semantic Web triplets. This ontology defines such relationships as `isUncleOf`, `isCousinOf`, and many others. All those relationships are defined in terms of basic predicates `isMotherOf`, `isFatherOf`, `isBrotherOf` and `isSisterOf`. We will use automatic reasoning to deduce all other relationships using the ontology.\n",
"\n",
"Here is a sample definition of `isAuntOf` property, which is defined as a composition of `isSisterOf` and `isParentOf` (*Aunt is a sister of one's parent*).\n",
"\n",
"```\n",
"fhkb:isAuntOf a owl:ObjectProperty ;\n",
" rdfs:domain fhkb:Woman ;\n",
" rdfs:range fhkb:Person ;\n",
" owl:propertyChainAxiom ( fhkb:isSisterOf fhkb:isParentOf ) .\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 29,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"@prefix fhkb: <http://www.example.com/genealogy.owl#> .\n",
"@prefix owl: <http://www.w3.org/2002/07/owl#> .\n",
"@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .\n",
"@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
"@prefix xml: <http://www.w3.org/XML/1998/namespace> .\n",
"@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n",
"\n",
"<http://www.example.com/genealogy.owl#> a owl:Ontology .\n",
"\n",
"fhkb:DomainEntity a owl:Class .\n",
"\n",
"fhkb:Man a owl:Class ;\n",
" owl:equivalentClass [ a owl:Class ;\n",
" owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n",
" owl:onProperty fhkb:hasSex ;\n",
" owl:someValuesFrom fhkb:Male ] ) ] .\n",
"\n",
"fhkb:Woman a owl:Class ;\n",
" owl:equivalentClass [ a owl:Class ;\n",
" owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n"
]
}
],
"source": [
"!head -20 data/onto.ttl"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Constructing Ontology for Inference\n",
"\n",
"For simplicity, we will create one ontology file that will include original rules from family ontology, and facts about individuals from our GEDCOM file. We will go through the GEDCOM file and extract information about families and individuals, and convert them to triplets."
]
},
{
"cell_type": "code",
"execution_count": 30,
"metadata": {
"trusted": true
},
"outputs": [],
"source": [
"!cp data/onto.ttl .\n",
"\n",
"gedcom_dict = g.get_element_dictionary()\n",
"individuals, marriages = {}, {}\n",
"\n",
"def term2id(el):\n",
" return \"i\" + el.get_pointer().replace('@', '').lower()\n",
"\n",
"out = open(\"onto.ttl\",\"a\")\n",
"\n",
"for k, v in gedcom_dict.items():\n",
" if isinstance(v,IndividualElement):\n",
" children, siblings = set(), set()\n",
" idx = term2id(v)\n",
"\n",
" title = v.get_name()[0] + \" \" + v.get_name()[1]\n",
" title = title.replace('\"', '').replace('[', '').replace(']', '').replace('(', '').replace(')', '').strip()\n",
"\n",
" own_families = g.get_families(v, 'FAMS')\n",
" for fam in own_families:\n",
" children |= set(term2id(i) for i in g.get_family_members(fam, \"CHIL\"))\n",
"\n",
" parent_families = g.get_families(v, 'FAMC')\n",
" if len(parent_families):\n",
" for member in g.get_family_members(parent_families[0], \"CHIL\"): # NB adoptive families i.e len(parent_families)>1 are not considered (TODO?)\n",
" if member.get_pointer() == v.get_pointer():\n",
" continue\n",
" siblings.add(term2id(member))\n",
"\n",
" if idx in individuals:\n",
" children |= individuals[idx].get('children', set())\n",
" siblings |= individuals[idx].get('siblings', set())\n",
" individuals[idx] = {'sex': v.get_gender().lower(), 'children': children, 'siblings': siblings, 'title': title}\n",
"\n",
" elif isinstance(v,FamilyElement):\n",
" wife, husb, children = None, None, set()\n",
" children = set(term2id(i) for i in g.get_family_members(v, \"CHIL\"))\n",
"\n",
" try:\n",
" wife = g.get_family_members(v, \"WIFE\")[0]\n",
" wife = term2id(wife)\n",
" if wife in individuals: individuals[wife]['children'] |= children\n",
" else: individuals[wife] = {'children': children}\n",
" except IndexError: pass\n",
" try:\n",
" husb = g.get_family_members(v, \"HUSB\")[0]\n",
" husb = term2id(husb)\n",
" if husb in individuals: individuals[husb]['children'] |= children\n",
" else: individuals[husb] = {'children': children}\n",
" except IndexError: pass\n",
"\n",
" if wife and husb: marriages[wife + husb] = (term2id(v), wife, husb)\n",
"\n",
"for idx, val in individuals.items():\n",
" added_terms = ''\n",
" if val['sex'] == 'f':\n",
" parent_predicate, sibl_predicate = \"isMotherOf\", \"isSisterOf\"\n",
" else:\n",
" parent_predicate, sibl_predicate = \"isFatherOf\", \"isBrotherOf\"\n",
" if len(val['children']):\n",
" added_terms += \" ;\\n fhkb:\" + parent_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['children']])\n",
" if len(val['siblings']):\n",
" added_terms += \" ;\\n fhkb:\" + sibl_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['siblings']])\n",
" out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing%s ;\\n rdfs:label \\\"%s\\\" .\\n\" % (idx, added_terms, val['title']))\n",
"\n",
"for k, v in marriages.items():\n",
" out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing ;\\n fhkb:hasFemalePartner fhkb:%s ;\\n fhkb:hasMalePartner fhkb:%s .\\n\" % v)\n",
"\n",
"out.write(\"[] a owl:AllDifferent ;\\n owl:distinctMembers (\")\n",
"for idx in individuals.keys():\n",
" out.write(\" fhkb:\" + idx)\n",
"for k, v in marriages.items():\n",
" out.write(\" fhkb:\" + v[0])\n",
"out.write(\" ) .\")\n",
"out.close()"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" fhkb:hasFemalePartner fhkb:i34 ;\n",
" fhkb:hasMalePartner fhkb:i33 .\n",
"fhkb:i54 a owl:NamedIndividual, owl:Thing ;\n",
" fhkb:hasFemalePartner fhkb:i36 ;\n",
" fhkb:hasMalePartner fhkb:i35 .\n",
"fhkb:i55 a owl:NamedIndividual, owl:Thing ;\n",
" fhkb:hasFemalePartner fhkb:i38 ;\n",
" fhkb:hasMalePartner fhkb:i37 .\n",
"[] a owl:AllDifferent ;\n",
" owl:distinctMembers ( fhkb:i0 fhkb:i1 fhkb:i2 fhkb:i3 fhkb:i4 fhkb:i5 fhkb:i6 fhkb:i7 fhkb:i8 fhkb:i9 fhkb:i10 fhkb:i11 fhkb:i12 fhkb:i13 fhkb:i14 fhkb:i15 fhkb:i16 fhkb:i17 fhkb:i18 fhkb:i19 fhkb:i20 fhkb:i21 fhkb:i22 fhkb:i23 fhkb:i24 fhkb:i25 fhkb:i26 fhkb:i27 fhkb:i28 fhkb:i29 fhkb:i30 fhkb:i31 fhkb:i32 fhkb:i33 fhkb:i34 fhkb:i35 fhkb:i36 fhkb:i37 fhkb:i38 fhkb:i39 fhkb:i40 fhkb:i41 fhkb:i42 fhkb:i43 fhkb:i44 fhkb:i45 fhkb:i46 fhkb:i47 fhkb:i48 fhkb:i49 fhkb:i50 fhkb:i51 fhkb:i52 fhkb:i53 fhkb:i54 fhkb:i55 ) .\n"
]
}
],
"source": [
"!tail onto.ttl"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Doing Inference \n",
"\n",
"Now we want to be able to use this ontology for inference and for querying. We will use [RDFLib](https://github.com/RDFLib), library for reading RDF Graph in different formats, querying it, etc. \n",
"\n",
"For logical inference, we will use [OWL-RL](https://github.com/RDFLib/OWL-RL) library, which allows us to build **Closure** of the RDF Graph, i.e. add all possible concepts and relations that can be inferred."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: rdflib in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (4.2.2)\n",
"Requirement already satisfied: isodate in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from rdflib) (0.6.0)\n",
"Requirement already satisfied: pyparsing in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from rdflib) (2.2.1)\n",
"Requirement already satisfied: six in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from isodate->rdflib) (1.11.0)\n",
"Collecting RDFClosure from git+git://github.com/RDFLib/OWL-RL.git#egg=RDFClosure\n",
" Cloning git://github.com/RDFLib/OWL-RL.git to /tmp/pip-install-3jouot5s/RDFClosure\n",
"Building wheels for collected packages: RDFClosure\n",
" Running setup.py bdist_wheel for RDFClosure ... \u001b[?25ldone\n",
"\u001b[?25h Stored in directory: /tmp/pip-ephem-wheel-cache-13in8fda/wheels/d8/db/e8/a1d3dea0a6029d7e76b153cfb7129a343a1e357c055c87896b\n",
"Successfully built RDFClosure\n",
"Installing collected packages: RDFClosure\n",
"Successfully installed RDFClosure-5.0.0\n"
]
}
],
"source": [
"!{sys.executable} -m pip install rdflib\n",
"!{sys.executable} -m pip install git+https://github.com/RDFLib/OWL-RL.git"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's open the ontology file and see how many triplets it contains:"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Triplets found:669\n"
]
}
],
"source": [
"import rdflib\n",
"from owlrl import DeductiveClosure, OWLRL_Extension\n",
"\n",
"g = rdflib.Graph()\n",
"g.parse(\"onto.ttl\", format=\"turtle\")\n",
"\n",
"print(\"Triplets found:%d\" % len(g))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now let's build the closure, and see how the number of triplets increase:"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Triplets after inference:4246\n"
]
}
],
"source": [
"DeductiveClosure(OWLRL_Extension).expand(g)\n",
"print(\"Triplets after inference:%d\" % len(g))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Querying for Relatives \n",
"\n",
"Now we can query the graph to see different relations between people. We can use **SPARQL** language together with `query` method. In our case, let's see all **uncles** in our family tree:"
]
},
{
"cell_type": "code",
"execution_count": 38,
"metadata": {
"trusted": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fedor Alekseevich Romanov is uncle of Ekaterina Ivanovna Romanova\n",
"Fedor Alekseevich Romanov is uncle of Anna Ivanovna Romanova\n",
"Aleksandr I Pavlovich Romanov is uncle of Aleksandr II Nikolaevich Romanov\n"
]
}
],
"source": [
"qres = g.query(\n",
" \"\"\"SELECT DISTINCT ?aname ?bname\n",
" WHERE {\n",
" ?a fhkb:isUncleOf ?b .\n",
" ?a rdfs:label ?aname .\n",
" ?b rdfs:label ?bname .\n",
" }\"\"\")\n",
"\n",
"for row in qres:\n",
" print(\"%s is uncle of %s\" % row)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Feel free to experiment with different other family relations. For example, you can have a look at `isAncestorOf` relation, which recurrently defines all ancestors of a given person.\n",
"\n",
"Finally, let's clean up!"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {
"trusted": true
},
"outputs": [],
"source": [
"!rm onto.ttl"
]
}
],
"metadata": {
"interpreter": {
"hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
},
"kernelspec": {
"display_name": "Python 3.6",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.5"
}
},
"nbformat": 4,
"nbformat_minor": 2
}microsoft/AI-For-Beginners
Publicmirrored fromhttps://github.com/microsoft/AI-For-BeginnersAvailable
2-Symbolic/FamilyOntology.ipynb
571lines · modepreview