{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true
      },
      "source": [
        "# Family Relationships Ontology\n",
        "\n",
        "This example is a part of [AI for Beginners Curriculum](http://github.com/microsoft/ai-for-beginners), and it has been inspired by [this blog post](https://habr.com/post/270857/).\n",
        "\n",
        "I always find it difficult to remember different relationships between people in a family. In this example, we will take an ontology that defines family relationships, and the actual genealogical tree, and show how we can then perform automatic inference to find all relatives.\n",
        "\n",
        "### Getting the Genealogical Tree\n",
        "\n",
        "As an example, we will take genealogical tree of [Romanov Tsar Family](https://en.wikipedia.org/wiki/House_of_Romanov). The most common format for describing family relationships is [GEDCOM](https://en.wikipedia.org/wiki/GEDCOM). We will take Romanov family tree in GEDCOM format:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0 HEAD\n",
            "1 CHAR UTF8\n",
            "1 GEDC\n",
            "2 VERS 5.5\n",
            "0 @0@ INDI\n",
            "1 NAME Mihail Fedorovich /Romanov/\n",
            "1 SEX M\n",
            "1 BIRT\n",
            "2 DATE 1613\n",
            "1 DEAT \n",
            "2 DATE 1645\n",
            "1 FAMS @41@\n",
            "0 @1@ INDI\n",
            "1 NAME Evdokija Lukjanovna /Streshneva/\n",
            "1 SEX F\n"
          ]
        }
      ],
      "source": [
        "!head -15 data/tsars.ged"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "To use GEDCOM file, we can use `python-gedcom` library:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: python-gedcom in c:\\winapp\\miniconda3\\lib\\site-packages (1.0.0)\n"
          ]
        }
      ],
      "source": [
        "import sys\n",
        "!{sys.executable} -m pip install python-gedcom"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This library takes away some of the technical problems with file parsing, but it still gives us pretty low-level access to all individuals and families in the tree. Here is how we can parse the file, and show the list of all individuals:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "source": [
        "from gedcom.parser import Parser\n",
        "from gedcom.element.individual import IndividualElement\n",
        "from gedcom.element.family import FamilyElement\n",
        "g = Parser()\n",
        "g.parse_file('data/tsars.ged')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "scrolled": true,
        "trusted": true
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[('@0@', ('Mihail Fedorovich', 'Romanov')),\n",
              " ('@1@', ('Evdokija Lukjanovna', 'Streshneva')),\n",
              " ('@2@', ('Aleksej Mihajlovich', 'Romanov')),\n",
              " ('@3@', ('Marija Ilinichna', 'Miloslavskaja')),\n",
              " ('@4@', ('Natalja Kirillovna', 'Naryshkina')),\n",
              " ('@5@', ('Marfa Matveevna', 'Apraksina')),\n",
              " ('@6@', ('Fedor Alekseevich', 'Romanov')),\n",
              " ('@7@', ('Sofja Aleksevna', 'Romanova')),\n",
              " ('@8@', ('Ivan V Alekseevich', 'Romanov')),\n",
              " ('@9@', ('Praskovja Fedorovna', 'Saltykova')),\n",
              " ('@10@', ('Ekaterina Ivanovna', 'Romanova')),\n",
              " ('@11@', ('Anna Ivanovna', 'Romanova')),\n",
              " ('@12@', ('Fridrih Vilgelm', 'Kurlandskij')),\n",
              " ('@13@', ('Karl Leopold', 'Meklenburg-Shverinskij')),\n",
              " ('@14@', ('Anna Leopoldovna', 'Meklenburg-Shverinskaja')),\n",
              " ('@15@', ('Anton Ulrih', 'Braunshvejg-Volfenbjuttelskij')),\n",
              " ('@16@', ('Ivan VI Antonovich', 'Braunshvejg-Volfenbjuttelskij')),\n",
              " ('@17@', ('Petr I Alekseevich', 'Romanov')),\n",
              " ('@18@', ('Evdokija Fedorovna', 'Lopuhina')),\n",
              " ('@19@', ('Ekaterina I Alekseevna', 'Mihajlova')),\n",
              " ('@20@', ('Aleksej Petrovich', 'Romanov')),\n",
              " ('@21@', ('Sharlotta Kristina', 'Braunshvejg-Volfenbjuttelskaja')),\n",
              " ('@22@', ('Petr II Alekseevich', 'Romanov')),\n",
              " ('@23@', ('Anna Petrovna', 'Romanova')),\n",
              " ('@24@', ('Elizaveta Petrovna', 'Romanova')),\n",
              " ('@25@', ('Karl Fridrih', 'Golshtejn-Gottorpskij')),\n",
              " ('@26@', ('Petr III Fedorovich', 'Romanov')),\n",
              " ('@27@', ('Ekaterina II', 'Alekseevna')),\n",
              " ('@28@', ('Pavel I Petrovich', 'Romanov')),\n",
              " ('@29@', ('Natalja Alekseevna', 'Gessen-Darmshtadskaja')),\n",
              " ('@30@', ('Marija Fedorovna', 'Vjurtembergskaja')),\n",
              " ('@31@', ('Aleksandr I Pavlovich', 'Romanov')),\n",
              " ('@32@', ('Elizaveta Alekseevna', 'Baden-Durlahskaja')),\n",
              " ('@33@', ('Nikolaj I Pavlovich', 'Romanov')),\n",
              " ('@34@', ('Aleksandra Fedorovna', 'Prusskaja')),\n",
              " ('@35@', ('Aleksandr II Nikolaevich', 'Romanov')),\n",
              " ('@36@', ('Marija Aleksandrovna', 'Gessenskaja')),\n",
              " ('@37@', ('Aleksandr III Aleksandrovich', 'Romanov')),\n",
              " ('@38@', ('Marija Fedorovna', 'Datskaja')),\n",
              " ('@39@', ('Nikolaj II Aleksandrovich', 'Romanov')),\n",
              " ('@40@', ('Aleksandra Fedorovna', 'Gessenskaja'))]"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "d = g.get_element_dictionary()\n",
        "[ (k,v.get_name()) for k,v in d.items() if isinstance(v,IndividualElement)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Here is how we can get information about families. Note that is gives us a list of **identifiers**, and we need to convert them to names if we want more clarity:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[('@41@', ['@0@', '@1@', '@2@']),\n",
              " ('@42@', ['@2@', '@3@', '@6@', '@7@', '@8@']),\n",
              " ('@43@', ['@8@', '@9@', '@10@', '@11@']),\n",
              " ('@44@', ['@13@', '@10@', '@14@']),\n",
              " ('@45@', ['@15@', '@14@', '@16@']),\n",
              " ('@46@', ['@2@', '@4@', '@17@']),\n",
              " ('@47@', ['@17@', '@18@', '@20@']),\n",
              " ('@48@', ['@20@', '@21@', '@22@']),\n",
              " ('@49@', ['@17@', '@19@', '@23@', '@24@']),\n",
              " ('@50@', ['@25@', '@23@', '@26@']),\n",
              " ('@51@', ['@26@', '@27@', '@28@']),\n",
              " ('@52@', ['@28@', '@30@', '@31@', '@33@']),\n",
              " ('@53@', ['@33@', '@34@', '@35@']),\n",
              " ('@54@', ['@35@', '@36@', '@37@']),\n",
              " ('@55@', ['@37@', '@38@', '@39@'])]"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "d = g.get_element_dictionary()\n",
        "[ (k,[x.get_value() for x in v.get_child_elements()]) for k,v in d.items() if isinstance(v,FamilyElement)]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Getting Family Ontology\n",
        "\n",
        "Next, let's have a look at [family ontology](https://raw.githubusercontent.com/blokhin/genealogical-trees/master/data/header.ttl) defined as a set of Semantic Web triplets. This ontology defines such relationships as `isUncleOf`, `isCousinOf`, and many others. All those relationships are defined in terms of basic predicates `isMotherOf`, `isFatherOf`, `isBrotherOf` and `isSisterOf`. We will use automatic reasoning to deduce all other relationships using the ontology.\n",
        "\n",
        "Here is a sample definition of `isAuntOf` property, which is defined as a composition of `isSisterOf` and `isParentOf` (*Aunt is a sister of one's parent*).\n",
        "\n",
        "```\n",
        "fhkb:isAuntOf a owl:ObjectProperty ;\n",
        "    rdfs:domain fhkb:Woman ;\n",
        "    rdfs:range fhkb:Person ;\n",
        "    owl:propertyChainAxiom ( fhkb:isSisterOf fhkb:isParentOf ) .\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 29,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "@prefix fhkb: <http://www.example.com/genealogy.owl#> .\n",
            "@prefix owl: <http://www.w3.org/2002/07/owl#> .\n",
            "@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .\n",
            "@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .\n",
            "@prefix xml: <http://www.w3.org/XML/1998/namespace> .\n",
            "@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .\n",
            "\n",
            "<http://www.example.com/genealogy.owl#> a owl:Ontology .\n",
            "\n",
            "fhkb:DomainEntity a owl:Class .\n",
            "\n",
            "fhkb:Man a owl:Class ;\n",
            "    owl:equivalentClass [ a owl:Class ;\n",
            "            owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n",
            "                        owl:onProperty fhkb:hasSex ;\n",
            "                        owl:someValuesFrom fhkb:Male ] ) ] .\n",
            "\n",
            "fhkb:Woman a owl:Class ;\n",
            "    owl:equivalentClass [ a owl:Class ;\n",
            "            owl:intersectionOf ( fhkb:Person [ a owl:Restriction ;\n"
          ]
        }
      ],
      "source": [
        "!head -20 data/onto.ttl"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Constructing Ontology for Inference\n",
        "\n",
        "For simplicity, we will create one ontology file that will include original rules from family ontology, and facts about individuals from our GEDCOM file. We will go through the GEDCOM file and extract information about families and individuals, and convert them to triplets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 30,
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "source": [
        "!cp data/onto.ttl .\n",
        "\n",
        "gedcom_dict = g.get_element_dictionary()\n",
        "individuals, marriages = {}, {}\n",
        "\n",
        "def term2id(el):\n",
        "    return \"i\" + el.get_pointer().replace('@', '').lower()\n",
        "\n",
        "out = open(\"onto.ttl\",\"a\")\n",
        "\n",
        "for k, v in gedcom_dict.items():\n",
        "    if isinstance(v,IndividualElement):\n",
        "        children, siblings = set(), set()\n",
        "        idx = term2id(v)\n",
        "\n",
        "        title = v.get_name()[0] + \" \" + v.get_name()[1]\n",
        "        title = title.replace('\"', '').replace('[', '').replace(']', '').replace('(', '').replace(')', '').strip()\n",
        "\n",
        "        own_families = g.get_families(v, 'FAMS')\n",
        "        for fam in own_families:\n",
        "            children |= set(term2id(i) for i in g.get_family_members(fam, \"CHIL\"))\n",
        "\n",
        "        parent_families = g.get_families(v, 'FAMC')\n",
        "        if len(parent_families):\n",
        "            for member in g.get_family_members(parent_families[0], \"CHIL\"): # NB adoptive families i.e len(parent_families)>1 are not considered (TODO?)\n",
        "                if member.get_pointer() == v.get_pointer():\n",
        "                    continue\n",
        "                siblings.add(term2id(member))\n",
        "\n",
        "        if idx in individuals:\n",
        "            children |= individuals[idx].get('children', set())\n",
        "            siblings |= individuals[idx].get('siblings', set())\n",
        "        individuals[idx] = {'sex': v.get_gender().lower(), 'children': children, 'siblings': siblings, 'title': title}\n",
        "\n",
        "    elif isinstance(v,FamilyElement):\n",
        "        wife, husb, children = None, None, set()\n",
        "        children = set(term2id(i) for i in g.get_family_members(v, \"CHIL\"))\n",
        "\n",
        "        try:\n",
        "            wife = g.get_family_members(v, \"WIFE\")[0]\n",
        "            wife = term2id(wife)\n",
        "            if wife in individuals: individuals[wife]['children'] |= children\n",
        "            else: individuals[wife] = {'children': children}\n",
        "        except IndexError: pass\n",
        "        try:\n",
        "            husb = g.get_family_members(v, \"HUSB\")[0]\n",
        "            husb = term2id(husb)\n",
        "            if husb in individuals: individuals[husb]['children'] |= children\n",
        "            else: individuals[husb] = {'children': children}\n",
        "        except IndexError: pass\n",
        "\n",
        "        if wife and husb: marriages[wife + husb] = (term2id(v), wife, husb)\n",
        "\n",
        "for idx, val in individuals.items():\n",
        "    added_terms = ''\n",
        "    if val['sex'] == 'f':\n",
        "        parent_predicate, sibl_predicate = \"isMotherOf\", \"isSisterOf\"\n",
        "    else:\n",
        "        parent_predicate, sibl_predicate = \"isFatherOf\", \"isBrotherOf\"\n",
        "    if len(val['children']):\n",
        "        added_terms += \" ;\\n    fhkb:\" + parent_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['children']])\n",
        "    if len(val['siblings']):\n",
        "        added_terms += \" ;\\n    fhkb:\" + sibl_predicate + \" \" + \", \".join([\"fhkb:\" + i for i in val['siblings']])\n",
        "    out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing%s ;\\n    rdfs:label \\\"%s\\\" .\\n\" % (idx, added_terms, val['title']))\n",
        "\n",
        "for k, v in marriages.items():\n",
        "    out.write(\"fhkb:%s a owl:NamedIndividual, owl:Thing ;\\n    fhkb:hasFemalePartner fhkb:%s ;\\n    fhkb:hasMalePartner fhkb:%s .\\n\" % v)\n",
        "\n",
        "out.write(\"[] a owl:AllDifferent ;\\n    owl:distinctMembers (\")\n",
        "for idx in individuals.keys():\n",
        "    out.write(\"    fhkb:\" + idx)\n",
        "for k, v in marriages.items():\n",
        "    out.write(\"    fhkb:\" + v[0])\n",
        "out.write(\"    ) .\")\n",
        "out.close()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 31,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "    fhkb:hasFemalePartner fhkb:i34 ;\n",
            "    fhkb:hasMalePartner fhkb:i33 .\n",
            "fhkb:i54 a owl:NamedIndividual, owl:Thing ;\n",
            "    fhkb:hasFemalePartner fhkb:i36 ;\n",
            "    fhkb:hasMalePartner fhkb:i35 .\n",
            "fhkb:i55 a owl:NamedIndividual, owl:Thing ;\n",
            "    fhkb:hasFemalePartner fhkb:i38 ;\n",
            "    fhkb:hasMalePartner fhkb:i37 .\n",
            "[] a owl:AllDifferent ;\n",
            "    owl:distinctMembers (    fhkb:i0    fhkb:i1    fhkb:i2    fhkb:i3    fhkb:i4    fhkb:i5    fhkb:i6    fhkb:i7    fhkb:i8    fhkb:i9    fhkb:i10    fhkb:i11    fhkb:i12    fhkb:i13    fhkb:i14    fhkb:i15    fhkb:i16    fhkb:i17    fhkb:i18    fhkb:i19    fhkb:i20    fhkb:i21    fhkb:i22    fhkb:i23    fhkb:i24    fhkb:i25    fhkb:i26    fhkb:i27    fhkb:i28    fhkb:i29    fhkb:i30    fhkb:i31    fhkb:i32    fhkb:i33    fhkb:i34    fhkb:i35    fhkb:i36    fhkb:i37    fhkb:i38    fhkb:i39    fhkb:i40    fhkb:i41    fhkb:i42    fhkb:i43    fhkb:i44    fhkb:i45    fhkb:i46    fhkb:i47    fhkb:i48    fhkb:i49    fhkb:i50    fhkb:i51    fhkb:i52    fhkb:i53    fhkb:i54    fhkb:i55    ) .\n"
          ]
        }
      ],
      "source": [
        "!tail onto.ttl"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Doing Inference \n",
        "\n",
        "Now we want to be able to use this ontology for inference and for querying. We will use [RDFLib](https://github.com/RDFLib), library for reading RDF Graph in different formats, querying it, etc. \n",
        "\n",
        "For logical inference, we will use [OWL-RL](https://github.com/RDFLib/OWL-RL) library, which allows us to build **Closure** of the RDF Graph, i.e. add all possible concepts and relations that can be inferred."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Requirement already satisfied: rdflib in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (4.2.2)\n",
            "Requirement already satisfied: isodate in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from rdflib) (0.6.0)\n",
            "Requirement already satisfied: pyparsing in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from rdflib) (2.2.1)\n",
            "Requirement already satisfied: six in /home/nbuser/anaconda3_501/lib/python3.6/site-packages (from isodate->rdflib) (1.11.0)\n",
            "Collecting RDFClosure from git+git://github.com/RDFLib/OWL-RL.git#egg=RDFClosure\n",
            "  Cloning git://github.com/RDFLib/OWL-RL.git to /tmp/pip-install-3jouot5s/RDFClosure\n",
            "Building wheels for collected packages: RDFClosure\n",
            "  Running setup.py bdist_wheel for RDFClosure ... \u001b[?25ldone\n",
            "\u001b[?25h  Stored in directory: /tmp/pip-ephem-wheel-cache-13in8fda/wheels/d8/db/e8/a1d3dea0a6029d7e76b153cfb7129a343a1e357c055c87896b\n",
            "Successfully built RDFClosure\n",
            "Installing collected packages: RDFClosure\n",
            "Successfully installed RDFClosure-5.0.0\n"
          ]
        }
      ],
      "source": [
        "!{sys.executable} -m pip install rdflib\n",
        "!{sys.executable} -m pip install git+https://github.com/RDFLib/OWL-RL.git"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Let's open the ontology file and see how many triplets it contains:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Triplets found:669\n"
          ]
        }
      ],
      "source": [
        "import rdflib\n",
        "from owlrl import DeductiveClosure, OWLRL_Extension\n",
        "\n",
        "g = rdflib.Graph()\n",
        "g.parse(\"onto.ttl\", format=\"turtle\")\n",
        "\n",
        "print(\"Triplets found:%d\" % len(g))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now let's build the closure, and see how the number of triplets increase:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Triplets after inference:4246\n"
          ]
        }
      ],
      "source": [
        "DeductiveClosure(OWLRL_Extension).expand(g)\n",
        "print(\"Triplets after inference:%d\" % len(g))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Querying for Relatives \n",
        "\n",
        "Now we can query the graph to see different relations between people. We can use **SPARQL** language together with `query` method. In our case, let's see all **uncles** in our family tree:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "trusted": true
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fedor Alekseevich Romanov is uncle of Ekaterina Ivanovna Romanova\n",
            "Fedor Alekseevich Romanov is uncle of Anna Ivanovna Romanova\n",
            "Aleksandr I Pavlovich Romanov is uncle of Aleksandr II Nikolaevich Romanov\n"
          ]
        }
      ],
      "source": [
        "qres = g.query(\n",
        "    \"\"\"SELECT DISTINCT ?aname ?bname\n",
        "       WHERE {\n",
        "          ?a fhkb:isUncleOf ?b .\n",
        "          ?a rdfs:label ?aname .\n",
        "          ?b rdfs:label ?bname .\n",
        "       }\"\"\")\n",
        "\n",
        "for row in qres:\n",
        "    print(\"%s is uncle of %s\" % row)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Feel free to experiment with different other family relations. For example, you can have a look at `isAncestorOf` relation, which recurrently defines all ancestors of a given person.\n",
        "\n",
        "Finally, let's clean up!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "trusted": true
      },
      "outputs": [],
      "source": [
        "!rm onto.ttl"
      ]
    }
  ],
  "metadata": {
    "interpreter": {
      "hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
    },
    "kernelspec": {
      "display_name": "Python 3.6",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
