Semantic_Grammar.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Semantic Grammar.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "machine_shape": "hm"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github//jayralencar/pictoBERT/blob/main/Semantic_Grammar.ipynb \"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/jayralencar/pictoBERT/blob/main/Semantic_Grammar.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>"
      ],
      "metadata": {
        "id": "v6OuTtm8YL1K"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# PictoBERT: Transformers for Pictogram Prediction (semantic grammar)\n",
        "\n",
        "This notebook contains the procedure for constructing the semantic grammar used to compare with PictoBERT fine-tuned to pictogram prediction based on a grammatical structure.\n",
        "\n",
        "In this notebook we replicated the method of [Pereira et al. (2020)](dx.doi.org/10.1007/978-3-030-58323-1_28). Refer to section 5.2.1 of PictoBERT paper."
      ],
      "metadata": {
        "id": "nY1ds4_8ZXzv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Install dependencies"
      ],
      "metadata": {
        "id": "WLSQBXI5aIa7"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4rxAFkTFNzZE",
        "outputId": "8c4f040c-bc0e-4d77-d688-28a39deaaf8e"
      },
      "source": [
        "!pip install transformers rdflib"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting transformers\n",
            "  Downloading transformers-4.17.0-py3-none-any.whl (3.8 MB)\n",
            "\u001b[K     |████████████████████████████████| 3.8 MB 4.2 MB/s \n",
            "\u001b[?25hCollecting rdflib\n",
            "  Downloading rdflib-6.1.1-py3-none-any.whl (482 kB)\n",
            "\u001b[K     |████████████████████████████████| 482 kB 68.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (1.21.5)\n",
            "Collecting sacremoses\n",
            "  Downloading sacremoses-0.0.49-py3-none-any.whl (895 kB)\n",
            "\u001b[K     |████████████████████████████████| 895 kB 50.0 MB/s \n",
            "\u001b[?25hCollecting tokenizers!=0.11.3,>=0.11.1\n",
            "  Downloading tokenizers-0.11.6-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.5 MB)\n",
            "\u001b[K     |████████████████████████████████| 6.5 MB 55.5 MB/s \n",
            "\u001b[?25hRequirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers) (4.11.3)\n",
            "Collecting huggingface-hub<1.0,>=0.1.0\n",
            "  Downloading huggingface_hub-0.4.0-py3-none-any.whl (67 kB)\n",
            "\u001b[K     |████████████████████████████████| 67 kB 5.4 MB/s \n",
            "\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from transformers) (2.23.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers) (3.6.0)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from transformers) (21.3)\n",
            "Collecting pyyaml>=5.1\n",
            "  Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)\n",
            "\u001b[K     |████████████████████████████████| 596 kB 66.0 MB/s \n",
            "\u001b[?25hRequirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers) (2019.12.20)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.7/dist-packages (from transformers) (4.63.0)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<1.0,>=0.1.0->transformers) (3.10.0.2)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.0->transformers) (3.0.7)\n",
            "Collecting isodate\n",
            "  Downloading isodate-0.6.1-py2.py3-none-any.whl (41 kB)\n",
            "\u001b[K     |████████████████████████████████| 41 kB 597 kB/s \n",
            "\u001b[?25hRequirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from rdflib) (57.4.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->transformers) (3.7.0)\n",
            "Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from isodate->rdflib) (1.15.0)\n",
            "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (1.24.3)\n",
            "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2.10)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (2021.10.8)\n",
            "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->transformers) (3.0.4)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (1.1.0)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers) (7.1.2)\n",
            "Installing collected packages: pyyaml, tokenizers, sacremoses, isodate, huggingface-hub, transformers, rdflib\n",
            "  Attempting uninstall: pyyaml\n",
            "    Found existing installation: PyYAML 3.13\n",
            "    Uninstalling PyYAML-3.13:\n",
            "      Successfully uninstalled PyYAML-3.13\n",
            "Successfully installed huggingface-hub-0.4.0 isodate-0.6.1 pyyaml-6.0 rdflib-6.1.1 sacremoses-0.0.49 tokenizers-0.11.6 transformers-4.17.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sx2OZb6INIkt",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b1b00a78-6e57-4033-c520-e30d4853a1f2"
      },
      "source": [
        "!pip install anytree"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting anytree\n",
            "  Downloading anytree-2.8.0-py2.py3-none-any.whl (41 kB)\n",
            "\u001b[?25l\r\u001b[K     |███████▉                        | 10 kB 30.0 MB/s eta 0:00:01\r\u001b[K     |███████████████▊                | 20 kB 9.4 MB/s eta 0:00:01\r\u001b[K     |███████████████████████▋        | 30 kB 7.8 MB/s eta 0:00:01\r\u001b[K     |███████████████████████████████▍| 40 kB 7.3 MB/s eta 0:00:01\r\u001b[K     |████████████████████████████████| 41 kB 293 kB/s \n",
            "\u001b[?25hRequirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.7/dist-packages (from anytree) (1.15.0)\n",
            "Installing collected packages: anytree\n",
            "Successfully installed anytree-2.8.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Download fiels"
      ],
      "metadata": {
        "id": "QxFDNEYUaMRu"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DtTuPyZIN5KQ",
        "outputId": "1b9c0f4a-26e7-467c-8b6a-364657cc3210"
      },
      "source": [
        "!wget http://jayr.clubedosgeeks.com.br/pictobert/tokenizer_sem_childes_uk_clean_2.json"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-03-24 00:52:47--  http://jayr.clubedosgeeks.com.br/pictobert/tokenizer_sem_childes_uk_clean_2.json\n",
            "Resolving jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)... 192.185.214.132\n",
            "Connecting to jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)|192.185.214.132|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 118904 (116K) [application/json]\n",
            "Saving to: ‘tokenizer_sem_childes_uk_clean_2.json’\n",
            "\n",
            "tokenizer_sem_child 100%[===================>] 116.12K   205KB/s    in 0.6s    \n",
            "\n",
            "2022-03-24 00:52:48 (205 KB/s) - ‘tokenizer_sem_childes_uk_clean_2.json’ saved [118904/118904]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://jayr.clubedosgeeks.com.br/pictobert/base.ttl"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZwyvjmSraf-3",
        "outputId": "a5412790-98b8-4ec8-e325-ab2f5a5e3b30"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-03-24 00:53:31--  http://jayr.clubedosgeeks.com.br/pictobert/base.ttl\n",
            "Resolving jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)... 192.185.214.132\n",
            "Connecting to jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)|192.185.214.132|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 1574 (1.5K) [text/turtle]\n",
            "Saving to: ‘base.ttl’\n",
            "\n",
            "base.ttl            100%[===================>]   1.54K  --.-KB/s    in 0s      \n",
            "\n",
            "2022-03-24 00:53:32 (204 MB/s) - ‘base.ttl’ saved [1574/1574]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://jayr.clubedosgeeks.com.br/pictobert/data_sem_childes_clean_2.zip\n",
        "!unzip ./data_sem_childes_clean_2.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bN2IiOp8au70",
        "outputId": "f9f62a23-3df3-4c24-8e91-98249076ed23"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-03-24 00:54:41--  http://jayr.clubedosgeeks.com.br/pictobert/data_sem_childes_clean_2.zip\n",
            "Resolving jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)... 192.185.214.132\n",
            "Connecting to jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)|192.185.214.132|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 2157229 (2.1M) [application/zip]\n",
            "Saving to: ‘data_sem_childes_clean_2.zip’\n",
            "\n",
            "data_sem_childes_cl 100%[===================>]   2.06M  1.56MB/s    in 1.3s    \n",
            "\n",
            "2022-03-24 00:54:43 (1.56 MB/s) - ‘data_sem_childes_clean_2.zip’ saved [2157229/2157229]\n",
            "\n",
            "Archive:  ./data_sem_childes_clean_2.zip\n",
            "   creating: data/\n",
            "  inflating: data/CS_new_test_data.pt  \n",
            "  inflating: data/CS_new_val_data.pt  \n",
            "  inflating: data/CS_new_train_data.pt  \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!wget http://jayr.clubedosgeeks.com.br/pictobert/semantic_grammar_basis.db"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "epAZK1VfgudD",
        "outputId": "d24624d3-d586-4188-a474-cb028652b619"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2022-03-24 01:22:46--  http://jayr.clubedosgeeks.com.br/pictobert/semantic_grammar_basis.db\n",
            "Resolving jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)... 192.185.214.132\n",
            "Connecting to jayr.clubedosgeeks.com.br (jayr.clubedosgeeks.com.br)|192.185.214.132|:80... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 679936 (664K)\n",
            "Saving to: ‘semantic_grammar_basis.db’\n",
            "\n",
            "semantic_grammar_ba 100%[===================>] 664.00K   710KB/s    in 0.9s    \n",
            "\n",
            "2022-03-24 01:22:48 (710 KB/s) - ‘semantic_grammar_basis.db’ saved [679936/679936]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Load tokenizer"
      ],
      "metadata": {
        "id": "vY9QRy82aysy"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "odl3anlyN7Q7"
      },
      "source": [
        "TOKENIZER_PATH = \"./tokenizer_sem_childes_uk_clean_2.json\" # you can change this path to use your custom tokenizer\n",
        "\n",
        "from transformers import PreTrainedTokenizerFast\n",
        "\n",
        "cs_tokenizer = PreTrainedTokenizerFast(tokenizer_file=TOKENIZER_PATH)\n",
        "cs_tokenizer.pad_token = \"[PAD]\"\n",
        "cs_tokenizer.sep_token = \"[SEP]\"\n",
        "cs_tokenizer.mask_token = \"[MASK]\"\n",
        "cs_tokenizer.cls_token = \"[CLS]\"\n",
        "cs_tokenizer.unk_token = \"[UNK]\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5LNOicqVOmTS",
        "outputId": "fe842523-516f-4433-ac3f-1979d41d4ed0"
      },
      "source": [
        "import nltk\n",
        "nltk.download(\"wordnet\")\n",
        "\n",
        "from nltk.corpus import wordnet as wn"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/wordnet.zip.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y0dDeyu2aht2"
      },
      "source": [
        "## Load encoded data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TT3PUTx1ashH"
      },
      "source": [
        "import pickle\n",
        "\n",
        "test_dataset = pickle.load(open(\"./data/CS_new_test_data.pt\",'rb'))\n",
        "train_dataset = pickle.load(open(\"./data/CS_new_train_data.pt\",'rb'))\n",
        "val_dataset = pickle.load(open(\"./data/CS_new_val_data.pt\",'rb'))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Create ontology"
      ],
      "metadata": {
        "id": "V4KsB3zQa6vr"
      }
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fA0Rzdm2PCcA"
      },
      "source": [
        "def create_concept(name):\n",
        "    \n",
        "    concept = URIRef(myns[name])\n",
        "    g.add((concept, RDF.type, ontolex.LexicalConcept))\n",
        "    g.add((concept, RDFS.label, Literal(name)))\n",
        "\n",
        "    # synset = wn.synset(name)\n",
        "    \n",
        "    return concept"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2LZNeIBMPo4y"
      },
      "source": [
        "def create_lexicalized_sense(sense_key, concept):\n",
        "  lexicalized_sense = URIRef(myns[sense_key])\n",
        "  g.add((lexicalized_sense, RDF.type, ontolex.LexicalizedSense))\n",
        "  g.add((lexicalized_sense, RDFS.label, Literal(sense_key)))\n",
        "\n",
        "  g.add((concept, ontolex.lexicalizedSense, lexicalized_sense))\n",
        "\n",
        "  return lexicalized_sense"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ObwsrzfUQPHH"
      },
      "source": [
        "def create_lexical_entry(word, pos, lexicalized_sense, concept):\n",
        "    pos = {\n",
        "        \"v\":'verb',\n",
        "        \"n\":\"noun\",\n",
        "        \"a\": \"adjective\",\n",
        "        \"s\": \"adjective\",\n",
        "        \"r\":'adverb',\n",
        "        'p':'pronoun'\n",
        "    }[pos]\n",
        "    lexicalEntry = URIRef(myns[word+\"_\"+pos+\"_lex\"])\n",
        "    g.add((lexicalEntry, RDF.type, ontolex.Word))\n",
        "    g.add((lexicalEntry, lexinfo.partOfSpeech, lexinfo[pos]))\n",
        "    g.add((lexicalEntry, ontolex.writtenRep, Literal(word)))\n",
        "\n",
        "    canonicalForm = URIRef(myns[word+\"_\"+pos+\"_form\"])\n",
        "    g.add((canonicalForm, RDF.type, ontolex.Form))\n",
        "    g.add((canonicalForm, ontolex.writtenRep, Literal(word)))\n",
        "    g.add((canonicalForm, lexinfo.partOfSpeech, lexinfo[pos]))\n",
        "\n",
        "    g.add((lexicalEntry, ontolex.canonicalForm, canonicalForm))\n",
        "\n",
        "    g.add((lexicalEntry, ontolex.sense, lexicalized_sense))\n",
        "    g.add((lexicalEntry, ontolex.evokes, concept))\n",
        "    g.add((concept, ontolex.isEvokedBy, lexicalEntry))\n",
        "\n",
        "    return lexicalEntry"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8-RgUoShTCCk"
      },
      "source": [
        "def get_all_hypernyms(s):\n",
        "    return set(\n",
        "        self_synset\n",
        "        for self_synsets in s._iter_hypernym_lists()\n",
        "        for self_synset in self_synsets\n",
        "    ).difference(set([s]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-SeAVQRisrVQ"
      },
      "source": [
        "## Complete missing nodes\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EaAIFkgBst5L"
      },
      "source": [
        "vocab = [i for i in cs_tokenizer.get_vocab()]\n",
        "synsets = [wn.lemma_from_key(k).synset() for k in vocab if '%' in k]\n",
        "verb_synsets = [s for s in synsets if s.pos() == 'v']\n",
        "noun_synsets = [s for s in synsets if s.pos() == 'n']"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QrxGXSBPtHb5"
      },
      "source": [
        "from tqdm import tqdm\n",
        "class MissingNodes():\n",
        "    def __init__(self):\n",
        "        self.all_hyps = {}\n",
        "        pass\n",
        "    \n",
        "    def get_missing(self, synsets, root = 'entity.n.01'):\n",
        "        new_nodes = [wn.synset(root)]\n",
        "        i = 1\n",
        "        while i == 1:\n",
        "            i += 1\n",
        "            print(len(new_nodes))\n",
        "            new_nodes = []\n",
        "            for i,s in enumerate(tqdm(synsets)):\n",
        "                all_hp = self.get_all_hypernyms(s)\n",
        "                # print(s,all_hp)\n",
        "                resulsts = []\n",
        "                for s2 in synsets[i:]:\n",
        "                    s2_all_hp = self.get_all_hypernyms(s2)\n",
        "                    if s2 not in all_hp and s not in s2_all_hp:\n",
        "                        subsumer = s.lowest_common_hypernyms(s2)\n",
        "                        score = s.wup_similarity(s2)\n",
        "                        if len(subsumer) > 0 and score > 0.9 :\n",
        "                            # print(s,s2,subsumer[0],score)\n",
        "                            if subsumer[0] not in synsets and subsumer[0] not in new_nodes:\n",
        "                                new_nodes.append(subsumer[0])\n",
        "                                # print(new_nodes)\n",
        "                        # print(s, s2)\n",
        "                    # print(s.wup_similarity(s2))\n",
        "                # ordered = sorted(resulsts,key=lambda k:k['score'],reverse=True)\n",
        "                # print(new_nodes)\n",
        "            synsets = synsets + new_nodes\n",
        "        \n",
        "        return synsets;\n",
        "\n",
        "    \n",
        "    def get_all_hypernyms(self,s):\n",
        "        if s.name() in self.all_hyps:\n",
        "          self.all_hyps[s.name()]\n",
        "        all = list(\n",
        "            self_synset\n",
        "            for self_synsets in s._iter_hypernym_lists()\n",
        "            for self_synset in self_synsets\n",
        "        )\n",
        "        self.all_hyps[s.name()] = all\n",
        "        return all"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3areVzRIhZll"
      },
      "source": [
        "## Process corpus"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SNM2ox2GhZD1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7f12bdba-c8a3-4248-8ebb-f63ab3c31ebc"
      },
      "source": [
        "from tqdm import tqdm\n",
        "semantic_model = {}\n",
        "roles = []\n",
        "for example in tqdm(train_dataset['input_ids'] + val_dataset['input_ids']):\n",
        "  cls, who_id, what_doing_id, what_id, where_id,to_whom_id, how_id, when_id,  sep = example\n",
        "  who = cs_tokenizer.convert_ids_to_tokens(who_id)\n",
        "  what_doing = cs_tokenizer.convert_ids_to_tokens(what_doing_id)\n",
        "  what = cs_tokenizer.convert_ids_to_tokens(what_id)\n",
        "  where = cs_tokenizer.convert_ids_to_tokens(where_id)\n",
        "  to_whom = cs_tokenizer.convert_ids_to_tokens(to_whom_id)\n",
        "  how = cs_tokenizer.convert_ids_to_tokens(how_id)\n",
        "  when = cs_tokenizer.convert_ids_to_tokens(when_id)\n",
        "  if \"%\" in what_doing:\n",
        "    synset = wn.lemma_from_key(what_doing).synset().name()\n",
        "    if synset not in semantic_model:\n",
        "      semantic_model[synset] = {\"hasAgent\":[],\"hasTheme\":[], \"hasLocation\":[],\"hasRecipient\":[],'hasManner':[],'hasTime':[]}\n",
        "    if who not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in who:\n",
        "        who_synset = wn.lemma_from_key(who).synset().name()\n",
        "      else:\n",
        "        who_synset = who\n",
        "      semantic_model[synset]['hasAgent'].append(who_synset)\n",
        "    if what not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in what:\n",
        "        what_synset = wn.lemma_from_key(what).synset().name()\n",
        "      else:\n",
        "        what_synset = what\n",
        "      semantic_model[synset]['hasTheme'].append(what_synset)\n",
        "    if where not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in where:\n",
        "        where_synset = wn.lemma_from_key(where).synset().name()\n",
        "      else:\n",
        "        where_synset = where\n",
        "      semantic_model[synset]['hasLocation'].append(where_synset)\n",
        "    if to_whom not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in to_whom:\n",
        "        to_whom_synset = wn.lemma_from_key(to_whom).synset().name()\n",
        "      else:\n",
        "        to_whom_synset = to_whom\n",
        "      semantic_model[synset]['hasRecipient'].append(to_whom_synset)\n",
        "    if how not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in how:\n",
        "        how_synset = wn.lemma_from_key(how).synset().name()\n",
        "      else:\n",
        "        how_synset = how\n",
        "      semantic_model[synset]['hasManner'].append(how_synset)\n",
        "    if when not in cs_tokenizer.all_special_tokens:\n",
        "      if '%' in when:\n",
        "        when_synset = wn.lemma_from_key(when).synset().name()\n",
        "      else:\n",
        "        when_synset = when\n",
        "      semantic_model[synset]['hasTime'].append(when_synset)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 78022/78022 [00:49<00:00, 1579.31it/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJIJ0FILL9An"
      },
      "source": [
        "## importance cut off"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8EVI9_kBL_FJ"
      },
      "source": [
        "from collections import Counter\n",
        "from scipy import stats\n",
        "\n",
        "import math\n",
        "from scipy.stats import shapiro\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "\n",
        "def cut_off(items, alpha, assume_normal = False, change_alpha = True):\n",
        "    # print(stats.norm.ppf(1-alpha))\n",
        "\n",
        "    \n",
        "    counter = Counter(items)\n",
        "    names = counter.keys()\n",
        "    counts = list(counter.values())\n",
        "    a = np.array(counts)\n",
        "    mean = a.mean()\n",
        "    std = a.std()\n",
        "    if len(counts) > 3 and not assume_normal:\n",
        "        k2, p = shapiro(counts)\n",
        "    else:\n",
        "        p = 0\n",
        "    chosen = []\n",
        "    not_chosen = []\n",
        "\n",
        "    if p > 0.05 or assume_normal:\n",
        "        z = stats.norm.ppf(1-alpha)\n",
        "\n",
        "        x = (z*std)+mean\n",
        "\n",
        "    else:\n",
        "        t = stats.t.ppf(1-alpha, len(items))\n",
        "        error = (t * float(std)) / math.sqrt(len(items));\n",
        "\n",
        "        x = error+mean\n",
        "\n",
        "    x = round(x)\n",
        "    for name, qt in counter.items():\n",
        "        if qt >= x:\n",
        "            chosen.append((name, qt))\n",
        "        else:\n",
        "            not_chosen.append((name, qt))\n",
        "    if len(chosen)==0 and alpha < 0.5 and change_alpha:\n",
        "        return cut_off(items, alpha+0.05)\n",
        "    return chosen,not_chosen"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XJfmlvSlNAEA"
      },
      "source": [
        "## Redundancy removing"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Lz2XkbEbNF9q"
      },
      "source": [
        "from rdflib import Graph, Namespace, RDFS, RDF, Literal, URIRef, OWL,BNode\n",
        "from rdflib.plugins.sparql import prepareQuery\n",
        "from collections import Counter\n",
        "from anytree import Node, RenderTree, ZigZagGroupIter,LevelOrderGroupIter\n",
        "from anytree.exporter import DotExporter\n",
        "\n",
        "class Redundancy():\n",
        "    def __init__(self, graph):\n",
        "        self.g = graph\n",
        "        self.ontolex = Namespace(\"http://www.w3.org/ns/lemon/ontolex#\")\n",
        "        self.decomp = Namespace(\"http://www.w3.org/ns/lemon/decomp#\")\n",
        "        self.lexinfo = Namespace(\"http://www.lexinfo.net/ontology/2.0/lexinfo#\")\n",
        "        self.synsem = Namespace(\"http://www.w3.org/ns/lemon/synsem#\")\n",
        "        self.myns = Namespace(\"http://assistive.cin.ufpe.br/aboard#\")\n",
        "    \n",
        "    def get_hypernyms(self,synset_name,reference):\n",
        "        el = URIRef(self.myns[synset_name])\n",
        "        hypernyms = []\n",
        "        for s,v,p in self.g.triples((el, self.lexinfo.hypernym, None)):\n",
        "            if self.g.qname(p) in reference:\n",
        "                hypernyms.append(self.g.qname(p))\n",
        "        \n",
        "        return hypernyms\n",
        "\n",
        "    def replace(self,_list, old, new):\n",
        "        return [new if x==old else x for x in _list]\n",
        "    \n",
        "    def by_frequency(self, initial):\n",
        "        counter = Counter(initial)\n",
        "        names = counter.keys()\n",
        "        counts = counter.values()\n",
        "        coisa = {}\n",
        "        pais = []\n",
        "        not_pai = []\n",
        "        for name, qt in counter.items():\n",
        "            hypernyms = self.get_hypernyms(name, names)\n",
        "            node_name =name\n",
        "            if len(hypernyms) > 0 and node_name not in not_pai:\n",
        "                not_pai.append(node_name)\n",
        "            for hypernym in hypernyms:\n",
        "                hypernym_node = hypernym;\n",
        "                if node_name in pais:\n",
        "                    pais.remove(node_name)\n",
        "                if hypernym_node not in pais and hypernym_node not in not_pai:\n",
        "                    pais.append(hypernym_node)\n",
        "                if hypernym_node not in coisa:\n",
        "                    coisa[hypernym_node] = Node(hypernym_node)\n",
        "                if node_name not in coisa:\n",
        "                    coisa[node_name] = Node(node_name)\n",
        "                coisa[node_name].parent = coisa[hypernym_node]\n",
        "\n",
        "\n",
        "        for a in pais:\n",
        "\n",
        "            asda = [[node.name for node in children] for children in ZigZagGroupIter(coisa[a])]\n",
        "            asda.reverse()\n",
        "            for level in asda:\n",
        "                for item in level:\n",
        "                    qt = counter[item]\n",
        "                    if coisa[item].parent != None:\n",
        "                        qt_parent = counter[coisa[item].parent.name]\n",
        "                        if qt_parent > qt:\n",
        "                            initial = self.replace(initial,item, coisa[item].parent.name)\n",
        "                            counter = Counter(initial)\n",
        "        return initial\n",
        "\n",
        "    def get_parent_recursive(self,synset_name, reference):\n",
        "        el = URIRef(self.myns[synset_name])\n",
        "        for s,v,p in self.g.triples((el, self.lexinfo.hypernym, None)):\n",
        "            if self.g.qname(p) in reference:\n",
        "                return self.g.qname(p)\n",
        "            _parent = self.get_parent_recursive(self.g.qname(p), reference)\n",
        "            if _parent is not None:\n",
        "                return _parent\n",
        "        return None\n",
        "\n",
        "    def parent_preference(self, initial, arr = False, keys = False):\n",
        "        _obj = {}\n",
        "        if arr:\n",
        "            counter = Counter(initial)\n",
        "            _obj = counter\n",
        "        else:\n",
        "            for name, qt in initial:\n",
        "                _obj[name] = qt\n",
        "\n",
        "            counter = Counter(_obj)\n",
        "        names = list(counter.keys())\n",
        "        isChanged = False\n",
        "\n",
        "        for name in names:\n",
        "            parent = self.get_parent_recursive(name, names)\n",
        "            if parent is not None:\n",
        "                if parent in _obj and name in _obj:\n",
        "                    _obj[parent] = _obj[parent] + _obj[name]\n",
        "                del _obj[name]\n",
        "                isChanged = True\n",
        "        \n",
        "        if isChanged:\n",
        "            if keys:\n",
        "                return _obj.keys()\n",
        "            f_return = []\n",
        "            for a in _obj:\n",
        "                f_return.append((a,_obj[a]))\n",
        "            return f_return\n",
        "        else:\n",
        "            if arr and not keys:\n",
        "                f_return = []\n",
        "                for a in _obj:\n",
        "                    f_return.append((a,_obj[a]))\n",
        "                return f_return\n",
        "            return initial;"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Lvbp-i54OuvA"
      },
      "source": [
        "## Building ontology"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eMXKDzSTOxR-"
      },
      "source": [
        "from rdflib import Graph, Namespace, RDFS, RDF, Literal, URIRef, OWL,BNode\n",
        "from rdflib.plugins.sparql import prepareQuery\n",
        "\n",
        "g = Graph()\n",
        "g.parse(\"./base.ttl\")\n",
        "\n",
        "ontolex = Namespace(\"http://www.w3.org/ns/lemon/ontolex#\")\n",
        "decomp = Namespace(\"http://www.w3.org/ns/lemon/decomp#\")\n",
        "lexinfo = Namespace(\"http://www.lexinfo.net/ontology/2.0/lexinfo#\")\n",
        "synsem = Namespace(\"http://www.w3.org/ns/lemon/synsem#\")\n",
        "myns = Namespace(\"http://assistive.cin.ufpe.br/aboard#\")\n",
        "skos = Namespace(\"http://www.w3.org/2004/02/skos/core#\")\n",
        "\n",
        "\n",
        "g.bind(\"ontolex\",ontolex)\n",
        "g.bind(\"decomp\",decomp)\n",
        "g.bind(\"lexinfo\", lexinfo)\n",
        "g.bind(\"synsem\", synsem)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PuxM-xHNh9vj"
      },
      "source": [
        "for token in cs_tokenizer.get_vocab():\n",
        "  if '%' not in token:\n",
        "    concept = create_concept(token)\n",
        "    lexicalized_sense = create_lexicalized_sense(token, concept)\n",
        "    lexical_entry = create_lexical_entry(token, \"p\", lexicalized_sense, concept)    "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3xT1hCmPQyrQ"
      },
      "source": [
        "nouns_synsets = []\n",
        "all_synsets = []\n",
        "for token in cs_tokenizer.get_vocab():\n",
        "  if '%' in token:\n",
        "    try:\n",
        "      l = wn.lemma_from_key(token)\n",
        "      concept = create_concept(l.synset().name())\n",
        "      lexicalized_sense = create_lexicalized_sense(token, concept)\n",
        "      lexical_entry = create_lexical_entry(l.name(), l.synset().pos(), lexicalized_sense, concept)\n",
        "      \n",
        "      if l.synset().pos() == 'n':\n",
        "        nouns_synsets.append(l.synset())\n",
        "      all_synsets.append(l.synset())\n",
        "    except:\n",
        "      print(token)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c7GnqFrZSZvR"
      },
      "source": [
        "for s in nouns_synsets:\n",
        "    # print(s)\n",
        "    concept = create_concept(s.name())\n",
        "    all_hyps = list(get_all_hypernyms(s))\n",
        "    in_list = set(all_hyps).intersection(set(nouns_synsets))\n",
        "\n",
        "    if len(in_list) > 0:\n",
        "        depts = [{\"synset\":s,\"depth\":s.max_depth()} for s in list(in_list)]\n",
        "        parent = sorted(depts,key=lambda k:k['depth'],reverse=True)[0]\n",
        "        hy_concept = create_concept(parent['synset'].name())\n",
        "        g.add((concept, lexinfo.hypernym, hy_concept))\n",
        "        g.add((hy_concept, lexinfo.hyponym, concept))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dZgASSwrN68v"
      },
      "source": [
        "## Create Semantic Roles in the graph"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MWIXfrvBN9a1"
      },
      "source": [
        "semanticRole = URIRef(myns[\"semanticRole\"])\n",
        "g.add((semanticRole, RDF.type, OWL.ObjectProperty))\n",
        "\n",
        "frequency = URIRef(myns['frequency'])\n",
        "g.add((frequency, RDF.type, OWL.AnnotationProperty))\n",
        "\n",
        "def create_semantic_role(name):\n",
        "    sm = URIRef(myns[name])\n",
        "    g.add((sm, RDF.type, OWL.ObjectProperty))\n",
        "    g.add((sm, RDFS.subPropertyOf, myns.semanticRole))\n",
        "    return sm"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Transform into database\n",
        "\n",
        "To facilitate queries we transform the ontologies in relational databases"
      ],
      "metadata": {
        "id": "724Zmef4eBUQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!mkdir db"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Xa8otWKTg_zy",
        "outputId": "df77ba81-517e-4ce8-95a4-29e8a4de1d29"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mkdir: cannot create directory ‘db’: File exists\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import sqlite3\n",
        "\n",
        "pos_map = {'n':'noun','v':'verb','a':'adjective','s':'adjective','r':'adverb'}\n",
        "\n",
        "def db_create_concept(conn,name):\n",
        "    c = conn.cursor()\n",
        "    for row in c.execute('SELECT * FROM concept WHERE name = \"'+name+'\"'):\n",
        "        return row[0]\n",
        "    c.execute(\"INSERT INTO concept (name) VALUES (?)\",(name,))\n",
        "    return c.lastrowid\n",
        "\n",
        "def db_create_word(conn,name, pos, w_type=1, parent=None):\n",
        "    c = conn.cursor()\n",
        "    for row in c.execute(\"SELECT * FROM pictogram WHERE name = ?  and pos = ?  and type= ?\",(name,pos,str(w_type)) ):\n",
        "        return row[0]\n",
        "    c.execute(\"INSERT INTO pictogram (name,pos,type,image_file,parent) VALUES (?,?,?,?,?)\",(name,pos,w_type,'...',parent,))\n",
        "    return c.lastrowid\n",
        "\n",
        "def db_create_word_concept(conn,concept_id, word_id):\n",
        "    c = conn.cursor()\n",
        "    for row in c.execute(\"SELECT * FROM pictogram_concept WHERE concept_id = ? AND pictogram_id = ?\", (concept_id, word_id,)):\n",
        "        return row[0]\n",
        "    c.execute(\"INSERT INTO pictogram_concept (concept_id, pictogram_id) VALUES (?,?)\",(concept_id, word_id,))\n",
        "    return c.lastrowid\n",
        "\n",
        "def db_create_tax(conn,hypernym_id, hyponym_id):\n",
        "    c = conn.cursor()\n",
        "    for row in c.execute(\"SELECT * FROM taxonomic_relationship WHERE hypernym_id = ? AND hyponym_id = ?\", (hypernym_id, hyponym_id,)):\n",
        "        return row[0]\n",
        "    c.execute(\"INSERT INTO taxonomic_relationship (hypernym_id, hyponym_id) VALUES (?,?)\",(hypernym_id, hyponym_id,))\n",
        "    return c.lastrowid\n",
        "\n",
        "def db_create_semantic_rel(conn,semantic_role, source_concept_id, destination_concept_id, frequency):\n",
        "    c = conn.cursor()\n",
        "    for row in c.execute(\"SELECT * FROM semantic_relationship WHERE semantic_role = '\"+semantic_role+\"' and source_concept_id =? AND destination_concept_id = ?\",(source_concept_id, destination_concept_id,)):\n",
        "        return row[0]\n",
        "    c.execute(\"INSERT INTO semantic_relationship (semantic_role, source_concept_id, destination_concept_id, frequency) VALUES (?,?,?,?)\",(semantic_role, source_concept_id, destination_concept_id, frequency,))\n",
        "    return c.lastrowid\n",
        "\n",
        "\n",
        "def to_database(g,percentil):\n",
        "  path = \"./db/semantic_grammar_{0}.db\".format(percentil)\n",
        "  !cp ./semantic_grammar_basis.db $path\n",
        "\n",
        "  conn = sqlite3.connect(path)\n",
        "\n",
        "  delete_cursor = conn.cursor()\n",
        "\n",
        "  delete_cursor.execute(\"DELETE FROM concept;\")\n",
        "  delete_cursor.execute(\"DELETE FROM sqlite_sequence WHERE name = 'concept';\")\n",
        "  # delete_cursor.execute(\"DELETE FROM pictogram; \")\n",
        "  # delete_cursor.execute(\"DELETE FROM sqlite_sequence WHERE name = 'pictogram';\")\n",
        "  delete_cursor.execute(\"DELETE FROM pictogram_concept; \")\n",
        "  delete_cursor.execute(\"DELETE FROM sqlite_sequence WHERE name = 'pictogram_concept';\")\n",
        "  delete_cursor.execute(\"DELETE FROM semantic_relationship; \")\n",
        "  delete_cursor.execute(\"DELETE FROM sqlite_sequence WHERE name = 'semantic_relationship';\")\n",
        "  delete_cursor.execute(\"DELETE FROM taxonomic_relationship; \")\n",
        "  delete_cursor.execute(\"DELETE FROM sqlite_sequence WHERE name = 'taxonomic_relationship';\")\n",
        "\n",
        "  delete_cursor.close()\n",
        "  for s, p, o in tqdm(g.triples((None, RDF.type, ontolex.LexicalConcept))):\n",
        "      concept_name = g.label(s)\n",
        "      concept_id = db_create_concept(conn,concept_name)\n",
        "  for word,p,o in tqdm(g.triples((None, RDF.type, ontolex.Word))):\n",
        "    form = g.value(word, ontolex.canonicalForm, None)\n",
        "    writtenRep = g.value(form, ontolex.writtenRep, None)\n",
        "    for s,p,synset in g.triples((word, ontolex.evokes, None)):\n",
        "        if synset is None:\n",
        "            synset = g.value(word, ontolex.evokes, None)\n",
        "        pos = g.qname(g.value(word, lexinfo.partOfSpeech, None)).split(':')[1]\n",
        "        \n",
        "        collection = g.value(None, skos.member, synset)\n",
        "\n",
        "        category_name = g.label(collection).split('-')[0]\n",
        "        parent_id = db_create_word(conn,\" \".join(category_name.lower().split(\"_\")),'folder',w_type=2)\n",
        "\n",
        "        word_id = db_create_word(conn,writtenRep.lower(),pos,parent=parent_id)\n",
        "\n",
        "        concept_name = g.label(synset)\n",
        "        concept_id = db_create_concept(conn,concept_name)\n",
        "        word_concept_id = db_create_word_concept(conn,concept_id,word_id)\n",
        "\n",
        "  frequencyQry = \"\"\"\n",
        "      SELECT ?frequency WHERE {\n",
        "          ?axiom rdf:type owl:Axiom .\n",
        "          ?axiom owl:annotatedProperty ?property .\n",
        "          ?axiom owl:annotatedSource ?source .\n",
        "          ?axiom owl:annotatedTarget ?target .\n",
        "          ?axiom myns:frequency ?frequency\n",
        "      }\n",
        "  \"\"\"\n",
        "  preparedQry = prepareQuery(frequencyQry,initNs={\"rdf\":RDF,\"owl\":OWL,\"myns\":myns})\n",
        "  i = 0\n",
        "  for s, p, o in tqdm(g.triples((None, RDF.type, ontolex.LexicalConcept))):\n",
        "      i = i + 1\n",
        "      concept_name = g.label(s)\n",
        "      concept_id = db_create_concept(conn,concept_name)\n",
        "\n",
        "      for hypo, prop, hyper in g.triples((s, lexinfo.hypernym, None)):\n",
        "          # print(hypo)\n",
        "          hypernym_id = db_create_concept(conn,g.qname(hyper))\n",
        "          rel = db_create_tax(conn,hypernym_id, concept_id)\n",
        "\n",
        "      for semantic_role, prop, o in g.triples((None, RDFS.subPropertyOf, myns.semanticRole)):\n",
        "          for _s,_p,destination in g.triples((s, semantic_role, None)):\n",
        "              destination_id = db_create_concept(conn,g.qname(destination))\n",
        "              frequency = 0\n",
        "              a = db_create_semantic_rel(conn,g.qname(semantic_role),concept_id,destination_id,frequency)\n",
        "\n",
        "  conn.commit()\n",
        "  conn.close()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NgCQcuZ6g4Kd",
        "outputId": "d6bcea53-6560-48d9-ed6d-ecc15b8e8287"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "4514it [00:00, 4956.21it/s]\n",
            "3511it [00:08, 432.78it/s]\n",
            "4514it [00:07, 618.04it/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## RUN"
      ],
      "metadata": {
        "id": "urpV575KkVdf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "percentiles = [0.01,0.05,0.1,0.15,0.2,6,12,24,32,40]"
      ],
      "metadata": {
        "id": "qX4_0jkace9i"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!mkdir ontologies"
      ],
      "metadata": {
        "id": "ah9xxk9ndwbQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from collections import Counter\n",
        "import copy\n",
        "\n",
        "for percentile in tqdm(percentiles):\n",
        "  my_g = copy.deepcopy(g)\n",
        "  redundancy = Redundancy(g)\n",
        "  most_common=None\n",
        "  if isinstance(percentile, int):\n",
        "    most_common=percentile\n",
        "  \n",
        "  for synset in semantic_model:\n",
        "    for arg in semantic_model[synset]:\n",
        "      items = semantic_model[synset][arg]\n",
        "      if len(items) > 0:\n",
        "        redundance_removed = redundancy.by_frequency(items)\n",
        "        if most_common is None:\n",
        "          accepted, rejected = cut_off(redundance_removed,percentile, assume_normal=True)\n",
        "        else:\n",
        "          c = Counter(redundance_removed)\n",
        "          accepted = c.most_common(most_common)\n",
        "        real_accepted = redundancy.parent_preference(accepted)\n",
        "        for complement_synset, frequency_qt in real_accepted:\n",
        "          g.add((myns[synset], create_semantic_role(arg), myns[complement_synset]))\n",
        "\n",
        "          a = BNode()\n",
        "          g.add((a, RDF.type, OWL.Axiom))\n",
        "          g.add((a, OWL.annotatedSource, myns[synset]))\n",
        "          g.add((a, OWL.annotatedProperty, create_semantic_role(arg)))\n",
        "          g.add((a, OWL.annotatedTarget, myns[complement_synset]))\n",
        "          g.add((a, frequency, Literal(str(frequency_qt))))\n",
        "  my_g.serialize(destination=\"./ontologies/output_CS_f{0}.ttl\".format(percentile), format=\"turtle\")\n",
        "  to_database(my_g, percentile)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ri9qKJLIdEuh",
        "outputId": "4bb0451c-6b8c-4aa8-fe08-0c9d21fffc1d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "  0%|          | 0/10 [00:00<?, ?it/s]\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "1080it [00:00, 10787.41it/s]\u001b[A\n",
            "2159it [00:00, 7309.68it/s] \u001b[A\n",
            "2954it [00:00, 5790.56it/s]\u001b[A\n",
            "3584it [00:00, 4860.62it/s]\u001b[A\n",
            "4514it [00:00, 4746.98it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "64it [00:00, 635.96it/s]\u001b[A\n",
            "128it [00:00, 395.26it/s]\u001b[A\n",
            "173it [00:00, 371.93it/s]\u001b[A\n",
            "213it [00:00, 327.47it/s]\u001b[A\n",
            "248it [00:00, 323.63it/s]\u001b[A\n",
            "282it [00:00, 309.70it/s]\u001b[A\n",
            "315it [00:00, 314.46it/s]\u001b[A\n",
            "353it [00:01, 327.40it/s]\u001b[A\n",
            "395it [00:01, 353.14it/s]\u001b[A\n",
            "432it [00:01, 357.36it/s]\u001b[A\n",
            "469it [00:01, 328.35it/s]\u001b[A\n",
            "503it [00:01, 295.80it/s]\u001b[A\n",
            "546it [00:01, 330.06it/s]\u001b[A\n",
            "581it [00:01, 331.11it/s]\u001b[A\n",
            "617it [00:01, 338.18it/s]\u001b[A\n",
            "653it [00:01, 343.68it/s]\u001b[A\n",
            "688it [00:02, 317.35it/s]\u001b[A\n",
            "727it [00:02, 336.22it/s]\u001b[A\n",
            "762it [00:02, 323.90it/s]\u001b[A\n",
            "796it [00:02, 327.71it/s]\u001b[A\n",
            "830it [00:02, 323.83it/s]\u001b[A\n",
            "864it [00:02, 328.07it/s]\u001b[A\n",
            "902it [00:02, 341.66it/s]\u001b[A\n",
            "944it [00:02, 361.82it/s]\u001b[A\n",
            "984it [00:02, 372.43it/s]\u001b[A\n",
            "1022it [00:03, 342.62it/s]\u001b[A\n",
            "1063it [00:03, 360.36it/s]\u001b[A\n",
            "1100it [00:03, 354.19it/s]\u001b[A\n",
            "1139it [00:03, 363.53it/s]\u001b[A\n",
            "1176it [00:03, 363.57it/s]\u001b[A\n",
            "1213it [00:03, 363.08it/s]\u001b[A\n",
            "1250it [00:03, 350.44it/s]\u001b[A\n",
            "1290it [00:03, 363.61it/s]\u001b[A\n",
            "1330it [00:03, 371.47it/s]\u001b[A\n",
            "1368it [00:03, 363.57it/s]\u001b[A\n",
            "1405it [00:04, 365.22it/s]\u001b[A\n",
            "1448it [00:04, 381.93it/s]\u001b[A\n",
            "1487it [00:04, 375.16it/s]\u001b[A\n",
            "1525it [00:04, 373.30it/s]\u001b[A\n",
            "1567it [00:04, 386.89it/s]\u001b[A\n",
            "1606it [00:04, 385.50it/s]\u001b[A\n",
            "1646it [00:04, 388.55it/s]\u001b[A\n",
            "1690it [00:04, 402.22it/s]\u001b[A\n",
            "1732it [00:04, 406.29it/s]\u001b[A\n",
            "1773it [00:04, 395.89it/s]\u001b[A\n",
            "1813it [00:05, 385.23it/s]\u001b[A\n",
            "1862it [00:05, 414.87it/s]\u001b[A\n",
            "1905it [00:05, 417.57it/s]\u001b[A\n",
            "1947it [00:05, 402.20it/s]\u001b[A\n",
            "1993it [00:05, 415.54it/s]\u001b[A\n",
            "2035it [00:05, 416.71it/s]\u001b[A\n",
            "2077it [00:05, 413.73it/s]\u001b[A\n",
            "2124it [00:05, 425.56it/s]\u001b[A\n",
            "2168it [00:05, 428.44it/s]\u001b[A\n",
            "2219it [00:06, 452.34it/s]\u001b[A\n",
            "2266it [00:06, 455.32it/s]\u001b[A\n",
            "2315it [00:06, 462.75it/s]\u001b[A\n",
            "2362it [00:06, 451.15it/s]\u001b[A\n",
            "2408it [00:06, 448.66it/s]\u001b[A\n",
            "2453it [00:06, 448.86it/s]\u001b[A\n",
            "2498it [00:06, 445.70it/s]\u001b[A\n",
            "2547it [00:06, 457.64it/s]\u001b[A\n",
            "2594it [00:06, 456.16it/s]\u001b[A\n",
            "2642it [00:06, 461.23it/s]\u001b[A\n",
            "2689it [00:07, 451.45it/s]\u001b[A\n",
            "2741it [00:07, 470.26it/s]\u001b[A\n",
            "2790it [00:07, 475.05it/s]\u001b[A\n",
            "2839it [00:07, 477.32it/s]\u001b[A\n",
            "2890it [00:07, 484.33it/s]\u001b[A\n",
            "2942it [00:07, 493.08it/s]\u001b[A\n",
            "2993it [00:07, 497.12it/s]\u001b[A\n",
            "3044it [00:07, 500.14it/s]\u001b[A\n",
            "3097it [00:07, 508.20it/s]\u001b[A\n",
            "3148it [00:07, 505.95it/s]\u001b[A\n",
            "3200it [00:08, 507.28it/s]\u001b[A\n",
            "3254it [00:08, 516.10it/s]\u001b[A\n",
            "3309it [00:08, 526.03it/s]\u001b[A\n",
            "3362it [00:08, 517.97it/s]\u001b[A\n",
            "3415it [00:08, 519.77it/s]\u001b[A\n",
            "3511it [00:08, 405.53it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "157it [00:00, 1548.59it/s]\u001b[A\n",
            "312it [00:00, 1009.55it/s]\u001b[A\n",
            "424it [00:00, 889.34it/s] \u001b[A\n",
            "519it [00:00, 779.57it/s]\u001b[A\n",
            "601it [00:00, 769.49it/s]\u001b[A\n",
            "681it [00:00, 772.07it/s]\u001b[A\n",
            "760it [00:00, 737.32it/s]\u001b[A\n",
            "839it [00:01, 751.35it/s]\u001b[A\n",
            "915it [00:01, 680.98it/s]\u001b[A\n",
            "985it [00:01, 620.27it/s]\u001b[A\n",
            "1063it [00:01, 659.79it/s]\u001b[A\n",
            "1131it [00:01, 592.66it/s]\u001b[A\n",
            "1193it [00:01, 563.84it/s]\u001b[A\n",
            "1251it [00:01, 566.18it/s]\u001b[A\n",
            "1325it [00:01, 611.83it/s]\u001b[A\n",
            "1388it [00:01, 599.22it/s]\u001b[A\n",
            "1449it [00:02, 578.80it/s]\u001b[A\n",
            "1508it [00:02, 577.73it/s]\u001b[A\n",
            "1567it [00:02, 567.39it/s]\u001b[A\n",
            "1645it [00:02, 626.74it/s]\u001b[A\n",
            "1709it [00:02, 562.91it/s]\u001b[A\n",
            "1775it [00:02, 579.12it/s]\u001b[A\n",
            "1852it [00:02, 631.05it/s]\u001b[A\n",
            "1917it [00:02, 630.57it/s]\u001b[A\n",
            "1981it [00:02, 616.04it/s]\u001b[A\n",
            "2044it [00:03, 557.30it/s]\u001b[A\n",
            "2102it [00:03, 536.45it/s]\u001b[A\n",
            "2173it [00:03, 579.80it/s]\u001b[A\n",
            "2245it [00:03, 617.46it/s]\u001b[A\n",
            "2308it [00:03, 604.70it/s]\u001b[A\n",
            "2370it [00:03, 574.66it/s]\u001b[A\n",
            "2441it [00:03, 611.06it/s]\u001b[A\n",
            "2513it [00:03, 637.90it/s]\u001b[A\n",
            "2578it [00:03, 609.54it/s]\u001b[A\n",
            "2646it [00:04, 628.40it/s]\u001b[A\n",
            "2715it [00:04, 644.54it/s]\u001b[A\n",
            "2780it [00:04, 610.52it/s]\u001b[A\n",
            "2842it [00:04, 553.27it/s]\u001b[A\n",
            "2899it [00:04, 503.60it/s]\u001b[A\n",
            "2958it [00:04, 525.56it/s]\u001b[A\n",
            "3012it [00:04, 523.11it/s]\u001b[A\n",
            "3072it [00:04, 543.92it/s]\u001b[A\n",
            "3128it [00:05, 539.23it/s]\u001b[A\n",
            "3190it [00:05, 561.86it/s]\u001b[A\n",
            "3247it [00:05, 560.86it/s]\u001b[A\n",
            "3308it [00:05, 571.58it/s]\u001b[A\n",
            "3366it [00:05, 542.91it/s]\u001b[A\n",
            "3421it [00:05, 543.29it/s]\u001b[A\n",
            "3476it [00:05, 536.20it/s]\u001b[A\n",
            "3541it [00:05, 567.85it/s]\u001b[A\n",
            "3599it [00:05, 560.96it/s]\u001b[A\n",
            "3658it [00:05, 562.09it/s]\u001b[A\n",
            "3725it [00:06, 572.04it/s]\u001b[A\n",
            "3785it [00:06, 579.57it/s]\u001b[A\n",
            "3844it [00:06, 553.98it/s]\u001b[A\n",
            "3900it [00:06, 534.80it/s]\u001b[A\n",
            "3975it [00:06, 592.54it/s]\u001b[A\n",
            "4035it [00:06, 566.67it/s]\u001b[A\n",
            "4097it [00:06, 581.10it/s]\u001b[A\n",
            "4161it [00:06, 595.83it/s]\u001b[A\n",
            "4221it [00:06, 571.61it/s]\u001b[A\n",
            "4280it [00:07, 566.16it/s]\u001b[A\n",
            "4337it [00:07, 546.95it/s]\u001b[A\n",
            "4392it [00:07, 430.69it/s]\u001b[A\n",
            "4447it [00:07, 455.25it/s]\u001b[A\n",
            "4514it [00:07, 594.35it/s]\n",
            " 10%|█         | 1/10 [01:09<10:29, 69.99s/it]\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "1072it [00:00, 10709.34it/s]\u001b[A\n",
            "2143it [00:00, 7361.01it/s] \u001b[A\n",
            "2940it [00:00, 5832.14it/s]\u001b[A\n",
            "3573it [00:00, 4997.38it/s]\u001b[A\n",
            "4514it [00:00, 4837.53it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "66it [00:00, 656.77it/s]\u001b[A\n",
            "132it [00:00, 420.98it/s]\u001b[A\n",
            "179it [00:00, 394.71it/s]\u001b[A\n",
            "221it [00:00, 334.70it/s]\u001b[A\n",
            "257it [00:00, 334.64it/s]\u001b[A\n",
            "292it [00:00, 320.12it/s]\u001b[A\n",
            "331it [00:00, 337.88it/s]\u001b[A\n",
            "366it [00:01, 337.66it/s]\u001b[A\n",
            "409it [00:01, 363.41it/s]\u001b[A\n",
            "446it [00:01, 350.95it/s]\u001b[A\n",
            "482it [00:01, 317.76it/s]\u001b[A\n",
            "515it [00:01, 309.50it/s]\u001b[A\n",
            "560it [00:01, 346.71it/s]\u001b[A\n",
            "599it [00:01, 357.58it/s]\u001b[A\n",
            "636it [00:01, 358.53it/s]\u001b[A\n",
            "673it [00:01, 352.80it/s]\u001b[A\n",
            "709it [00:02, 340.48it/s]\u001b[A\n",
            "744it [00:02, 339.73it/s]\u001b[A\n",
            "779it [00:02, 328.89it/s]\u001b[A\n",
            "815it [00:02, 331.94it/s]\u001b[A\n",
            "852it [00:02, 342.35it/s]\u001b[A\n",
            "890it [00:02, 352.93it/s]\u001b[A\n",
            "931it [00:02, 368.72it/s]\u001b[A\n",
            "974it [00:02, 385.43it/s]\u001b[A\n",
            "1013it [00:02, 363.70it/s]\u001b[A\n",
            "1051it [00:02, 364.87it/s]\u001b[A\n",
            "1089it [00:03, 368.10it/s]\u001b[A\n",
            "1126it [00:03, 366.86it/s]\u001b[A\n",
            "1165it [00:03, 367.78it/s]\u001b[A\n",
            "1203it [00:03, 369.77it/s]\u001b[A\n",
            "1241it [00:03, 363.34it/s]\u001b[A\n",
            "1284it [00:03, 382.43it/s]\u001b[A\n",
            "1326it [00:03, 390.32it/s]\u001b[A\n",
            "1366it [00:03, 376.45it/s]\u001b[A\n",
            "1405it [00:03, 380.17it/s]\u001b[A\n",
            "1449it [00:04, 396.91it/s]\u001b[A\n",
            "1489it [00:04, 382.02it/s]\u001b[A\n",
            "1529it [00:04, 387.04it/s]\u001b[A\n",
            "1569it [00:04, 387.88it/s]\u001b[A\n",
            "1612it [00:04, 396.40it/s]\u001b[A\n",
            "1653it [00:04, 395.08it/s]\u001b[A\n",
            "1700it [00:04, 415.25it/s]\u001b[A\n",
            "1742it [00:04, 412.23it/s]\u001b[A\n",
            "1784it [00:04, 410.22it/s]\u001b[A\n",
            "1826it [00:04, 408.58it/s]\u001b[A\n",
            "1872it [00:05, 419.40it/s]\u001b[A\n",
            "1918it [00:05, 427.89it/s]\u001b[A\n",
            "1961it [00:05, 424.80it/s]\u001b[A\n",
            "2006it [00:05, 429.49it/s]\u001b[A\n",
            "2049it [00:05, 426.30it/s]\u001b[A\n",
            "2094it [00:05, 431.96it/s]\u001b[A\n",
            "2140it [00:05, 438.64it/s]\u001b[A\n",
            "2186it [00:05, 444.87it/s]\u001b[A\n",
            "2237it [00:05, 463.30it/s]\u001b[A\n",
            "2289it [00:05, 475.32it/s]\u001b[A\n",
            "2337it [00:06, 463.03it/s]\u001b[A\n",
            "2384it [00:06, 460.72it/s]\u001b[A\n",
            "2431it [00:06, 462.26it/s]\u001b[A\n",
            "2478it [00:06, 452.17it/s]\u001b[A\n",
            "2527it [00:06, 460.95it/s]\u001b[A\n",
            "2579it [00:06, 477.49it/s]\u001b[A\n",
            "2627it [00:06, 468.68it/s]\u001b[A\n",
            "2674it [00:06, 466.73it/s]\u001b[A\n",
            "2726it [00:06, 480.07it/s]\u001b[A\n",
            "2778it [00:07, 489.57it/s]\u001b[A\n",
            "2829it [00:07, 489.63it/s]\u001b[A\n",
            "2883it [00:07, 504.24it/s]\u001b[A\n",
            "2937it [00:07, 513.94it/s]\u001b[A\n",
            "2989it [00:07, 515.59it/s]\u001b[A\n",
            "3041it [00:07, 512.27it/s]\u001b[A\n",
            "3095it [00:07, 518.34it/s]\u001b[A\n",
            "3147it [00:07, 517.99it/s]\u001b[A\n",
            "3201it [00:07, 523.94it/s]\u001b[A\n",
            "3256it [00:07, 530.75it/s]\u001b[A\n",
            "3312it [00:08, 538.32it/s]\u001b[A\n",
            "3366it [00:08, 525.30it/s]\u001b[A\n",
            "3420it [00:08, 527.33it/s]\u001b[A\n",
            "3511it [00:08, 417.71it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "156it [00:00, 1541.91it/s]\u001b[A\n",
            "311it [00:00, 986.36it/s] \u001b[A\n",
            "421it [00:00, 880.09it/s]\u001b[A\n",
            "515it [00:00, 785.61it/s]\u001b[A\n",
            "597it [00:00, 774.16it/s]\u001b[A\n",
            "681it [00:00, 774.61it/s]\u001b[A\n",
            "760it [00:00, 738.60it/s]\u001b[A\n",
            "840it [00:01, 753.41it/s]\u001b[A\n",
            "917it [00:01, 688.57it/s]\u001b[A\n",
            "987it [00:01, 636.90it/s]\u001b[A\n",
            "1068it [00:01, 678.63it/s]\u001b[A\n",
            "1138it [00:01, 605.78it/s]\u001b[A\n",
            "1201it [00:01, 576.97it/s]\u001b[A\n",
            "1274it [00:01, 613.51it/s]\u001b[A\n",
            "1337it [00:01, 595.25it/s]\u001b[A\n",
            "1413it [00:02, 611.65it/s]\u001b[A\n",
            "1475it [00:02, 582.34it/s]\u001b[A\n",
            "1547it [00:02, 617.11it/s]\u001b[A\n",
            "1610it [00:02, 576.33it/s]\u001b[A\n",
            "1679it [00:02, 566.55it/s]\u001b[A\n",
            "1743it [00:02, 585.24it/s]\u001b[A\n",
            "1810it [00:02, 608.28it/s]\u001b[A\n",
            "1880it [00:02, 633.89it/s]\u001b[A\n",
            "1945it [00:02, 592.90it/s]\u001b[A\n",
            "2006it [00:03, 529.21it/s]\u001b[A\n",
            "2070it [00:03, 557.68it/s]\u001b[A\n",
            "2128it [00:03, 540.35it/s]\u001b[A\n",
            "2202it [00:03, 592.89it/s]\u001b[A\n",
            "2264it [00:03, 598.40it/s]\u001b[A\n",
            "2328it [00:03, 604.45it/s]\u001b[A\n",
            "2390it [00:03, 572.10it/s]\u001b[A\n",
            "2473it [00:03, 643.07it/s]\u001b[A\n",
            "2539it [00:03, 633.33it/s]\u001b[A\n",
            "2604it [00:04, 613.31it/s]\u001b[A\n",
            "2673it [00:04, 633.45it/s]\u001b[A\n",
            "2737it [00:04, 609.86it/s]\u001b[A\n",
            "2801it [00:04, 605.58it/s]\u001b[A\n",
            "2862it [00:04, 541.08it/s]\u001b[A\n",
            "2918it [00:04, 494.20it/s]\u001b[A\n",
            "2979it [00:04, 523.70it/s]\u001b[A\n",
            "3033it [00:04, 510.03it/s]\u001b[A\n",
            "3094it [00:04, 534.18it/s]\u001b[A\n",
            "3149it [00:05, 533.91it/s]\u001b[A\n",
            "3217it [00:05, 565.29it/s]\u001b[A\n",
            "3278it [00:05, 576.56it/s]\u001b[A\n",
            "3337it [00:05, 550.69it/s]\u001b[A\n",
            "3393it [00:05, 540.63it/s]\u001b[A\n",
            "3448it [00:05, 517.67it/s]\u001b[A\n",
            "3508it [00:05, 539.92it/s]\u001b[A\n",
            "3563it [00:05, 537.33it/s]\u001b[A\n",
            "3627it [00:05, 565.19it/s]\u001b[A\n",
            "3685it [00:06, 565.40it/s]\u001b[A\n",
            "3742it [00:06, 547.24it/s]\u001b[A\n",
            "3798it [00:06, 541.94it/s]\u001b[A\n",
            "3853it [00:06, 540.74it/s]\u001b[A\n",
            "3908it [00:06, 527.38it/s]\u001b[A\n",
            "3979it [00:06, 579.57it/s]\u001b[A\n",
            "4038it [00:06, 568.56it/s]\u001b[A\n",
            "4102it [00:06, 588.83it/s]\u001b[A\n",
            "4162it [00:06, 587.79it/s]\u001b[A\n",
            "4221it [00:06, 565.55it/s]\u001b[A\n",
            "4279it [00:07, 568.89it/s]\u001b[A\n",
            "4337it [00:07, 531.17it/s]\u001b[A\n",
            "4391it [00:07, 419.68it/s]\u001b[A\n",
            "4445it [00:07, 446.26it/s]\u001b[A\n",
            "4514it [00:07, 590.26it/s]\n",
            " 20%|██        | 2/10 [02:21<09:28, 71.05s/it]\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "1069it [00:00, 10685.02it/s]\u001b[A\n",
            "2138it [00:00, 7117.78it/s] \u001b[A\n",
            "2918it [00:00, 5857.41it/s]\u001b[A\n",
            "3550it [00:00, 4986.03it/s]\u001b[A\n",
            "4514it [00:00, 4806.47it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "65it [00:00, 641.63it/s]\u001b[A\n",
            "130it [00:00, 400.76it/s]\u001b[A\n",
            "176it [00:00, 377.46it/s]\u001b[A\n",
            "217it [00:00, 318.12it/s]\u001b[A\n",
            "251it [00:00, 314.90it/s]\u001b[A\n",
            "284it [00:00, 299.98it/s]\u001b[A\n",
            "320it [00:00, 312.25it/s]\u001b[A\n",
            "356it [00:01, 322.83it/s]\u001b[A\n",
            "399it [00:01, 351.07it/s]\u001b[A\n",
            "435it [00:01, 345.39it/s]\u001b[A\n",
            "471it [00:01, 307.83it/s]\u001b[A\n",
            "503it [00:01, 291.89it/s]\u001b[A\n",
            "547it [00:01, 329.52it/s]\u001b[A\n",
            "582it [00:01, 334.76it/s]\u001b[A\n",
            "617it [00:01, 337.89it/s]\u001b[A\n",
            "654it [00:01, 345.42it/s]\u001b[A\n",
            "689it [00:02, 322.45it/s]\u001b[A\n",
            "726it [00:02, 335.10it/s]\u001b[A\n",
            "761it [00:02, 323.40it/s]\u001b[A\n",
            "794it [00:02, 321.20it/s]\u001b[A\n",
            "827it [00:02, 320.54it/s]\u001b[A\n",
            "861it [00:02, 325.70it/s]\u001b[A\n",
            "897it [00:02, 335.13it/s]\u001b[A\n",
            "938it [00:02, 351.81it/s]\u001b[A\n",
            "979it [00:02, 367.30it/s]\u001b[A\n",
            "1016it [00:03, 350.83it/s]\u001b[A\n",
            "1052it [00:03, 347.51it/s]\u001b[A\n",
            "1087it [00:03, 348.20it/s]\u001b[A\n",
            "1125it [00:03, 350.50it/s]\u001b[A\n",
            "1162it [00:03, 355.39it/s]\u001b[A\n",
            "1198it [00:03, 341.08it/s]\u001b[A\n",
            "1233it [00:03, 335.38it/s]\u001b[A\n",
            "1271it [00:03, 342.67it/s]\u001b[A\n",
            "1316it [00:03, 370.85it/s]\u001b[A\n",
            "1354it [00:03, 348.44it/s]\u001b[A\n",
            "1393it [00:04, 357.50it/s]\u001b[A\n",
            "1430it [00:04, 359.84it/s]\u001b[A\n",
            "1470it [00:04, 370.32it/s]\u001b[A\n",
            "1508it [00:04, 364.12it/s]\u001b[A\n",
            "1552it [00:04, 384.27it/s]\u001b[A\n",
            "1591it [00:04, 379.24it/s]\u001b[A\n",
            "1630it [00:04, 380.43it/s]\u001b[A\n",
            "1670it [00:04, 384.39it/s]\u001b[A\n",
            "1713it [00:04, 395.89it/s]\u001b[A\n",
            "1753it [00:05, 395.85it/s]\u001b[A\n",
            "1793it [00:05, 387.97it/s]\u001b[A\n",
            "1838it [00:05, 404.22it/s]\u001b[A\n",
            "1879it [00:05, 403.43it/s]\u001b[A\n",
            "1923it [00:05, 412.11it/s]\u001b[A\n",
            "1965it [00:05, 408.28it/s]\u001b[A\n",
            "2008it [00:05, 412.82it/s]\u001b[A\n",
            "2053it [00:05, 421.32it/s]\u001b[A\n",
            "2096it [00:05, 420.23it/s]\u001b[A\n",
            "2139it [00:05, 422.63it/s]\u001b[A\n",
            "2186it [00:06, 435.40it/s]\u001b[A\n",
            "2236it [00:06, 453.07it/s]\u001b[A\n",
            "2284it [00:06, 459.29it/s]\u001b[A\n",
            "2330it [00:06, 448.75it/s]\u001b[A\n",
            "2376it [00:06, 447.85it/s]\u001b[A\n",
            "2421it [00:06, 445.56it/s]\u001b[A\n",
            "2466it [00:06, 441.47it/s]\u001b[A\n",
            "2511it [00:06, 425.37it/s]\u001b[A\n",
            "2564it [00:06, 453.07it/s]\u001b[A\n",
            "2610it [00:06, 449.03it/s]\u001b[A\n",
            "2656it [00:07, 452.08it/s]\u001b[A\n",
            "2702it [00:07, 452.03it/s]\u001b[A\n",
            "2751it [00:07, 461.43it/s]\u001b[A\n",
            "2803it [00:07, 476.65it/s]\u001b[A\n",
            "2851it [00:07, 475.14it/s]\u001b[A\n",
            "2903it [00:07, 486.67it/s]\u001b[A\n",
            "2956it [00:07, 497.03it/s]\u001b[A\n",
            "3006it [00:07, 490.47it/s]\u001b[A\n",
            "3058it [00:07, 498.24it/s]\u001b[A\n",
            "3109it [00:07, 500.65it/s]\u001b[A\n",
            "3160it [00:08, 499.58it/s]\u001b[A\n",
            "3211it [00:08, 499.98it/s]\u001b[A\n",
            "3264it [00:08, 506.52it/s]\u001b[A\n",
            "3317it [00:08, 511.64it/s]\u001b[A\n",
            "3369it [00:08, 506.93it/s]\u001b[A\n",
            "3423it [00:08, 515.37it/s]\u001b[A\n",
            "3511it [00:08, 399.64it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "155it [00:00, 1540.61it/s]\u001b[A\n",
            "310it [00:00, 990.39it/s] \u001b[A\n",
            "421it [00:00, 875.12it/s]\u001b[A\n",
            "515it [00:00, 783.66it/s]\u001b[A\n",
            "597it [00:00, 773.64it/s]\u001b[A\n",
            "681it [00:00, 774.06it/s]\u001b[A\n",
            "760it [00:00, 733.42it/s]\u001b[A\n",
            "840it [00:01, 750.81it/s]\u001b[A\n",
            "916it [00:01, 681.69it/s]\u001b[A\n",
            "986it [00:01, 634.73it/s]\u001b[A\n",
            "1068it [00:01, 678.85it/s]\u001b[A\n",
            "1138it [00:01, 606.07it/s]\u001b[A\n",
            "1201it [00:01, 580.71it/s]\u001b[A\n",
            "1276it [00:01, 623.93it/s]\u001b[A\n",
            "1341it [00:01, 604.68it/s]\u001b[A\n",
            "1413it [00:02, 600.64it/s]\u001b[A\n",
            "1474it [00:02, 581.73it/s]\u001b[A\n",
            "1543it [00:02, 610.09it/s]\u001b[A\n",
            "1605it [00:02, 576.94it/s]\u001b[A\n",
            "1679it [00:02, 575.06it/s]\u001b[A\n",
            "1741it [00:02, 585.12it/s]\u001b[A\n",
            "1802it [00:02, 591.26it/s]\u001b[A\n",
            "1876it [00:02, 631.87it/s]\u001b[A\n",
            "1940it [00:02, 606.62it/s]\u001b[A\n",
            "2002it [00:03, 552.01it/s]\u001b[A\n",
            "2062it [00:03, 559.25it/s]\u001b[A\n",
            "2119it [00:03, 552.52it/s]\u001b[A\n",
            "2198it [00:03, 616.92it/s]\u001b[A\n",
            "2261it [00:03, 607.20it/s]\u001b[A\n",
            "2328it [00:03, 611.37it/s]\u001b[A\n",
            "2390it [00:03, 570.77it/s]\u001b[A\n",
            "2474it [00:03, 632.52it/s]\u001b[A\n",
            "2539it [00:03, 624.67it/s]\u001b[A\n",
            "2602it [00:04, 602.87it/s]\u001b[A\n",
            "2671it [00:04, 626.44it/s]\u001b[A\n",
            "2735it [00:04, 603.83it/s]\u001b[A\n",
            "2801it [00:04, 609.23it/s]\u001b[A\n",
            "2863it [00:04, 536.85it/s]\u001b[A\n",
            "2919it [00:04, 501.85it/s]\u001b[A\n",
            "2981it [00:04, 531.52it/s]\u001b[A\n",
            "3036it [00:04, 513.30it/s]\u001b[A\n",
            "3095it [00:04, 531.67it/s]\u001b[A\n",
            "3150it [00:05, 531.62it/s]\u001b[A\n",
            "3217it [00:05, 562.70it/s]\u001b[A\n",
            "3275it [00:05, 567.08it/s]\u001b[A\n",
            "3333it [00:05, 563.54it/s]\u001b[A\n",
            "3390it [00:05, 536.17it/s]\u001b[A\n",
            "3445it [00:05, 506.41it/s]\u001b[A\n",
            "3508it [00:05, 536.91it/s]\u001b[A\n",
            "3563it [00:05, 535.83it/s]\u001b[A\n",
            "3627it [00:05, 564.52it/s]\u001b[A\n",
            "3690it [00:06, 580.15it/s]\u001b[A\n",
            "3749it [00:06, 557.07it/s]\u001b[A\n",
            "3806it [00:06, 541.93it/s]\u001b[A\n",
            "3865it [00:06, 552.35it/s]\u001b[A\n",
            "3921it [00:06, 552.26it/s]\u001b[A\n",
            "3991it [00:06, 591.53it/s]\u001b[A\n",
            "4051it [00:06, 576.22it/s]\u001b[A\n",
            "4112it [00:06, 581.09it/s]\u001b[A\n",
            "4186it [00:06, 623.38it/s]\u001b[A\n",
            "4249it [00:06, 558.85it/s]\u001b[A\n",
            "4317it [00:07, 576.89it/s]\u001b[A\n",
            "4376it [00:07, 438.49it/s]\u001b[A\n",
            "4426it [00:07, 443.41it/s]\u001b[A\n",
            "4514it [00:07, 592.93it/s]\n",
            " 30%|███       | 3/10 [03:35<08:27, 72.44s/it]\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "1083it [00:00, 10826.47it/s]\u001b[A\n",
            "2166it [00:00, 7377.06it/s] \u001b[A\n",
            "2967it [00:00, 5845.47it/s]\u001b[A\n",
            "3603it [00:00, 4904.05it/s]\u001b[A\n",
            "4514it [00:00, 4801.98it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "67it [00:00, 668.06it/s]\u001b[A\n",
            "134it [00:00, 431.82it/s]\u001b[A\n",
            "182it [00:00, 399.60it/s]\u001b[A\n",
            "225it [00:00, 332.57it/s]\u001b[A\n",
            "261it [00:00, 333.27it/s]\u001b[A\n",
            "296it [00:00, 313.70it/s]\u001b[A\n",
            "332it [00:00, 325.69it/s]\u001b[A\n",
            "371it [00:01, 343.24it/s]\u001b[A\n",
            "412it [00:01, 360.27it/s]\u001b[A\n",
            "449it [00:01, 353.01it/s]\u001b[A\n",
            "485it [00:01, 312.70it/s]\u001b[A\n",
            "518it [00:01, 308.84it/s]\u001b[A\n",
            "561it [00:01, 341.21it/s]\u001b[A\n",
            "601it [00:01, 356.84it/s]\u001b[A\n",
            "638it [00:01, 354.89it/s]\u001b[A\n",
            "675it [00:01, 348.94it/s]\u001b[A\n",
            "711it [00:02, 336.96it/s]\u001b[A\n",
            "746it [00:02, 337.73it/s]\u001b[A\n",
            "780it [00:02, 329.84it/s]\u001b[A\n",
            "815it [00:02, 330.67it/s]\u001b[A\n",
            "854it [00:02, 345.70it/s]\u001b[A\n",
            "891it [00:02, 352.25it/s]\u001b[A\n",
            "932it [00:02, 369.00it/s]\u001b[A\n",
            "973it [00:02, 380.06it/s]\u001b[A\n",
            "1012it [00:02, 355.32it/s]\u001b[A\n",
            "1050it [00:02, 362.05it/s]\u001b[A\n",
            "1087it [00:03, 363.72it/s]\u001b[A\n",
            "1125it [00:03, 363.30it/s]\u001b[A\n",
            "1165it [00:03, 365.70it/s]\u001b[A\n",
            "1203it [00:03, 369.77it/s]\u001b[A\n",
            "1241it [00:03, 364.23it/s]\u001b[A\n",
            "1282it [00:03, 376.96it/s]\u001b[A\n",
            "1323it [00:03, 384.73it/s]\u001b[A\n",
            "1362it [00:03, 362.65it/s]\u001b[A\n",
            "1404it [00:03, 376.24it/s]\u001b[A\n",
            "1444it [00:04, 381.91it/s]\u001b[A\n",
            "1483it [00:04, 381.49it/s]\u001b[A\n",
            "1522it [00:04, 382.99it/s]\u001b[A\n",
            "1565it [00:04, 396.02it/s]\u001b[A\n",
            "1606it [00:04, 398.44it/s]\u001b[A\n",
            "1646it [00:04, 398.18it/s]\u001b[A\n",
            "1690it [00:04, 410.02it/s]\u001b[A\n",
            "1732it [00:04, 411.56it/s]\u001b[A\n",
            "1774it [00:04, 403.42it/s]\u001b[A\n",
            "1815it [00:04, 393.13it/s]\u001b[A\n",
            "1864it [00:05, 420.52it/s]\u001b[A\n",
            "1908it [00:05, 425.65it/s]\u001b[A\n",
            "1951it [00:05, 422.53it/s]\u001b[A\n",
            "1998it [00:05, 432.74it/s]\u001b[A\n",
            "2042it [00:05, 432.15it/s]\u001b[A\n",
            "2086it [00:05, 424.40it/s]\u001b[A\n",
            "2132it [00:05, 433.32it/s]\u001b[A\n",
            "2177it [00:05, 437.58it/s]\u001b[A\n",
            "2229it [00:05, 460.20it/s]\u001b[A\n",
            "2279it [00:05, 470.84it/s]\u001b[A\n",
            "2327it [00:06, 459.57it/s]\u001b[A\n",
            "2374it [00:06, 461.10it/s]\u001b[A\n",
            "2421it [00:06, 458.62it/s]\u001b[A\n",
            "2467it [00:06, 458.09it/s]\u001b[A\n",
            "2513it [00:06, 448.76it/s]\u001b[A\n",
            "2566it [00:06, 471.02it/s]\u001b[A\n",
            "2614it [00:06, 461.46it/s]\u001b[A\n",
            "2661it [00:06, 454.79it/s]\u001b[A\n",
            "2707it [00:06, 449.78it/s]\u001b[A\n",
            "2757it [00:07, 463.63it/s]\u001b[A\n",
            "2808it [00:07, 474.67it/s]\u001b[A\n",
            "2860it [00:07, 486.19it/s]\u001b[A\n",
            "2912it [00:07, 494.28it/s]\u001b[A\n",
            "2966it [00:07, 507.23it/s]\u001b[A\n",
            "3018it [00:07, 508.36it/s]\u001b[A\n",
            "3070it [00:07, 509.11it/s]\u001b[A\n",
            "3121it [00:07, 506.33it/s]\u001b[A\n",
            "3172it [00:07, 506.58it/s]\u001b[A\n",
            "3224it [00:07, 510.29it/s]\u001b[A\n",
            "3277it [00:08, 515.95it/s]\u001b[A\n",
            "3329it [00:08, 515.90it/s]\u001b[A\n",
            "3381it [00:08, 514.95it/s]\u001b[A\n",
            "3436it [00:08, 524.08it/s]\u001b[A\n",
            "3511it [00:08, 413.37it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "152it [00:00, 1511.84it/s]\u001b[A\n",
            "304it [00:00, 958.56it/s] \u001b[A\n",
            "412it [00:00, 862.09it/s]\u001b[A\n",
            "504it [00:00, 784.05it/s]\u001b[A\n",
            "586it [00:00, 768.20it/s]\u001b[A\n",
            "670it [00:00, 787.70it/s]\u001b[A\n",
            "751it [00:00, 730.58it/s]\u001b[A\n",
            "832it [00:01, 751.32it/s]\u001b[A\n",
            "909it [00:01, 709.74it/s]\u001b[A\n",
            "981it [00:01, 619.80it/s]\u001b[A\n",
            "1057it [00:01, 652.85it/s]\u001b[A\n",
            "1125it [00:01, 608.37it/s]\u001b[A\n",
            "1188it [00:01, 571.22it/s]\u001b[A\n",
            "1247it [00:01, 572.61it/s]\u001b[A\n",
            "1321it [00:01, 615.60it/s]\u001b[A\n",
            "1384it [00:01, 612.51it/s]\u001b[A\n",
            "1447it [00:02, 580.85it/s]\u001b[A\n",
            "1506it [00:02, 573.98it/s]\u001b[A\n",
            "1566it [00:02, 567.76it/s]\u001b[A\n",
            "1644it [00:02, 625.40it/s]\u001b[A\n",
            "1708it [00:02, 562.69it/s]\u001b[A\n",
            "1772it [00:02, 582.16it/s]\u001b[A\n",
            "1849it [00:02, 633.31it/s]\u001b[A\n",
            "1914it [00:02, 635.41it/s]\u001b[A\n",
            "1979it [00:02, 613.71it/s]\u001b[A\n",
            "2042it [00:03, 553.33it/s]\u001b[A\n",
            "2099it [00:03, 534.28it/s]\u001b[A\n",
            "2163it [00:03, 560.89it/s]\u001b[A\n",
            "2236it [00:03, 601.94it/s]\u001b[A\n",
            "2298it [00:03, 602.20it/s]\u001b[A\n",
            "2359it [00:03, 558.33it/s]\u001b[A\n",
            "2428it [00:03, 593.98it/s]\u001b[A\n",
            "2505it [00:03, 641.51it/s]\u001b[A\n",
            "2571it [00:03, 624.36it/s]\u001b[A\n",
            "2635it [00:04, 611.71it/s]\u001b[A\n",
            "2699it [00:04, 617.16it/s]\u001b[A\n",
            "2762it [00:04, 578.67it/s]\u001b[A\n",
            "2821it [00:04, 538.71it/s]\u001b[A\n",
            "2876it [00:04, 518.17it/s]\u001b[A\n",
            "2929it [00:04, 490.37it/s]\u001b[A\n",
            "2988it [00:04, 516.08it/s]\u001b[A\n",
            "3041it [00:04, 506.03it/s]\u001b[A\n",
            "3100it [00:05, 520.06it/s]\u001b[A\n",
            "3158it [00:05, 529.01it/s]\u001b[A\n",
            "3226it [00:05, 570.89it/s]\u001b[A\n",
            "3291it [00:05, 565.50it/s]\u001b[A\n",
            "3348it [00:05, 543.26it/s]\u001b[A\n",
            "3403it [00:05, 539.50it/s]\u001b[A\n",
            "3458it [00:05, 506.70it/s]\u001b[A\n",
            "3516it [00:05, 524.13it/s]\u001b[A\n",
            "3570it [00:05, 522.67it/s]\u001b[A\n",
            "3646it [00:05, 584.46it/s]\u001b[A\n",
            "3705it [00:06, 585.17it/s]\u001b[A\n",
            "3764it [00:06, 560.44it/s]\u001b[A\n",
            "3821it [00:06, 552.17it/s]\u001b[A\n",
            "3877it [00:06, 519.08it/s]\u001b[A\n",
            "3954it [00:06, 585.51it/s]\u001b[A\n",
            "4014it [00:06, 543.42it/s]\u001b[A\n",
            "4084it [00:06, 580.83it/s]\u001b[A\n",
            "4146it [00:06, 590.37it/s]\u001b[A\n",
            "4209it [00:06, 600.37it/s]\u001b[A\n",
            "4270it [00:07, 587.68it/s]\u001b[A\n",
            "4330it [00:07, 526.37it/s]\u001b[A\n",
            "4385it [00:07, 419.39it/s]\u001b[A\n",
            "4436it [00:07, 437.66it/s]\u001b[A\n",
            "4514it [00:07, 586.62it/s]\n",
            " 40%|████      | 4/10 [04:56<07:33, 75.65s/it]\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "1082it [00:00, 10810.37it/s]\u001b[A\n",
            "2164it [00:00, 7050.74it/s] \u001b[A\n",
            "2943it [00:00, 5727.20it/s]\u001b[A\n",
            "3566it [00:00, 4934.83it/s]\u001b[A\n",
            "4514it [00:00, 4784.76it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "67it [00:00, 667.61it/s]\u001b[A\n",
            "134it [00:00, 426.64it/s]\u001b[A\n",
            "182it [00:00, 395.41it/s]\u001b[A\n",
            "224it [00:00, 328.11it/s]\u001b[A\n",
            "260it [00:00, 335.89it/s]\u001b[A\n",
            "296it [00:00, 321.75it/s]\u001b[A\n",
            "333it [00:00, 333.94it/s]\u001b[A\n",
            "373it [00:01, 351.60it/s]\u001b[A\n",
            "413it [00:01, 361.33it/s]\u001b[A\n",
            "450it [00:01, 357.27it/s]\u001b[A\n",
            "487it [00:01, 313.98it/s]\u001b[A\n",
            "520it [00:01, 312.12it/s]\u001b[A\n",
            "564it [00:01, 344.42it/s]\u001b[A\n",
            "604it [00:01, 359.69it/s]\u001b[A\n",
            "642it [00:01, 364.02it/s]\u001b[A\n",
            "679it [00:01, 353.15it/s]\u001b[A\n",
            "715it [00:02, 341.68it/s]\u001b[A\n",
            "750it [00:02, 342.15it/s]\u001b[A\n",
            "785it [00:02, 328.07it/s]\u001b[A\n",
            "819it [00:02, 323.95it/s]\u001b[A\n",
            "855it [00:02, 332.28it/s]\u001b[A\n",
            "895it [00:02, 349.80it/s]\u001b[A\n",
            "936it [00:02, 365.63it/s]\u001b[A\n",
            "976it [00:02, 375.09it/s]\u001b[A\n",
            "1014it [00:02, 356.75it/s]\u001b[A\n",
            "1051it [00:02, 359.33it/s]\u001b[A\n",
            "1090it [00:03, 363.05it/s]\u001b[A\n",
            "1129it [00:03, 368.02it/s]\u001b[A\n",
            "1166it [00:03, 368.35it/s]\u001b[A\n",
            "1203it [00:03, 365.76it/s]\u001b[A\n",
            "1240it [00:03, 358.67it/s]\u001b[A\n",
            "1283it [00:03, 378.27it/s]\u001b[A\n",
            "1324it [00:03, 385.96it/s]\u001b[A\n",
            "1363it [00:03, 366.46it/s]\u001b[A\n",
            "1405it [00:03, 377.54it/s]\u001b[A\n",
            "1448it [00:04, 390.82it/s]\u001b[A\n",
            "1488it [00:04, 385.11it/s]\u001b[A\n",
            "1528it [00:04, 388.45it/s]\u001b[A\n",
            "1568it [00:04, 389.71it/s]\u001b[A\n",
            "1608it [00:04, 391.16it/s]\u001b[A\n",
            "1649it [00:04, 396.16it/s]\u001b[A\n",
            "1695it [00:04, 415.00it/s]\u001b[A\n",
            "1737it [00:04, 410.95it/s]\u001b[A\n",
            "1779it [00:04, 410.22it/s]\u001b[A\n",
            "1821it [00:04, 406.12it/s]\u001b[A\n",
            "1871it [00:05, 429.26it/s]\u001b[A\n",
            "1914it [00:05, 428.15it/s]\u001b[A\n",
            "1957it [00:05, 423.70it/s]\u001b[A\n",
            "2003it [00:05, 431.11it/s]\u001b[A\n",
            "2047it [00:05, 429.44it/s]\u001b[A\n",
            "2090it [00:05, 426.89it/s]\u001b[A\n",
            "2136it [00:05, 435.06it/s]\u001b[A\n",
            "2185it [00:05, 449.98it/s]\u001b[A\n",
            "2237it [00:05, 465.82it/s]\u001b[A\n",
            "2290it [00:05, 484.40it/s]\u001b[A\n",
            "2339it [00:06, 470.72it/s]\u001b[A\n",
            "2387it [00:06, 471.11it/s]\u001b[A\n",
            "2436it [00:06, 474.32it/s]\u001b[A\n",
            "2484it [00:06, 453.74it/s]\u001b[A\n",
            "2534it [00:06, 465.89it/s]\u001b[A\n",
            "2587it [00:06, 483.26it/s]\u001b[A\n",
            "2636it [00:06, 476.70it/s]\u001b[A\n",
            "2684it [00:06, 477.13it/s]\u001b[A\n",
            "2738it [00:06, 495.29it/s]\u001b[A\n",
            "2788it [00:07, 493.39it/s]\u001b[A\n",
            "2838it [00:07, 493.45it/s]\u001b[A\n",
            "2891it [00:07, 504.16it/s]\u001b[A\n",
            "2947it [00:07, 518.15it/s]\u001b[A\n",
            "2999it [00:07, 511.99it/s]\u001b[A\n",
            "3053it [00:07, 519.31it/s]\u001b[A\n",
            "3105it [00:07, 519.40it/s]\u001b[A\n",
            "3157it [00:07, 518.42it/s]\u001b[A\n",
            "3211it [00:07, 522.78it/s]\u001b[A\n",
            "3264it [00:07, 521.13it/s]\u001b[A\n",
            "3320it [00:08, 529.88it/s]\u001b[A\n",
            "3374it [00:08, 530.21it/s]\u001b[A\n",
            "3431it [00:08, 540.69it/s]\u001b[A\n",
            "3511it [00:08, 418.34it/s]\n",
            "\n",
            "0it [00:00, ?it/s]\u001b[A\n",
            "156it [00:00, 1530.77it/s]\u001b[A\n",
            "310it [00:00, 1002.00it/s]\u001b[A\n",
            "421it [00:00, 886.66it/s] \u001b[A\n",
            "515it [00:00, 783.23it/s]\u001b[A\n",
            "597it [00:00, 778.83it/s]\u001b[A\n",
            "681it [00:00, 781.66it/s]\u001b[A\n",
            "761it [00:00, 743.07it/s]\u001b[A\n",
            "840it [00:01, 754.63it/s]\u001b[A\n",
            "917it [00:01, 678.81it/s]\u001b[A\n",
            "987it [00:01, 630.44it/s]\u001b[A\n",
            "1068it [00:01, 671.00it/s]\u001b[A\n",
            "1137it [00:01, 593.26it/s]\u001b[A\n",
            "1199it [00:01, 569.97it/s]\u001b[A\n",
            "1273it [00:01, 612.61it/s]\u001b[A\n",
            "1337it [00:01, 580.54it/s]\u001b[A\n",
            "1412it [00:02, 623.09it/s]\u001b[A\n",
            "1476it [00:02, 563.12it/s]\u001b[A\n",
            "1548it [00:02, 601.49it/s]\u001b[A\n",
            "1611it [00:02, 566.25it/s]\u001b[A\n",
            "1679it [00:02, 565.09it/s]\u001b[A\n",
            "1741it [00:02, 578.25it/s]\u001b[A\n",
            "1805it [00:02, 594.50it/s]\u001b[A\n",
            "1876it [00:02, 625.51it/s]\u001b[A\n",
            "1940it [00:02, 596.55it/s]\u001b[A\n",
            "2001it [00:03, 550.28it/s]\u001b[A\n",
            "2060it [00:03, 559.05it/s]\u001b[A\n",
            "2117it [00:03, 540.40it/s]\u001b[A\n",
            "2195it [00:03, 605.89it/s]\u001b[A\n",
            "2257it [00:03, 605.08it/s]\u001b[A\n",
            "2319it [00:03, 602.50it/s]\u001b[A\n",
            "2380it [00:03, 555.32it/s]\u001b[A\n",
            "2471it [00:03, 652.15it/s]\u001b[A\n",
            "2538it [00:03, 638.28it/s]\u001b[A\n",
            "2603it [00:04, 621.86it/s]\u001b[A\n",
            "2669it [00:04, 630.61it/s]\u001b[A\n",
            "2733it [00:04, 622.03it/s]\u001b[A\n",
            "2796it [00:04, 612.74it/s]\u001b[A\n",
            "2858it [00:04, 538.66it/s]\u001b[A\n",
            "2914it [00:04, 495.76it/s]\u001b[A\n",
            "2976it [00:04, 526.49it/s]\u001b[A\n",
            "3031it [00:04, 512.30it/s]\u001b[A\n",
            "3094it [00:04, 541.16it/s]\u001b[A\n",
            "3150it [00:05, 542.06it/s]\u001b[A\n",
            "3211it [00:05, 559.58it/s]\u001b[A\n",
            "3273it [00:05, 575.98it/s]\u001b[A\n",
            "3332it [00:05, 570.98it/s]\u001b[A\n",
            "3390it [00:05, 547.65it/s]\u001b[A\n",
            "3446it [00:05, 514.89it/s]\u001b[A\n",
            "3508it [00:05, 541.15it/s]\u001b[A\n",
            "3565it [00:05, 540.16it/s]\u001b[A\n",
            "3639it [00:05, 596.17it/s]\u001b[A\n",
            "3700it [00:06, 586.05it/s]\u001b[A\n",
            "3760it [00:06, 541.60it/s]\u001b[A\n",
            "3816it [00:06, 545.23it/s]\u001b[A\n",
            "3872it [00:06, 538.04it/s]\u001b[A\n",
            "3934it [00:06, 560.14it/s]\u001b[A\n",
            "4003it [00:06, 596.90it/s]\u001b[A\n",
            "4064it [00:06, 580.67it/s]\u001b[A\n",
            "4125it [00:06, 588.50it/s]\u001b[A\n",
            "4191it [00:06, 590.28it/s]\u001b[A\n",
            "4251it [00:06, 561.56it/s]\u001b[A\n",
            "4317it [00:07, 581.80it/s]\u001b[A\n",
            "4376it [00:07, 439.72it/s]\u001b[A\n",
            "4426it [00:07, 445.50it/s]\u001b[A\n",
            "4514it [00:07, 593.07it/s]\n",
            " 50%|█████     | 5/10 [06:19<06:32, 78.44s/it]"
          ]
        }
      ]
    }
  ]
}