You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
325 lines
11 KiB
325 lines
11 KiB
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "raw",
|
|
"metadata": {},
|
|
"source": [
|
|
"Alle Funktionen der Klasse Passiv2Aktiv werden importiert"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 1,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from Passiv2Aktiv import *\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "raw",
|
|
"metadata": {},
|
|
"source": [
|
|
"Nun wird die Klasse initialisiert"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 2,
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"p2a = Passiv2Aktiv(None, None, None)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "raw",
|
|
"metadata": {},
|
|
"source": [
|
|
"Die nötigen Datenbanken werden in den Arbeitsspeicher der Session geladen. Hier zeigt sich auch die Stärke des Jupyter Notebooks, da der code einmal ausgeführt werden muss, und dann die folgenden Codezeilen neuausgeführt werden können, ohne das die Datenbanken neu eingeladen werden müssen. Das heißt, die Datenbanken können die ganze Zeit im Arbeitsspeicher ruhen, und trotzdem kann neuer Text atest1 geschrieben werden. Dieser wird dann neu eingeladen mit der oi Klasse und anschließend werden die Passivformen der neuen Sätze ausgetauscht."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"p2a.create_hklDB_from_csv('Aktiv.txt', 'None')\n",
|
|
"p2a.create_hklDB_from_csv('Vorgangspassiv.txt', 'None')\n",
|
|
"p2a.create_hklDB_from_csv('Zustandspassiv.txt', 'None')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stderr",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"/home/alpcentaur/ProjektA/LeichteSprache/code200110/SentSeg/venv/lib/python3.5/site-packages/sklearn/base.py:251: UserWarning: Trying to unpickle estimator CountVectorizer from version 0.22.2 when using version 0.20.0. This might lead to breaking code or invalid results. Use at your own risk.\n",
|
|
" UserWarning)\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"loading spacy..\n",
|
|
"done\n",
|
|
"loading vectorizer..\n",
|
|
"done\n",
|
|
"loading the SGD model..\n",
|
|
"done\n",
|
|
"loading spacy..\n",
|
|
"done\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"p2a.load_DB_into_FASTsearch()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from oi import *\n",
|
|
"oi = oi()\n",
|
|
"\n",
|
|
"sentences, punctuations = oi.ReadDoc2Sent('atest1')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 5,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"verbs of sentence string gehe 4\n",
|
|
"1\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string habe verdammt 13\n",
|
|
"2\n",
|
|
"[39974, 1.0]\n",
|
|
"[39913, 1.0]\n",
|
|
"[['platztest rein 14'], ['2. Person Singular Präteritum reinplatzen']]\n",
|
|
"[['reinplatztest 13'], ['2. Person Singular Präteritum reinplatzen']]\n",
|
|
"[['waren verhext 13'], ['3. Person Plural Präteritum verhexen']]\n",
|
|
"verbs of sentence string gibt 4\n",
|
|
"1\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string wird 4\n",
|
|
"1\n",
|
|
"[40069, 1.0]\n",
|
|
"[40069, 1.0]\n",
|
|
"[['wird verheddert haben 21'], ['3. Person Singular Futur II verheddern']]\n",
|
|
"[['wird verheddert worden sein 27'], ['3. Person Singular Futur II verheddern']]\n",
|
|
"[['wird verheddert gewesen sein 28'], ['3. Person Singular Futur II verheddern']]\n",
|
|
"verbs of sentence string gingen regnete 14\n",
|
|
"2\n",
|
|
"[39913, 1.0]\n",
|
|
"[40045, 1.0]\n",
|
|
"[['verhexten 9'], ['3. Person Plural Präteritum verhexen']]\n",
|
|
"[['wurden verhext 14'], ['3. Person Plural Präteritum verhexen']]\n",
|
|
"[['war verheddert 14'], ['3. Person Singular Präteritum verheddern']]\n",
|
|
"verbs of sentence string war angemalt funktionierte 26\n",
|
|
"3\n",
|
|
"[37394, 2.0]\n",
|
|
"[38744, 2.0]\n",
|
|
"[['hatte dahingeschleppt 21'], ['3. Person Singular Plusquamperfekt dahinschleppen']]\n",
|
|
"[['war dahingeschleppt worden 26'], ['3. Person Singular Plusquamperfekt dahinschleppen']]\n",
|
|
"[['war inventarisiert gewesen 26'], ['3. Person Singular Plusquamperfekt inventarisieren']]\n",
|
|
"verbs of sentence string wurde geliebt 13\n",
|
|
"2\n",
|
|
"[2, 3.0]\n",
|
|
"[39975, 1.0]\n",
|
|
"[['liebte 6'], ['3. Person Singular Präteritum lieben']]\n",
|
|
"[['wurde geliebt 13'], ['3. Person Singular Präteritum lieben']]\n",
|
|
"[['wurde reingeplatzt 18'], ['3. Person Singular Präteritum reinplatzen']]\n",
|
|
"subjectofsentence ['Er']\n",
|
|
"there is a subjecter 1\n",
|
|
"get the tuples and triples to check..\n",
|
|
"done\n",
|
|
"['ihn', 'liebte', 'jemand']\n",
|
|
"genrating the permutations\n",
|
|
"done\n",
|
|
"classifying the probability for right grammar in the filtered permutations..\n",
|
|
"jemand liebte ihn\n",
|
|
"done\n",
|
|
"verbs of sentence string habe 4\n",
|
|
"1\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string war 3\n",
|
|
"1\n",
|
|
"[40057, 1.0]\n",
|
|
"[40057, 1.0]\n",
|
|
"[['hatte verheddert 16'], ['3. Person Singular Plusquamperfekt verheddern']]\n",
|
|
"[['war verheddert worden 21'], ['3. Person Singular Plusquamperfekt verheddern']]\n",
|
|
"[['war verheddert gewesen 22'], ['3. Person Singular Plusquamperfekt verheddern']]\n",
|
|
"verbs of sentence string 0\n",
|
|
"0\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string ist 3\n",
|
|
"1\n",
|
|
"[40051, 1.0]\n",
|
|
"[40051, 1.0]\n",
|
|
"[['hat verheddert 14'], ['3. Person Singular Perfekt verheddern']]\n",
|
|
"[['ist verheddert worden 21'], ['3. Person Singular Perfekt verheddern']]\n",
|
|
"[['ist verheddert gewesen 22'], ['3. Person Singular Perfekt verheddern']]\n",
|
|
"verbs of sentence string 0\n",
|
|
"0\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string 0\n",
|
|
"0\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string ginge 5\n",
|
|
"1\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"verbs of sentence string lieben 6\n",
|
|
"1\n",
|
|
"[40072, 0.0]\n",
|
|
"[40072, 0.0]\n",
|
|
"[['werden verheddert haben 23'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert worden sein 29'], ['3. Person Plural Futur II verheddern']]\n",
|
|
"[['werden verheddert gewesen sein 30'], ['3. Person Plural Futur II verheddern']]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"outsentences = p2a.replacePassivForms(sentences)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 6,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"[['Hallo', 'was', 'gehe', 'denn', 'hier', 'so'], ['Ich', 'habe', 'echt', 'keine', 'Ahnung', 'verdammt'], ['I.', 'd.', 'R.', 'gibt', 'es', 'keine', 'Abschiebungen'], ['Ende', 'd.', 'J.', 'wird', 'alles', 'problematisch'], ['Sie', 'gingen', 'nach', 'Hause,', 'weil', 'es', 'in', 'Strömen', 'regnete'], ['Heute', 'war', 'die', 'Straße', 'blau', 'angemalt,', 'damit', 'der', 'Marathon', 'funktionierte'], ['ihn', 'liebte', 'jemand'], ['Er', 'habe', 'es', 'sehr', 'schwer'], ['Es', 'war', 'die', 'Hose', 'des', 'Gauners'], ['Bliblablub'], ['Sie', 'ist', 'nicht', 'schön', 'heute'], ['Oleoleole'], ['Mannoman'], ['Er', 'ginge', 'nicht', 'schnell'], ['Die', 'Hühner', 'lieben', 'sich', 'nicht']]\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"print(outsentences)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n",
|
|
".\n"
|
|
]
|
|
},
|
|
{
|
|
"data": {
|
|
"text/plain": [
|
|
"'OK'"
|
|
]
|
|
},
|
|
"execution_count": 7,
|
|
"metadata": {},
|
|
"output_type": "execute_result"
|
|
}
|
|
],
|
|
"source": [
|
|
" oi.PrintSplitSentencesToTextFile(punctuations, outsentences, 'atest1out')"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.5.3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|