alpcentaur
/
wd_importPEP
1
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 0 Wiki Activity
automated Pipeline for parsing profiles of politically exposed persons (PEP) into Wikidata
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
20 Commits
1 Branch
329 KiB
Tree: 69480ecc26
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '69480ecc26'
${ noResults }
wd_importPEP/crawlers/useMembersParliamentCrawler.py

16 lines
377 B
Raw Normal View History

changed dir and import structure to be used in root of project
1 year ago
added first two functions of class MembersOfParliamentCrawler
1 year ago
​
formatted with black
1 year ago
​
added first two functions of class MembersOfParliamentCrawler
1 year ago
added function parseMemberData2dictionary()
1 year ago
added first two functions of class MembersOfParliamentCrawler
1 year ago
​
formatted with black
1 year ago
added function parseMemberData2dictionary()
1 year ago
formatted with black
1 year ago
added memberData function with first step loading json, renamed memberListData function
1 year ago
formatted with black
1 year ago
added function parseMemberData2dictionary() for first property political party
1 year ago
added function parseMemberData2dictionary()
1 year ago
 
  1. from crawlers.MembersParliamentCrawler import *
  2. 

  3. 

  4. config = "config.yaml"
  5. listOfCountries = ["nicaragua"]
  6. 

  7. 

  8. Crawler = membersParliamentCrawler(config)
  9. 

  10. # Crawler.downloadMemberListPagesOfCountries(listOfCountries)
  11. 

  12. # Crawler.parseMemberListData2dictionary(listOfCountries)
  13. 

  14. # Crawler.downloadMemberDataHtmls(listOfCountries)
  15. 

  16. Crawler.parseMemberData2dictionary(listOfCountries)