automated Pipeline for parsing profiles of politically exposed persons (PEP) into Wikidata
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

20 lines
942 B

  1. # Settings for the PEP crawler per country to crawl
  2. # Follow the syntax and dont use tbody as it gets added by the browser (when researching xpath through inspector)
  3. # xpath syntax: https://www.w3schools.com/xml/xpath_syntax.asp
  4. # lxml xpath syntax: https://www.geeksforgeeks.org/web-scraping-using-lxml-and-xpath-in-python/
  5. nicaragua:
  6. domain: 'http://legislacion.asamblea.gob.ni'
  7. memberList:
  8. link: 'http://legislacion.asamblea.gob.ni/Tablas%20Generales.nsf/Main.xsp'
  9. parent: '//html//body//form//table//tr//td//table//tr'
  10. child-name: '//td//a/text()'
  11. child-link: '//td//a/@href'
  12. member:
  13. info-1:
  14. parent: '//html//body//form//table'
  15. #child-name: '//html//body//form//table//tr[1]//td[2]//span'
  16. #child-image: '//html//body//form//table//tr[2]//td[1]//span//img'
  17. #child-role: '//html/body/form/table/tr[2]/td[3]/span + label.1'
  18. child-politicalParty: '//tr[6]//td//span//text()'