alpcentaur
/
fdb-spider
1
0
Fork 0
Code Issues 0 Pull Requests 0 Projects 0 Releases 2 Wiki Activity
25 Commits
2 Branches
1.5 MiB
Tree: 54daad8dfa
Branches Tags
${ item.name }
Create branch ${ searchTerm }
from '54daad8dfa'
${ noResults }
Commit Graph

16 Commits (54daad8dfa62c561c91923de3c737362abf16b0c)

Author SHA1 Message Date
  alpcentaur 89dcca2031 added further handling for javascript links not being urls, made config for giz work 1 year ago
  alpcentaur a0075e429d added further database in config.yaml, added new exception for downloading js generated html pages 1 year ago
  alpcentaur df4a8289b8 added pdf parser if entry link is direct pdf 1 year ago
  alpcentaur d3335f203b added trafilatura exception 1 year ago
  alpcentaur 14ece9bceb added functions for uniform and not uniform entry end points - non uniform endpoints are generally parsed as text from any paragraph xml element p 1 year ago
  alpcentaur b2cf4b67ce added first config parameters for search on not uniform entries 1 year ago
  alpcentaur 42841ee650 added some exceptions for bad encoding and get errors 1 year ago
  alpcentaur 317ef99720 changed code in entrylist data2dictionary to handle empty or missing xml elements 1 year ago
  alpcentaur ff23c22e3c added working bund.de-bekanntmachungen config with new example of xpath contains 1 year ago
  alpcentaur 06fa81e549 added function find config parameter and changed core spider 1 year ago
  alpcentaur a846ce04cc specifying the links, new exception clause if soupparser does not work 1 year ago
  alpcentaur c078ee4b1b first function works, actuall xml parser has still problems with certain xml types 1 year ago
  alpcentaur 8b20bc178f added multi pages configuration and code 1 year ago
  alpcentaur 7aa903883b update to config.yaml 1 year ago
  alpcentaur 59838bb8e1 added main.py importing and using the spider functions 1 year ago
  alpcentaur 5ac07d151a added first config.yaml template and started creating folder structure 1 year ago