You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

27 lines
658 B

11 months ago
11 months ago
11 months ago
  1. ```
  2. __ _ _ _ _
  3. / _| __| | |__ ___ _ __ (_) __| | ___ _ __
  4. | |_ / _` | '_ \ _____/ __| '_ \| |/ _` |/ _ | '__|
  5. | _| (_| | |_) |_____\__ | |_) | | (_| | __| |
  6. |_| \__,_|_.__/ |___| .__/|_|\__,_|\___|_|
  7. |_|
  8. ```
  9. Configure fdb-spider in a yaml file.
  10. Spider Multi page databases of links.
  11. Filter and serialize content to json.
  12. Filter either by xpath syntax.
  13. Or Filter with the help of Artificial Neural Networks (work in progress).
  14. To run this, create a python3 virtualenv, pip install -r requirements,
  15. and
  16. ```
  17. apt install xvfb
  18. apt install chromium
  19. apt install chromium-webdriver
  20. ```