fdb-spider/spider.log
2024-02-29 14:19:45 +01:00

37308 lines
1.8 MiB

No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x55f9c7e81f13 <unknown>
#1 0x55f9c7b5f177 <unknown>
#2 0x55f9c7b9435e <unknown>
#3 0x55f9c7b9041a <unknown>
#4 0x55f9c7b8c65b <unknown>
#5 0x55f9c7bcd712 <unknown>
#6 0x55f9c7bc46a3 <unknown>
#7 0x55f9c7b97a85 <unknown>
#8 0x55f9c7b98ff2 <unknown>
#9 0x55f9c7e56a47 <unknown>
#10 0x55f9c7e59b7d <unknown>
#11 0x55f9c7e59628 <unknown>
#12 0x55f9c7e5a095 <unknown>
#13 0x55f9c7e48d6f <unknown>
#14 0x55f9c7e5a422 <unknown>
#15 0x55f9c7e32e76 <unknown>
#16 0x55f9c7e72c95 <unknown>
#17 0x55f9c7e72e7b <unknown>
#18 0x55f9c7e8149f <unknown>
#19 0x7ff6f68a8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x556c78da5f13 <unknown>
#1 0x556c78a83177 <unknown>
#2 0x556c78ab835e <unknown>
#3 0x556c78ab441a <unknown>
#4 0x556c78ab065b <unknown>
#5 0x556c78af1712 <unknown>
#6 0x556c78ae86a3 <unknown>
#7 0x556c78abba85 <unknown>
#8 0x556c78abcff2 <unknown>
#9 0x556c78d7aa47 <unknown>
#10 0x556c78d7db7d <unknown>
#11 0x556c78d7d628 <unknown>
#12 0x556c78d7e095 <unknown>
#13 0x556c78d6cd6f <unknown>
#14 0x556c78d7e422 <unknown>
#15 0x556c78d56e76 <unknown>
#16 0x556c78d96c95 <unknown>
#17 0x556c78d96e7b <unknown>
#18 0x556c78da549f <unknown>
#19 0x7f6ea8ea8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x55e133d3cf13 <unknown>
#1 0x55e133a1a177 <unknown>
#2 0x55e133a4f35e <unknown>
#3 0x55e133a4b41a <unknown>
#4 0x55e133a4765b <unknown>
#5 0x55e133a88712 <unknown>
#6 0x55e133a7f6a3 <unknown>
#7 0x55e133a52a85 <unknown>
#8 0x55e133a53ff2 <unknown>
#9 0x55e133d11a47 <unknown>
#10 0x55e133d14b7d <unknown>
#11 0x55e133d14628 <unknown>
#12 0x55e133d15095 <unknown>
#13 0x55e133d03d6f <unknown>
#14 0x55e133d15422 <unknown>
#15 0x55e133cede76 <unknown>
#16 0x55e133d2dc95 <unknown>
#17 0x55e133d2de7b <unknown>
#18 0x55e133d3c49f <unknown>
#19 0x7f033e0a8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x55dde0a45f13 <unknown>
#1 0x55dde0723177 <unknown>
#2 0x55dde075835e <unknown>
#3 0x55dde075441a <unknown>
#4 0x55dde075065b <unknown>
#5 0x55dde0791712 <unknown>
#6 0x55dde07886a3 <unknown>
#7 0x55dde075ba85 <unknown>
#8 0x55dde075cff2 <unknown>
#9 0x55dde0a1aa47 <unknown>
#10 0x55dde0a1db7d <unknown>
#11 0x55dde0a1d628 <unknown>
#12 0x55dde0a1e095 <unknown>
#13 0x55dde0a0cd6f <unknown>
#14 0x55dde0a1e422 <unknown>
#15 0x55dde09f6e76 <unknown>
#16 0x55dde0a36c95 <unknown>
#17 0x55dde0a36e7b <unknown>
#18 0x55dde0a4549f <unknown>
#19 0x7f9eacea8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x5562c5cf6f13 <unknown>
#1 0x5562c59d4177 <unknown>
#2 0x5562c5a0935e <unknown>
#3 0x5562c5a0541a <unknown>
#4 0x5562c5a0165b <unknown>
#5 0x5562c5a42712 <unknown>
#6 0x5562c5a396a3 <unknown>
#7 0x5562c5a0ca85 <unknown>
#8 0x5562c5a0dff2 <unknown>
#9 0x5562c5ccba47 <unknown>
#10 0x5562c5cceb7d <unknown>
#11 0x5562c5cce628 <unknown>
#12 0x5562c5ccf095 <unknown>
#13 0x5562c5cbdd6f <unknown>
#14 0x5562c5ccf422 <unknown>
#15 0x5562c5ca7e76 <unknown>
#16 0x5562c5ce7c95 <unknown>
#17 0x5562c5ce7e7b <unknown>
#18 0x5562c5cf649f <unknown>
#19 0x7fbc918a8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x561194db3f13 <unknown>
#1 0x561194a91177 <unknown>
#2 0x561194ac635e <unknown>
#3 0x561194ac241a <unknown>
#4 0x561194abe65b <unknown>
#5 0x561194aff712 <unknown>
#6 0x561194af66a3 <unknown>
#7 0x561194ac9a85 <unknown>
#8 0x561194acaff2 <unknown>
#9 0x561194d88a47 <unknown>
#10 0x561194d8bb7d <unknown>
#11 0x561194d8b628 <unknown>
#12 0x561194d8c095 <unknown>
#13 0x561194d7ad6f <unknown>
#14 0x561194d8c422 <unknown>
#15 0x561194d64e76 <unknown>
#16 0x561194da4c95 <unknown>
#17 0x561194da4e7b <unknown>
#18 0x561194db349f <unknown>
#19 0x7fb3db6a8044 <unknown>
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
trying to get element
1
scrolling..
clicking..
length of the window handles 2
trying to get element
2
scrolling..
clicking..
length of the window handles 2
trying to get element
3
scrolling..
clicking..
length of the window handles 2
trying to get element
4
scrolling..
clicking..
length of the window handles 2
trying to get element
5
scrolling..
clicking..
length of the window handles 2
trying to get element
6
scrolling..
clicking..
length of the window handles 2
trying to get element
7
scrolling..
clicking..
length of the window handles 2
trying to get element
8
scrolling..
clicking..
length of the window handles 2
trying to get element
9
scrolling..
clicking..
length of the window handles 2
trying to get element
10
scrolling..
clicking..
length of the window handles 2
trying to get element
11
scrolling..
clicking..
length of the window handles 2
trying to get element
12
scrolling..
clicking..
length of the window handles 2
trying to get element
13
scrolling..
clicking..
length of the window handles 2
trying to get element
14
scrolling..
clicking..
length of the window handles 2
trying to get element
15
scrolling..
clicking..
length of the window handles 2
trying to get element
16
scrolling..
clicking..
length of the window handles 2
trying to get element
17
scrolling..
clicking..
length of the window handles 2
trying to get element
18
scrolling..
clicking..
length of the window handles 2
trying to get element
19
scrolling..
clicking..
length of the window handles 2
trying to get element
20
scrolling..
clicking..
length of the window handles 2
trying to get element
21
scrolling..
clicking..
length of the window handles 2
trying to get element
22
scrolling..
clicking..
length of the window handles 2
trying to get element
23
scrolling..
clicking..
length of the window handles 2
trying to get element
24
scrolling..
clicking..
length of the window handles 2
trying to get element
25
scrolling..
clicking..
length of the window handles 2
trying to get element
26
scrolling..
clicking..
length of the window handles 2
trying to get element
27
scrolling..
clicking..
length of the window handles 2
trying to get element
28
scrolling..
clicking..
length of the window handles 2
trying to get element
29
scrolling..
clicking..
length of the window handles 2
trying to get element
30
scrolling..
clicking..
length of the window handles 2
trying to get element
31
scrolling..
clicking..
length of the window handles 2
trying to get element
32
scrolling..
clicking..
length of the window handles 2
trying to get element
33
scrolling..
clicking..
length of the window handles 2
trying to get element
34
scrolling..
clicking..
length of the window handles 2
trying to get element
35
scrolling..
clicking..
length of the window handles 2
trying to get element
36
scrolling..
clicking..
length of the window handles 2
trying to get element
37
scrolling..
clicking..
length of the window handles 2
trying to get element
38
scrolling..
clicking..
length of the window handles 2
trying to get element
39
scrolling..
clicking..
length of the window handles 2
trying to get element
40
scrolling..
clicking..
length of the window handles 2
trying to get element
41
scrolling..
clicking..
length of the window handles 2
trying to get element
42
scrolling..
clicking..
length of the window handles 2
trying to get element
43
scrolling..
clicking..
length of the window handles 2
trying to get element
44
scrolling..
clicking..
length of the window handles 2
trying to get element
45
scrolling..
clicking..
length of the window handles 2
trying to get element
46
scrolling..
clicking..
length of the window handles 2
trying to get element
47
scrolling..
clicking..
length of the window handles 2
trying to get element
48
scrolling..
clicking..
length of the window handles 2
trying to get element
49
scrolling..
clicking..
length of the window handles 2
trying to get element
50
scrolling..
clicking..
length of the window handles 2
trying to get element
51
scrolling..
clicking..
length of the window handles 2
trying to get element
52
scrolling..
clicking..
length of the window handles 2
trying to get element
53
scrolling..
clicking..
length of the window handles 2
trying to get element
54
scrolling..
clicking..
length of the window handles 2
trying to get element
55
scrolling..
clicking..
length of the window handles 2
trying to get element
56
scrolling..
clicking..
length of the window handles 2
trying to get element
57
scrolling..
clicking..
length of the window handles 2
trying to get element
58
scrolling..
clicking..
length of the window handles 2
trying to get element
59
scrolling..
clicking..
length of the window handles 2
trying to get element
60
scrolling..
clicking..
length of the window handles 2
trying to get element
61
scrolling..
clicking..
length of the window handles 2
trying to get element
62
scrolling..
clicking..
length of the window handles 2
trying to get element
63
scrolling..
clicking..
length of the window handles 2
trying to get element
64
scrolling..
clicking..
length of the window handles 2
trying to get element
65
scrolling..
clicking..
length of the window handles 2
trying to get element
66
scrolling..
clicking..
length of the window handles 2
trying to get element
67
scrolling..
clicking..
length of the window handles 2
trying to get element
68
scrolling..
clicking..
length of the window handles 2
trying to get element
69
scrolling..
clicking..
length of the window handles 2
trying to get element
70
scrolling..
clicking..
length of the window handles 2
trying to get element
71
scrolling..
clicking..
length of the window handles 2
trying to get element
72
scrolling..
clicking..
length of the window handles 2
trying to get element
73
scrolling..
clicking..
length of the window handles 2
trying to get element
74
scrolling..
clicking..
length of the window handles 2
trying to get element
75
scrolling..
clicking..
length of the window handles 2
trying to get element
76
scrolling..
clicking..
length of the window handles 2
trying to get element
77
scrolling..
clicking..
length of the window handles 2
trying to get element
78
scrolling..
clicking..
length of the window handles 2
trying to get element
79
scrolling..
clicking..
length of the window handles 2
trying to get element
80
scrolling..
clicking..
length of the window handles 2
trying to get element
81
scrolling..
clicking..
length of the window handles 2
trying to get element
82
scrolling..
clicking..
length of the window handles 2
trying to get element
83
scrolling..
clicking..
length of the window handles 2
trying to get element
84
scrolling..
clicking..
length of the window handles 2
trying to get element
85
scrolling..
clicking..
length of the window handles 2
trying to get element
86
scrolling..
clicking..
length of the window handles 2
trying to get element
87
scrolling..
clicking..
length of the window handles 2
trying to get element
88
scrolling..
clicking..
length of the window handles 2
trying to get element
89
scrolling..
clicking..
length of the window handles 2
trying to get element
90
scrolling..
clicking..
length of the window handles 2
trying to get element
91
scrolling..
clicking..
length of the window handles 2
trying to get element
92
scrolling..
clicking..
length of the window handles 2
trying to get element
93
scrolling..
clicking..
length of the window handles 2
trying to get element
94
scrolling..
clicking..
length of the window handles 2
trying to get element
95
scrolling..
clicking..
length of the window handles 2
trying to get element
96
scrolling..
clicking..
length of the window handles 2
trying to get element
97
scrolling..
clicking..
length of the window handles 2
trying to get element
98
scrolling..
clicking..
length of the window handles 2
trying to get element
99
scrolling..
clicking..
length of the window handles 2
trying to get element
100
scrolling..
clicking..
length of the window handles 2
trying to get element
101
scrolling..
clicking..
length of the window handles 2
trying to get element
102
scrolling..
clicking..
length of the window handles 2
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
No jsiteration-var-list defined in config.yaml - the original error message is: eval() arg 1 must be a string, bytes or code object
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
decoding the respone in utf8 did not work, try to decode latin1 now - the original error message is: 'utf-8' codec can't decode byte 0xe4 in position 5596: invalid continuation byte
getting the html page through urllib did not work, trying with requests librarys function get
Traceback (most recent call last):
File "/root/fdb-spider/main.py", line 16, in <module>
spider.download_entry_list_pages_of_funding_databases(list_of_fdbs)
File "/root/fdb-spider/spiders/fdb_spider.py", line 166, in download_entry_list_pages_of_funding_databases
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 56, in __init__
super().__init__(
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "/root/fdb-spider/venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: disconnected: unable to connect to renderer
Stacktrace:
#0 0x563577440f13 <unknown>
#1 0x56357711e177 <unknown>
#2 0x56357715335e <unknown>
#3 0x56357714f41a <unknown>
#4 0x56357714b65b <unknown>
#5 0x56357718c712 <unknown>
#6 0x5635771836a3 <unknown>
#7 0x563577156a85 <unknown>
#8 0x563577157ff2 <unknown>
#9 0x563577415a47 <unknown>
#10 0x563577418b7d <unknown>
#11 0x563577418628 <unknown>
#12 0x563577419095 <unknown>
#13 0x563577407d6f <unknown>
#14 0x563577419422 <unknown>
#15 0x5635773f1e76 <unknown>
#16 0x563577431c95 <unknown>
#17 0x563577431e7b <unknown>
#18 0x56357744049f <unknown>