5 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
puckoprutt	b0a8633432	set concurrent request per domain to 8	2025-02-18 18:54:08 +01:00
puckoprutt	d9e7d6dc01	dont include scraped files	2025-02-18 18:39:09 +01:00
puckoprutt	4d4e03d933	run all spiders in diffrent threads	2025-02-18 18:36:49 +01:00
puckoprutt	4c877e346f	added a max spider	2025-02-18 18:33:35 +01:00
puckoprutt	25541bf06f	first version	2025-02-17 21:33:36 +01:00