What language are you writing it in? We can buy some proxy lists, or use public lists to load and use for each http request or scrape, and keep rotating them. You could batch fire off more than one request if they all go through a subsection of the proxy list, and continue down the list until it restarts. That should alleviate any spam/block concerns.
You could also run the scripts on some cheapo online webservers like on digital ocean and use their larger bandwidth capacity to do these bigger batches as I described previously.
There are options, whatever language you are writing this in, i HIGHLY recommend take the time to scrape public proxy lists, or use a list from NordVPN (if possible and if you pay) and do each single request 1 IP at a time.
Example - 10 checks use 10 different IPs and do 10 scrapes at a time, move down the list, do it again. Keep upping the batch until if shits out, then back it down. (You will have to test and see how much these hosts of the information you are querying are able to handle as well).
What language are you writing it in? We can buy some proxy lists, or use public lists to load and use for each http request or scrape, and keep rotating them. You could batch fire off more than one request if they all go through a subsection of the proxy list, and continue down the list until it restarts. That should alleviate any spam/block concerns. (litterally how some ddos attacks work).
You could also run the scripts on some cheapo online webservers like on digital ocean and use their larger bandwidth capacity to do these bigger batches as I described previously.
There are options, whatever language you are writing this in, i HIGHLY recommend take the time to scrape public proxy lists, or use a list from NordVPN (if possible and if you pay) and do each single request 1 IP at a time.
Example - 10 checks use 10 different IPs and do 10 scrapes at a time, move down the list, do it again. Keep upping the batch until if shits out, then back it down. (You will have to test and see how much these hosts of the information you are querying are able to handle as well).
EDIT: If you are writing this script in NODEJS i have a sample ddos attack that was used against my website that rotates using public proxy lists. If you need it I can try to find it for you, and you can use it as an example on where/how to pull these proxy lists before you start your scrape requests later in the script.
What language are you writing it in? We can buy some proxy lists, or use public lists to load and use for each http request or scrape, and keep rotating them. You could batch fire off more than one request if they all go through a subsection of the proxy list, and continue down the list until it restarts. That should alleviate any spam/block concerns. (litterally how some ddos attacks work).
You could also run the scripts on some cheapo online webservers like on digital ocean and use their larger bandwidth capacity to do these bigger batches as I described previously.
There are options, whatever language you are writing this in, i HIGHLY recommend take the time to scrape public proxy lists, or use a list from NordVPN (if possible and if you pay) and do each single request 1 IP at a time.
Example - 10 checks use 10 different IPs and do 10 scrapes at a time, move down the list, do it again. Keep upping the batch until if shits out, then back it down.
This is scalable if its written properly. Or the shitty public information database hosts may just shit the bed thinking they are under a DDOS attack. But do write the script to be scalable.
EDIT: If you are writing this script in NODEJS i have a sample ddos attack that was used against my website that rotates using public proxy lists. If you need it I can try to find it for you, and you can use it as an example on where/how to pull these proxy lists before you start your scrape requests later in the script.
What language are you writing it in? We can buy some proxy lists, or use public lists to load and use for each http request or scrape, and keep rotating them. You could batch fire off more than one request if they all go through a subsection of the proxy list, and continue down the list until it restarts. That should alleviate any spam/block concerns. (litterally how some ddos attacks work).
You could also run the scripts on some cheapo online webservers like on digital ocean and use their larger bandwidth capacity to do these bigger batches as I described previously.
There are options, whatever language you are writing this in, i HIGHLY recommend take the time to scrape public proxy lists, or use a list from NordVPN (if possible and if you pay) and do each single request 1 IP at a time.
Example - 10 checks use 10 different IPs and do 10 scrapes at a time, move down the list, do it again. Keep upping the batch until if shits out, then back it down.
This is scalable if its written properly.
EDIT: If you are writing this script in NODEJS i have a sample ddos attack that was used against my website that rotates using public proxy lists. If you need it I can try to find it for you, and you can use it as an example on where/how to pull these proxy lists before you start your scrape requests later in the script.
What language are you writing it in? We can buy some proxy lists, or use public lists to load and use for each http request or scrape, and keep rotating them. You could batch fire off more than one request if they all go through a subsection of the proxy list, and continue down the list until it restarts. That should alleviate any spam/block concerns. (litterally how some ddos attacks work).
You could also run the scripts on some cheapo online webservers like on digital ocean and use their larger bandwidth capacity to do these bigger batches as I described previously.
There are options, whatever language you are writing this in, i HIGHLY recommend take the time to scrape public proxy lists, or use a list from NordVPN (if possible and if you pay) and do each single request 1 IP at a time.
Example - 10 checks use 10 different IPs and do 10 scrapes at a time, move down the list, do it again. Keep upping the batch until if shits out, then back it down.
This is scalable if its written properly.