I read the ifastnet tos and theres nothing about a form related to web crawlers.
Hi,
I, personally feel that web spiders are okay. If your site is having anything against the ToS, I suggest you not to use them.
So, I think that they are allowed, since most people use it…
Are you asking a question related to doing web crawling?
If so, then yes it’s perfectly fine.
However, infinityfree has limits that may interfere with the efficiency of your program (like CPU usage, web crawlers usually take up a lot of resources like CPU).
3 Likes
However, infinityfree has limits that may interfere with the efficiency of your program (like CPU usage, web crawlers usually take up a lot of resources like CPU).
Yes. Here’s a tip for you (@duckyishot93)
Set your crawling speed to a slower rate.
1 Like
What do you mean “a slower rate”?
As in crawl over a more even rate?
That’s actually a good idea.
1 Like
The term crawl rate means how many requests per second Googlebot makes to your site when it is crawling it: for example, 5 requests per second.
To quickly reduce the crawl rate, you can change the Googlebot crawl rate in Search Console. Changes made to this setting are generally reflected within days. To use this setting, first verify your site ownership. Make sure that you avoid setting the crawl rate to a value that’s too low for your site’s needs.
Oh ok.
I didn’t know that!
2 Likes
Hi and welcome to the forum
If you wanted to generate a sitemap with some online tool ?
Online tools are blocked by the server so they are unable to view all the content of your site and create a corresponding sitemaps !
(Google is allowed - major popular crawlers are allowed)
The solution is to use some offline Sitemap generator and instruct it to review the local content of the folder in which is your website. (or simply write it yourself)
Wordpress plugin = https://yoast.com/wordpress/plugins/seo/
If you wanted to make a BOT here
then I think it is a violation of TOS 10. (Website scraping/crawling/downloading scripts)
4 Likes
Now why would infinityfree ban the making of bots here?
Aside from the obvious: misuse of bots
1 Like
This here is the answer. It does say something about web crawlers and it specifically says it’s not allowed.
We provide a website hosting service, not a data harvesting platform. There are fair usage restrictions in place to help enforce this.
4 Likes
This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.