So I tried to use the Site Audit feature on Ahrefs but it keeps showing this error every single day.
That probably happens because of this:
3 Likes
Probably due to this
1 Like
Oh man 
Is there no way to bypass this or an alternative? I mean Ahrefs crawlers are good bots.
No, you cannot bypass this security measure, you can, however, use an alternate service.
1 Like
Use cloud flare.
I am using cloudflare (and not with the broken cpanel integration, I’ve changed to cloudflare’s nameservers). I still get the crawl failed message.
Huh, idk then…
Check your firewall logs in cloudflare
That was the first thing I did. But the log was empty. I even paused Cloudflare just to be sure but it was still failing. That’s how I came to the conclusion that this was a hosting problem.
How would Cloudflare allow the bots? Cloudflare is just a DNS and protection service, all the traffic still goes to Infinity where it is than blocked by the security protocol.
@Citrus_Titan, I believe that premium hosting allows this, but you would have to ask them before upgrading if you would like to do so. There is no way to bypass this measure on the free service. And since most other free hosts are based on this service, it is unlikely you will find a free host that allows this.
Thanks
Not quite. When using Cloudflare, our browser validation system is disabled under the assumption that Cloudflare’s security settings block all bad traffic. Those same security settings could be blocking that crawler.
1 Like
Oh, did not know that, thanks for the info and apologies to @Deveroonie (Although maybe you could give more info in your answer so I don’t make that mistake next time)
1 Like
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.