How to protect your website that is on Cloudflare plus Logs

You did not specify how you know google is blocked
maybe robots.txt is to blame

or you do not have a valid TLS certificate
or something else in the CF settings

You can check here https://search.google.com/test/rich-results

and then look at the log…you should see google bot there

2 Likes

its getting blocked in cf

I made an update of the article here
if it helps you and others

The only thing I saw in your case was that a simple visit from Google to the domain was already blocked and it seems to me that you have some protection plugin (soft) which is probably the culprit for the block,
and not some trigger you mentioned in your post.

1 Like

Well done and thanks for sharing.

3 Likes

Thank you so much for this useful information.

3 Likes

i wanted to try this so i installed the app but its way too complicated, i cant even find the edit button to generate a link :frowning:

1 Like

++ the cloudflare interface has changed .



  1. click your “source”


  1. find EDIT on top right
4 Likes

Here’s how to prevent this: Block ALL bots (expect trusted bots)!

This way you won’t have an insane 20GB of traffic (maybe more, maybe less)

3 Likes

thanks everyone :smiley:

2 Likes

I forgot to mention one important action
so I added it to the article today, and a few more sentences around.

2 Likes

Yes, normalization is important.

Heck, I should normalize URLs before I handle them all the time (I have always had this problem).

Sometimes, redundancy really screws up our security XD

2 Likes

I use a firewall rule that logs every request rather than use Logflare as IMO it’s easier to read

Are you talking about Cloudflare WAF?
As far as I know, you need to be an enterprise user to access logs (log push , instant logs, WAF payload, etc.).

If the rule hasn’t been triggered, then there won’t be any logs in CF FW.
Or do you challenge everything? so to some extent, you get some form of log?

Logflare records any request
and not just something that triggered FW activity (e.q. only block, allow, challenge)

In order for something to trigger FW activity there must be some rule in FW,
but how to write a rule if you don’t know what someone is looking for (vuln.scan) on your website? or trying something else not specified in your FW rule?
that’s why it makes sense to use Logflare.

I don’t know how it’s clearer to you
when the list of logs in CF WAF is divided into pages
page1-555+ and you have to click the mouse many times for a particular page
and it is difficult to take a step back …
and waits relatively long to load especially when you have more than 100 pages,
and sometimes a single page contains only two or three events instead of a full space.




If you are talking about an application level firewall plugin for WP and the like
it doesn’t make any sense to use it
because it only protects you in the sense that it is harder for someone to hack your WP because some attempts were rejected by FW,

but all requests still arrive on the server (this hosting) if you do not use DNS Level Firewall (Cloudflare and the like) and every bad requests will still increase the load on your server. Because it’s an application-level firewall which means in terms of DDoS or a large number of scans you lose 50,000 daily hits and that is not the meaning of protection.

3 Likes

I have a FW rule that if it’s not my ip, bypass hotlink protection. I don’t use hotlink protection so it doesn’t really bypass anything. However, this still logs the request. I find it easier to read because I can set it to see the IP, Action, and Path. If I scroll past a path that most definitely isn’t a real path, I can easily copy and paste it into a FW rule and block that path to prevent it from going to the server.

4 Likes

Or you can do what I do and log everything on the origin then check the logs and anything that is not a true path block. But both work just as well :slight_smile:

2 Likes

Thanks for explaining

I guess then you only see GET requests and not POST, HEAD, etc?


I put it to my test (hotlink should be disabled on CF)

so we’ll see :smile:

P.S. you need to wait a “X” time for CF to refresh the list (not in real time) so a few refresh buttons need to be pressed - to make sure it works




I guess this would work too

1 Like

Yeah, that’s basically how my rule looks like. Also, I looked through my logs and I can see GET, POST, and HEAD requests.

3 Likes

:red_circle: ignore this post - see the reason in the post below




In addition to everything mentioned at the beginning of the topic
it is preferable to add permission for only CF servers to access the origin


.htaccess (localhost was added just for testing on the local PC - if you use web server stack package)

# Only Localhost and Cloudflare 
Require ip 127.0.0.1
Require ip 103.21.244.0/22
Require ip 103.22.200.0/22
Require ip 103.31.4.0/22
Require ip 104.16.0.0/13
Require ip 104.24.0.0/14
Require ip 108.162.192.0/18
Require ip 131.0.72.0/22
Require ip 141.101.64.0/18
Require ip 162.158.0.0/15
Require ip 172.64.0.0/13
Require ip 173.245.48.0/20
Require ip 188.114.96.0/20
Require ip 190.93.240.0/20
Require ip 197.234.240.0/22
Require ip 198.41.128.0/17
Require ip 2400:cb00::/32
Require ip 2606:4700::/32
Require ip 2803:f800::/32
Require ip 2405:b500::/32
Require ip 2405:8100::/32
Require ip 2a06:98c0::/29
Require ip 2c0f:f248::/32

The list is updated by visiting this LINK
and check if everything is the same as in your .htaccess and correct it if necessary.

6 Likes

The post above about .htaccess code and Cloudflare IPs, which was part of the article
is no longer possible to apply here on hosting.

The admin and I discovered “something” that led to recent changes on the servers.

Because of those changes, now there is no need to retrieve the CF-Connecting-IP (visitor IP) through a PHP script,
now the server does it by itself !

But for that reason, the specified .htaccess code can’t see anymore the Cloudflare server’s IPs - it sees the visitor IP
and if it compares it with the existing IP list → of course it will return 403.

I deleted that part from the article and an alternative solution was added
and I hereby thank the admin for that.

7 Likes