"search engine crawlers can still access your website" no they can't

I hear you and recognise the evidence, but I will counter with my own experience:

Bing is unable to read even the page basics because it is getting the ‘javascript required’ response instead. I wonder if there is something different in the setup of our respective accounts?

Ok, it works for some sites. For others it doesn’t. Again, this is how my site appears on Bing:


It seems that the aes.js is blocking Bing, and there’s nothing I can do about it.
The Google issues have been fixed, though.

Assuming this is our URL: http://jaguar.lovestoblog.com/, it has never been crawled by Bing.

image

In the screenshot you shared, select the “Bing Index” tab at the top to see the response from Bings actual crawler. What you are seeing is the result of this:


Can you try request a re-index?

5 Likes

Yes, I have requested re-indexing many times, until I realised that Bing doesn’t even get to read the pages.

Since your blog is about adult content, it is possible that the content has been filtered
or once upon a time when you told Bing that you have a website
he found something he didn’t like (it doesn’t comply with Bing guidelines) and has been ignoring it ever since.

Your best bet is to contact Bing webmaster support and ask them what the problem is

You will receive a ticket by email and there will also be an answer within 2-3 days.
Often they just report that they found no problem in Bing but the next day everything starts working as it should :smile:

8 Likes

It hasn’t changed after multiple attempts to reindex it.
One of Bing’s India tech support guys said he would look into it, but all I got was “We have investigated your query and escalated issue to our engineering team, and we will inform you once we get the response.” That was 2 weeks ago.
I don’t have an adult site.
Bing behaves the same as if I try to use the Internet Archive. It runs into the aes.js protection and gets redirected to Turn cookies on or off - Computer - Google Account Help.

1 Like

That is the exact response I got from Bing last year. As I said previously, since a very low percentage use Bing anyway, I decided it wasn’t worth pursuing. I remain convinced that the problem lies with IF’s anti-spam measures (possibly combined with patrons actually daring to make use of .htaccess for redirects, Wordpress etc?) Being an adult site is not an issue for Bing, so long as the site states as much in its meta (I read their guidelines), so that is simply an easy cop-out. Note the “Discovered but not crawled”, Bing cannot take exception to content it hasn’t even read. The crawler is denied access.

I have however raised another ticket with Bing, but I’m not holding my breath.

Probably something went wrong on Bing

because here it reports this
Screenshot 2023-06-12 090247

and see this on page 2 when searching with the address

5 Likes

You must be searching from a different location (US?), because my UK Bing results do not include anything directly from the website, just my twitter stuff that happens to include the site URI. I went down to page 6 before giving up.

Interesting to see that one of your results demonstrates the problem neatly: “this site requires Javascript to work etc”

put this in search

site:.lovestoblog.com
site:.rf.gd
site:.epizy.com

and see how many websites with that subdomain are indexed

I don’t know why Bing for the UK doesn’t give a result (again a question for Bing)
but it is certainly strange that it says “there are no results” and then shows them on page 2.

As far as javascript and cookies are concerned, I don’t know what to tell you except that Bing actually uses different versions of the crawlers and when it comes with a version that doesn’t save cookies, it takes it to the Google page about cookies, which we then see in the results…It also uses a very wide range of IPs, from Azure all the way to…Maybe some of those “newly introduced proxies” are not whitelisted on the hosting
but then again, Bing should be smarter, because, in contrast to it, everything works fine on Google.

We can discuss here indefinitely,
but we can’t generalize because it depends on the website itself, on the will of Bing, some problem long ago that Bing remembered and doesn’t want to go any further (your fault or theirs or hosting), a possible penalties/shadowban/ban, etc. For many, it works normally while some like you have a problem.

6 Likes

I do appreciate your effort, Oxy. However, I am sure you will agree that when other search engines have no problem, it is annoying that Bing is such a problem for some IF sites. I hope the mods recognise this as a real problem for ‘some’ sites and so avoid dishing out easy ‘brush off’ answers, which is what caused me to pop up in support of another frustrated member. I have no doubt that many people suffer from ‘site design’ issues which prevent them from ever getting listed, but not all. As I said before, Bing really isn’t worth getting too hung up on, so I will get back in my box.

2 Likes

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.