phpwcms Support Forum

Hello,

I am hoping someone here can help me.

It has come to my attention that some crawlers get a 400 bad request when trying to crawl my website http://www.ceint.com/

Example:
http://whois.domaintools.com/ceint.com

The site is valid html 4.01 and I've spent hours trying to figure out why some search engine crawlers like google get the 400 bad request error when spidering the site.

I am hoping you can help me with this. google has dropped the site from the search results and I believe this is the reason.

Is it because off a missing bot entry here?
$phpwcms['BOTS'] = array('googlebot', 'msnbot', 'ia_archiver', 'altavista', 'slurp', 'yahoo', 'jeeves', 'teoma', 'lycos', 'crawler');

I contacted my web hosting provider regrding this and got the following response.

--
The error is coming out of your php scripts. Looks like a common issue with those trying to stop spam bots. You will need to check with your developer (or the developer of the web application you are using) and get a fix for this.
--

I noticed the same problem while browsing with opera.

Do you still get the error in Opera?

Yes, i noticed it only in OPERA.

The bot entries have nothing to do with it.

That Google is kicking your site off the search index is nothing you should wonder about. So many words in relation with spamming: p o ker, f r ee, ca sino, bet - oh man....

Are you really sure Google gets error 400 too? The only place where you can see it is the access log.

You can replace "include/inc_module/mod_bad_behavior/bad-behavior-phpwcms.php" by an empty file then no spam bot blocking is processed.

I can open your page in Opera.

Oliver

@oliver - das war bei mir auch zwischendurch und willkürlich... keine ahnung hab mich noch nicht näher mit außeinandergesetzt.

OK,

I have implemented an additional new setting.

Oliver