It may not index a 404, but it still checks the 404. For usability's sake I'd still prefer to redirect than to send a 404. Although we were discussing bots, we have to keep the user in mind as well. I have personally traversed the URL path to see what may be found on some sites, and if Safari has the feature included out of the box, well...I'd rather present the user with something than nothing at all, and a 404 isn't my idea of proper degredation within the path. Either way, it's simply a matter of personal preference.
Google was not the first search engine to incorporate robots.txt by the way...they were the first to incorporate the rel="nofollow" and also I think the SiteMap.xml idea. On Nov 6, 12:05 pm, Mathew <[EMAIL PROTECTED]> wrote: > > I'd actually say using a permanent redirect (301, I believe) to your > > root (or that controller's index), rather than to the 404 page might > > be a better solution. If your users/visitors won't see it since > > you're not linking to it, it isn't really a bad solution, and I doubt > > you'd want any search engines indexing 404 errors in association with > > your site/domain. If it was a hacker, I don't think I'd send them a > > 404 message either, I'd just redirect them...if it was a Safari user, > > You should not redirect unless the content has been moved. Sending the > wrong response codes to incorrect URIs makes it difficult for web > crawl operators to correctly crawl your site. Should a web crawl > operator come to the conclusion that your site provides incorrect > response codes, then they might choose to crawl it aggressively since > the server's responses can not be trusted. > > Indexing bots will not index a 404 response code from the Http header. > That response code tells the bots the URI points to no content. Bots > will only index pages when the 404 error message is sent with a Http > 200 response code and a text/html content-type in the header, which is > incorrect and more of an error on the server side then a problem with > the bot. > > If you send a 301/302 response code you are telling the bot, this URI > is valid, it has been moved, now the source URI and the redirected URI > will continue to be processed by the bot. Where as if you tell the bot > 404, then the bot knows this URI is invalid, the source page that URI > comes from is generating invalid URIs, and it can drop other URIs from > that source. > > Sending a hacker a 301, 302 does nothing to change their behavior, and > provides them no extra information then a 404. > > Blocking a remote computer from making to many invalid requests from > your server does change the behavior of that remote computer. It stops > it. Which is about all you can do at this point. A hacker will return > with a different IP address, and attack. So, hackers are a completely > different topic :) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "CakePHP" group. To post to this group, send email to cake-php@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/cake-php?hl=en -~----------~----~----~----~------~----~------~--~---
