correct https://www.google.com/adsense/list-auth Use this section to allow the AdSense crawler to access pages that are behind a login. Our crawler will access these pages only to determine content for ad targeting purposes and will fully comply with Google's privacy policy <http://www.google.com/privacy.html>
While they are determining content for ad targeting, they are looking at the pages content. This content will not show up in their index but logic says that it will be used to effect your ranking. I've seen search results that show a site and when I click on it, I get the logic. This says that Google has indexed past the login. On 10/12/08, Adrian Lynch <[EMAIL PROTECTED]> wrote: > > And not include what it finds in its index? > > Adrian > > > -----Original Message----- > From: Michael Dinowitz > Sent: 12 October 2008 22:51 > To: cf-talk > Subject: Re: How does Security affect search engine spiders? > > > There is an option in adsense to bypass login based security in order to > index pages for ads. While your pages may not have ads on them, using this > option guarantees that Google will get through your security. > > On 10/12/08, Doug Boude (rhymes with 'loud') <[EMAIL PROTECTED]> wrote: > > > > Hi all. I am curious if anybody knows how securing a site affects a > search > > engine spider's ability to crawl it. For instance, if I have my entire > site > > secured by means of authentication so that any page request is redirected > to > > the login page if the appropriate security creds are not present in > session, > > do spiders receive the same treatment? Are they also prohibited by my > > security from crawling any page except the login page? If this is true, > what > > can I do to allow spiders to have access to crawl content but still apply > > security to regular "human" visitors? My only thought on that is to > detect > > the fact that they are a spider (not sure how to do that though) and not > > implement security in that case. > > > > Thanks for your ideas and thoughts. Feel free to email them to me at > > [EMAIL PROTECTED] > > > > Doug :0) > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Adobe® ColdFusion® 8 software 8 is the most important and dramatic release to date Get the Free Trial http://ad.doubleclick.net/clk;207172674;29440083;f Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:313794 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4
