Yes Google does actually utilize directives from robots txt files and a robots path statement of no index, no follow would do it generic pw-ing would also do it
On Thu, Jul 15, 2010 at 7:52 AM, Abe Loveless <[email protected]>wrote: > does google pay attention to robots.txt files? Is there any interest in > password protecting the archive with some generic password? That would keep > out a robot. > > > > > > Dylan Haines wrote: > >> Just a though, is there a way (or a desire) to prevent Google from >> indexing >> that archive? >> >> Dylan >> >> Dylan Haines >> Systems Technical Support Manager >> >> HB Studios Multimedia Ltd. Box 725, The Hive >> 37 Hall Street >> Lunenburg, NS B0J 2C0 >> >> Tel: (902)634-8316 ext. 237 >> Cell: (902)553-0706 >> Fax: (902)634-3647 >> >> www.hb-studios.com >> >> -----Original Message----- >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Abe Loveless >> Sent: Thursday, July 15, 2010 11:43 AM >> To: Tech-Geeks Mailing List >> Subject: [tech-geeks] About the New List Manager >> >> Here's a little info about the new List Manager software. We are using >> "Mailman", a very popular list manager for *nix that has been around for >> years and years. >> >> It has a lot more options, and puts the user more in control of the >> experience. With the old system, you were either subscribed to the list >> or >> to the digest, and the only option you had was to unsubscribe if you >> wanted >> to change. >> >> The new system will let you change your options on the fly. It also has >> the >> ability to Disable Mail Sending. If you go on vacation, and don't want to >> see the messages clogging up your phone/Inbox, you can disable your e-mail >> delivery, without unsubscribing. Then, when you return just re-enable >> delivery. >> >> To Subscribe / Un-Subscribe / Edit Options: >> http://www.tech-geeks.org/cgi-bin/mailman/listinfo/tech-geeks >> >> Enter your subscribed e-mail address at the very bottom of the page, and >> click the "Unsubscribe or Edit Options" button. >> >> The List Archive is available here: >> http://www.tech-geeks.org/pipermail/tech-geeks/ >> >> The archive is searchable, and it is fast. One thing I noticed is that >> you >> have to be specific in your search criteria. >> >> A search for "fog" pulls 111 results, while a search for "fog laptop" >> pulls 19 results. >> >> Also, the system is slowly importing the historical archive from the old >> server as well. It will take awhile for the process to complete, but when >> it is done, we should have a pretty complete searchable archive (that is >> actually usable). >> >> Thanks, >> Abe >> >> >> _______________________________________________ >> Tech-geeks mailing list >> [email protected] >> http://lists.illinicloud.org/cgi-bin/mailman/listinfo/tech-geeks >> >> _______________________________________________ >> Tech-geeks mailing list >> [email protected] >> http://lists.illinicloud.org/cgi-bin/mailman/listinfo/tech-geeks >> >> >> _______________________________________________ > Tech-geeks mailing list > [email protected] > http://lists.illinicloud.org/cgi-bin/mailman/listinfo/tech-geeks >
| Subscription info at http://www.tech-geeks.org |
