Hi Joachim, I can use a robots.txt to allow or deny bots accessing portions of the site. But about 80% of the site has data that needs to be accessed by bots so as to not affect the search rank. Since I have enabled a CAS gateway, every request is intercepted and redirected to cas. I need to modify this so that requests from bots are not redirected to CAS.
Not sure if robots.txt is the way to go for what I'm trying to accomplish. Prasanna. On Tue, Jul 13, 2010 at 9:01 PM, Joachim Fritschi < [email protected]> wrote: > Hi, > > i guess a classic robots.txt should solve the issue for search engines. > > Cheers, > > Joachim > > Am 13.07.2010 11:18, schrieb prasanna h: > >> Hi All, >> >> We use CAS as a gateway where each request is intercepted and redirected >> to CAS to check for a ticket. I noticed that the redirect to CAS happens >> for users as well as bots. Has anyone using CAS encountered this and if >> yes, can you let me know the solution? >> >> Right now, I'm planning to check the user-agent against a list of common >> bots to determine whether to redirect to CAS. >> >> Looking forward to your thoughts as well. >> >> Prasanna >> >> -- >> You are currently subscribed [email protected] <mailto: >> [email protected]> as: [email protected] >> To unsubscribe, change settings or access archives, see >> http://www.ja-sig.org/wiki/display/JSG/cas-dev >> >> > > -- > Joachim Fritschi > Hochschulrechenzentrum (HRZ) > L1|01 Raum 248 > Petersenstr. 30 > 64287 Darmstadt > > Tel. +49 6151 16-5638 > Fax. +49 6151 16-3050 > E-Mail: [email protected] > > -- You are currently subscribed to [email protected] as: [email protected] To unsubscribe, change settings or access archives, see http://www.ja-sig.org/wiki/display/JSG/cas-dev
