Re: Sloppy - Yahoo! Slurp throwing CFerrors
If you have an error handler, then at the top of it test http_user_agent and, if it turns out to be Slurp, feed them something that makes them go away, like as Dave suggested a 200 error rather than showing them whatever your friendly error is. I've got the same problem and think I'll try this as soon as possible. Googlebot drives a client of mine nuts and this may be his ticket as well. -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196130 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: Sloppy - Yahoo! Slurp throwing CFerrors
I'm sorry if I've missed something in this thread. I have had the same problem before with a few different spiders. I like it happening though because it allows me to find something wrong that would get missed if it is rarely or ever browsed by an actual user. When an error would be generated I would go hunt it down and correct it. On problems like a URL change or URL var change or something no longer existing, I just had it catch the error and return nothing. However I like the idea of returning an http status error. That way it may make it update their index. -Original Message- From: Matt Robertson [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 23, 2005 12:03 PM To: CF-Talk Subject: Re: Sloppy - Yahoo! Slurp throwing CFerrors If you have an error handler, then at the top of it test http_user_agent and, if it turns out to be Slurp, feed them something that makes them go away, like as Dave suggested a 200 error rather than showing them whatever your friendly error is. I've got the same problem and think I'll try this as soon as possible. Googlebot drives a client of mine nuts and this may be his ticket as well. ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196133 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
What Mark is talking about is Slurp putting in its own url vars, if I read him correctly. I've seen the same thing. When I first saw it I thought it was someone hacking. Heck, it still might be. Amen to using errors to detect site flaws. Reminds you of stuff like the need to handle, for example, a form submission page that gets hit with a GET instead of a POST. I personally no longer email errors. They get stored to disk on the server. Don't like cfdumping entire scopes like I do and then emailing them in the clear. -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196138 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: Sloppy - Yahoo! Slurp throwing CFerrors
Strange, you're right about emailing the dump. I like the email though. I guess when you write the full info to disk you can email yourself that an error occurred and the basic message with a link to your log or something. I still email the dump, but I think I may just do what you mentioned. I like it emailing though so I can get notified on my BlackBerry when there is an error. Warning to anyone that wants to try emailing your Cell/PDA: Don't do it if your server generates many errors. I've had a few times when someone tried to take down a server by looping over a URL with invalid numerical values for an ID var. They put it in a loop for if I remember correctly a thousand iterations at 3AM! (Case in point to code for this possibility in your applications) -Original Message- From: Matt Robertson [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 23, 2005 12:41 PM To: CF-Talk Subject: Re: Sloppy - Yahoo! Slurp throwing CFerrors What Mark is talking about is Slurp putting in its own url vars, if I read him correctly. I've seen the same thing. When I first saw it I thought it was someone hacking. Heck, it still might be. Amen to using errors to detect site flaws. Reminds you of stuff like the need to handle, for example, a form submission page that gets hit with a GET instead of a POST. I personally no longer email errors. They get stored to disk on the server. Don't like cfdumping entire scopes like I do and then emailing them in the clear. -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196140 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
It strikes me that you shouldn't be allowing those munged or extraneous URL variable values to cause CF to throw errors. For example, I'd never use something like the following cfparam name=url.detailedview default=no type=boolean Instead, I'd use cfparam name=url.showdetails default=no cfif not IsBoolean(url.showdetails) cfset url.showdetails = no /cfif You might accomplish something similar by wrapping the first in a cftry/cfcatch, but this is so much simpler, why bother? If passing a numeric value directly from the url scope into a query, do something like cfqueryparam cfsqltype=CF_SQL_NUMERIC value=#Val(url.view)# Your query may come up empty, but at least you're not throwing errors because someone (or something, in your case) decided to get creative with the URL. - Original Message - From: Mark W. Breneman [EMAIL PROTECTED] To: CF-Talk cf-talk@houseoffusion.com Sent: Monday, February 21, 2005 10:21 AM Subject: Sloppy - Yahoo! Slurp throwing CFerrors Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com http://www.vividmedia.com/ 608.270.9770 ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196142 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
On Wed, 23 Feb 2005 12:53:01 -0500, Bryan F. Hogan [EMAIL PROTECTED] wrote: I guess when you write the full info to disk you can email yourself that an error occurred and the basic message with a link to your log or something. Bingo. I write the files to .html and protect the site they go to with Windows permissions. Then I view the errors over ssl. You could save the files as .cfm's and then use CF to handle the security. I wrote up the whole process, with all the code, at my site in the tutorial section. -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196146 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
On Wed, 23 Feb 2005 10:59:01 -0700, Jim McAtee [EMAIL PROTECTED] wrote: cfqueryparam cfsqltype=CF_SQL_NUMERIC value=#Val(url.view)# Holy cow thats simple. Always thought of cfqueryparam as a replacement for val() in raw sql. (sound of head bonking repeatedly on wall) -- --mattRobertson-- Janitor, MSB Web Systems mysecretbase.com ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:196150 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
If you know that your id must be numeric as in your example, then why not clean out any non numeric chars before you do anything. That should remove any extraneous crap added by slurp. cfset url.id = rereplace([^0-9],url.id,,all) Or something like that. On Mon, 21 Feb 2005 14:31:50 -0500, Dave Watts [EMAIL PROTECTED] wrote: Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? robots.txt? Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195906 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
If you are running Apache, you could use mod_rewrite to change the incoming bad URLs into good URLs using regular expressions. I don't think IIS has quite as flexible a tool, but it does have a re-director that you could use per directory or per URL to forward bad URLs to good URLs. FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195998 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
You could also do the custom 404/error page thing and create a per-URL mapping to good URLS. If you do that, use header tags to forward to the good URLS with a 301 Moved Permanently status code. That will tell the bot to stop using the old URL and use the new one. FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195999 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Sloppy - Yahoo! Slurp throwing CFerrors
Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com http://www.vividmedia.com/ 608.270.9770 ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195758 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
Generally by thworing 404 headers would do it I would think that tells it that the page that it is asking for does not exist On Mon, 21 Feb 2005 11:21:42 -0600, Mark W. Breneman [EMAIL PROTECTED] wrote: Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com http://www.vividmedia.com/ 608.270.9770 ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195761 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
Re: Sloppy - Yahoo! Slurp throwing CFerrors
see below... Bryan Stevenson B.Comm. VP Director of E-Commerce Development Electric Edge Systems Group Inc. phone: 250.480.0642 fax: 250.480.1264 cell: 250.920.8830 e-mail: [EMAIL PROTECTED] web: www.electricedgesystems.com - Original Message - From: Mark W. Breneman [EMAIL PROTECTED] To: CF-Talk cf-talk@houseoffusion.com Sent: Monday, February 21, 2005 9:21 AM Subject: Sloppy - Yahoo! Slurp throwing CFerrors Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 OK this should be fixable by making your URLs spider friendly... www.yourdomain.com/index.cfm?varA=5 changes to www.yourdomain.com/index.cfm/varA/5 I can send ya the details on making this work or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? Yes you can use a robots.txt file to explicitly tell bots where they can and cannot go. I beleive you can also use an .HTAccess file to stop bots from indexing certain pages (or tells them the page is gonebit fuzzy on this one). HTH Cheers FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com http://www.vividmedia.com/ 608.270.9770 ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195763 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: Sloppy - Yahoo! Slurp throwing CFerrors
I don't think using 404 headers would work, since it is not a missing page, but a bad URL parameter. Using 500 server error header would be much closer to what I want, I think. How would one go about sending a 500 server error header at the top of a custom error page? I assume it is more than just putting 500 server error in the title tag. Thanks! Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com 608.270.9770 -Original Message- From: Mark Drew [mailto:[EMAIL PROTECTED] Sent: Monday, February 21, 2005 11:36 AM To: CF-Talk Subject: Re: Sloppy - Yahoo! Slurp throwing CFerrors Generally by thworing 404 headers would do it I would think that tells it that the page that it is asking for does not exist On Mon, 21 Feb 2005 11:21:42 -0600, Mark W. Breneman [EMAIL PROTECTED] wrote: Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? FYI I have a custom cf error page. After thinking about this for a little while I can see where my custom cf error page may not look like an error page to Slurp. Any idea of what I can do to make Slurp understand it is an error? Thanks Mark W. Breneman -Cold Fusion Developer -Network Administrator Vivid Media [EMAIL PROTECTED] www.vividmedia.com http://www.vividmedia.com/ 608.270.9770 ~| Logware (www.logware.us): a new and convenient web-based time tracking application. Start tracking and documenting hours spent on a project or with a client with Logware today. Try it for free with a 15 day trial account. http://www.houseoffusion.com/banners/view.cfm?bannerid=67 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195768 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: Sloppy - Yahoo! Slurp throwing CFerrors
How would one go about sending a 500 server error header at the top of a custom error page? I assume it is more than just putting 500 server error in the title tag. Actually, if you have CFMX configured to use HTTP status codes, any uncaught errors will return an HTTP 500 status code by default. You can customize your error message using CFTHROW if you like. Keep in mind, however, that if you do this, the browser may suppress the actual error message - in IE 6, there's a setting called Show friendly HTTP error messages or something like that. So, rather than return an HTTP status code indicating the error, most people just generate a regular HTTP 200 status code with accompanying HTML to indicate that things didn't work out as desired. Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~| Find out how CFTicket can increase your company's customer support efficiency by 100% http://www.houseoffusion.com/banners/view.cfm?bannerid=49 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195778 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=89.70.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
RE: Sloppy - Yahoo! Slurp throwing CFerrors
Does anyone else see CFerrors from Yahoo's spider / bot Slurp? We monitor all CFerrors that are generated on our servers. It seems that Yahoo! Slurp is responsible for over 60% of the errors per week that are thrown by our production webservers. It looks like Slurp is appending almost random values to the query strings. id=5%E2%84%91=2 or it may append an old URL var that has not been used for years like view=archive. (This one throws a CF_SQL_NUMERIC error now.) Does anyone know of a way I can get Slurp to stop hitting these pages with invalid URL vars? And still keep Slurp indexing the site normaly? robots.txt? Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ Fig Leaf Software provides the highest caliber vendor-authorized instruction at our training centers in Washington DC, Atlanta, Chicago, Baltimore, Northern Virginia, or on-site at your location. Visit http://training.figleaf.com/ for more information! ~| Discover CFTicket - The leading ColdFusion Help Desk and Trouble Ticket application http://www.houseoffusion.com/banners/view.cfm?bannerid=48 Message: http://www.houseoffusion.com/lists.cfm/link=i:4:195779 Archives: http://www.houseoffusion.com/cf_lists/threads.cfm/4 Subscription: http://www.houseoffusion.com/lists.cfm/link=s:4 Unsubscribe: http://www.houseoffusion.com/cf_lists/unsubscribe.cfm?user=11502.10531.4 Donations Support: http://www.houseoffusion.com/tiny.cfm/54
