RE: How to recognize robots

2005-11-01 Thread Snake
Sorry, couldn't resist... -Original Message- From: Matt Robertson [mailto:[EMAIL PROTECTED] Sent: 31 October 2005 22:28 To: CF-Talk Subject: Re: How to recognize robots On 10/31/05, Snake <[EMAIL PROTECTED]> wrote: > This might help > > http://www.the-robotman.com/

Re: How to recognize robots

2005-11-01 Thread Matt Robertson
Real nice, Claude. I spent a fair amount of time playing with that htmldog script yesterday and I have it to the point where its decent-looking in Firefox, but not quite perfect. Had to play with it to get If you can make that thing into a commercial product, come up with developer pricing so I

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>An updated, even lighter-weight version of those menus is outlined here: http://www.htmldog.com/articles/suckerfish/dropdowns/ I remember I had a look at this one. What I didn't like is that you need to define CSS stuff for each level in the menu, which can get quite complicated. I solved the

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>Do you have a particular favorite in mind for this sort of menu? Sure: my own mine! ;-) I'm using a first version here: www.fafo.on.ca I may release a commercial version for it, the only problem now is that I need to make some tools to edit the CSS file and the menu items in the database, and

Re: How to recognize robots

2005-10-31 Thread Bryan Stevenson
they are made of metal and are full of wires and gyros and stuff (sorry...been kiling me all day not to send that...carry on) ;-) Bryan Stevenson B.Comm. VP & Director of E-Commerce Development Electric Edge Systems Group Inc. phone: 250.480.0642 fax: 250.480.1264 cell: 250.920.8830 e-mail: [EMAI

Re: How to recognize robots

2005-10-31 Thread Kay Smoljak
On 11/1/05, Brian Peddle <[EMAIL PROTECTED]> wrote: > http://www.alistapart.com/articles/dropdowns/ An updated, even lighter-weight version of those menus is outlined here: http://www.htmldog.com/articles/suckerfish/dropdowns/ They are the only dropdowns we use now. -- Kay Smoljak http://kay.zom

RE: How to recognize robots

2005-10-31 Thread Brian Peddle
http://www.alistapart.com/articles/dropdowns/ Also google accessible css dropdowns there are a ton. Everything in that book is out there. -Original Message- From: Matt Robertson [mailto:[EMAIL PROTECTED] Sent: Monday, October 31, 2005 5:28 PM To: CF-Talk Subject: Re: How to recognize

Re: How to recognize robots

2005-10-31 Thread Matt Robertson
On 10/31/05, Snake <[EMAIL PROTECTED]> wrote: > This might help > > http://www.the-robotman.com/ Help to tell me how I can get a life-size Robby the Robot? (which by the way I wouldn't mind having but its SOT :-) ) Dave, thanks for the css link. I might pick that book up. I have some very basic

RE: How to recognize robots

2005-10-31 Thread Snake
This might help http://www.the-robotman.com/ Russ -Original Message- From: Dave Francis [mailto:[EMAIL PROTECTED] Sent: 31 October 2005 21:59 To: CF-Talk Subject: RE: How to recognize robots Try this, which is really a resource site for a book of the same name. And look for the link

RE: How to recognize robots

2005-10-31 Thread Dave Francis
: How to recognize robots On 10/31/05, Claude Schneegans <[EMAIL PROTECTED]> wrote: > Yes, and this is why the best dynamic menu systems nowadays use pure > UL/LI lists for the menu items and links, and JS only for the layout, so > that search engines do not hit a wall. Do you ha

Re: How to recognize robots

2005-10-31 Thread Matt Robertson
On 10/31/05, Claude Schneegans <[EMAIL PROTECTED]> wrote: > Yes, and this is why the best dynamic menu systems nowadays use pure > UL/LI lists for the menu items and links, and JS only for the layout, so > that search engines do not hit a wall. Do you have a particular favorite in mind for this so

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>So a rollover script using JS would be a wall for the search engines? Depends it if relies on JS to create and to open the links. Personnally, I use a menu with only UL, LI and >I've talked to few people that manage PPC accounts as their sole business (biased) and they keep telling me SEO is d

Re: How to recognize robots

2005-10-31 Thread Matthew Blatchley
in titles and links etc. Any thoughts? Matt - Original Message - From: "Claude Schneegans" <[EMAIL PROTECTED]> To: "CF-Talk" Sent: Monday, October 31, 2005 10:47 AM Subject: SPAM-LOW: Re: How to recognize robots > >>This is pretty much the ca

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>This is pretty much the case not just for email harvesters, but for search engines in general. Very few can evaluate JavaScript for indexing purposes. Yes, and this is why the best dynamic menu systems nowadays use pure UL/LI lists for the menu items and links, and JS only for the layout, so

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>So the basic idea behind Wpoison is to trap unwary and badly engineered address harvesting web crawlers, and to fool them into adding enormous quantities of completely bogus e-mail addresses to the E-mail address data bases of the spammers... Ok, but if I was an address harvester, I think I wou

RE: How to recognize robots

2005-10-31 Thread Dawson, Michael
The email addresses don't look to be very valid. They are missing the "place". But, I like the concept. ;^) M!ke -Original Message- From: Les Mizzell [mailto:[EMAIL PROTECTED] Sent: Monday, October 31, 2005 9:27 AM To: CF-Talk Subject: Re: How to recognize robots >

RE: How to recognize robots

2005-10-31 Thread Dave Watts
> I don't want to rain on your parade, but I'd be very > surprised if an email harvesting bot would be intelligent > enough to parse that javascript and suck up the generated > output. Remember that the Javascript has to be processed > client side. > > I might be wrong though. I know people do

Re: How to recognize robots

2005-10-31 Thread Les Mizzell
> Ok, but my point is WHY give a do not crawl command to good bots such as > Google? Don't you want you site to be indexed? It's only this one page I don't want indexed. The site this is on has historically been in the top 5 or so on Google and Yahoo for the last two or three years. > I alrea

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>The only bots that ever actually hit it are spam harvestors. Ok, I see. -- ___ REUSE CODE! Use custom tags; See http://www.contentbox.com/claude/customtags/tagstore.cfm (Please send any spam to this address: [EMAIL PROTECTED]) Thanks.

Re: How to recognize robots

2005-10-31 Thread Claude Schneegans
>>I think his point was that good bots such as google will obey his do not crawl command. Ok, but my point is WHY give a do not crawl command to good bots such as Google? Don't you want you site to be indexed? >> He is trying to annoy the scumbags who crawl websites to steal email address so t

Re: How to recognize robots

2005-10-31 Thread Ian Buzer
>One bot loaded this page 15 times before it left! I don't want to rain on your parade, but I'd be very surprised if an email harvesting bot would be intelligent enough to parse that javascript and suck up the generated output. Remember that the Javascript has to be processed client side. I mi

Re: How to recognize robots

2005-10-30 Thread Les Mizzell
> I for one applaud his efforts and would be very interested in seeing the > code if you don't mind Les One bot loaded this page 15 times before it left! That's 75,000 email addresses it harvested. So, the spammer then uses this list to send spam, and if the network admin of the system being use

Re: How to recognize robots

2005-10-30 Thread Les Mizzell
> I think you're having a weird idea of what the robots are doing. > I certainly do not want to annoy Google's bot or others ;-) No, the page I wrote about is specifically EXCLUDED from the googlebot or any other "friendly" bot. The only bots that ever actually hit it are spam harvestors. -- -

Re: How to recognize robots

2005-10-30 Thread Alan Rother
I think his point was that good bots such as google will obey his do not crawl command. He is trying to annoy the scumbags who crawl websites to steal email address so they can spam people, these jerks ignore the robots file and love to follow do not follow links. I for one applaud his efforts an

Re: How to recognize robots

2005-10-30 Thread Claude Schneegans
>>I have a page I've been using on a site just to annoy harvester bots. I think you're having a weird idea of what the robots are doing. I certainly do not want to annoy Google's bot or others ;-) -- ___ REUSE CODE! Use custom tags; See http://www.contentbox.

Re: How to recognize robots

2005-10-30 Thread Les Mizzell
I have a page I've been using on a site just to annoy harvester bots. This page is listed in my "robots" file as a "do not crawl" page, which the harvester bots love to visit! The page contains a javascript version of wpoison written by D.K.Merriman. To a bot, the page in question looks like a

Re: How to recognize robots

2005-10-30 Thread Claude Schneegans
>>The HTTP_User_Agent. Yeah, I should have gessed! ;-) I think, I'll just add the user agent in my stats and work something around it. Are you running CF 7 enterprise or pro? This particular site is still under CF 5 -- ___ REUSE CODE! Use custom tags; See

Re: How to recognize robots

2005-10-30 Thread Michael Dinowitz
>>>The good ones identify themselves. > >Ok, and how do they do that? The HTTP_User_Agent. Google and Yahoo will show up with their name in the agent. Others will as well. On the other hand, when you see something that looks like IE but doesn't act like a human would, then it's time to think it

Re: How to recognize robots

2005-10-30 Thread Claude Schneegans
>>One way is to use an image in your page that is actually a cfm file. Hmmm, this will also be blocked by some email readers, including my own, and I have some pages sent by email (News letter). -- ___ REUSE CODE! Use custom tags; See http://www.contentbox.c

RE: How to recognize robots

2005-10-30 Thread Matthew Walker
> The reason I ask, is that I just implemented some statistics facility > on some of my customers sites. This will count all hits, including those > from robots, so I'd like to be able to discriminates visitors hits from > others. One way is to use an image in your page that is actually a cfm fil

RE: How to recognize robots

2005-10-30 Thread Brian Peddle
: Re: How to recognize robots >>The good ones identify themselves. Ok, and how do they do that? If a series of page requests from the same agent/ip is showing a different cfid/cftoken per request, then either its a user that does not support cookies (even session ones) or its a bot.

Re: How to recognize robots

2005-10-30 Thread Claude Schneegans
>>The good ones identify themselves. Ok, and how do they do that? If a series of page requests from the same agent/ip is showing a different cfid/cftoken per request, then either its a user that does not support cookies (even session ones) or its a bot. Well, actually not really easy to imple

Re: How to recognize robots

2005-10-30 Thread Michael Dinowitz
The good ones identify themselves. The bad ones can be tracked by logging every page request while setting a cfid/cftoken. If a series of page requests from the same agent/ip is showing a different cfid/cftoken per request, then either its a user that does not support cookies (even session ones)

How to recognize robots

2005-10-30 Thread Claude Schneegans
Has any one a trick to detect and identify robots and search engine visiting CF pages? -- ___ REUSE CODE! Use custom tags; See http://www.contentbox.com/claude/customtags/tagstore.cfm (Please send any spam to this address: [EMAIL PROTECTED]) Thanks.