x27;t done!
- Original Message -
From: "Randell B Adkins" <[EMAIL PROTECTED]>
To: "CF-Community" <[EMAIL PROTECTED]>
Sent: Thursday, July 17, 2003 11:06 AM
Subject: Re: Search Engine / robot.txt
| Then what is the best way to prohibit directories from
| being
]>
Sent: Thursday, July 17, 2003 8:11 AM
Subject: Re: Search Engine / robot.txt
| oi Randell!!
|
| yeah, I don't even know if they will index the url but they go
no further
than that(if they
| use the robots.txt file)
|
|
|
|
|
| Thursday, J
17, 2003 8:11 AM
Subject: Re: Search Engine / robot.txt
| oi Randell!!
|
| yeah, I don't even know if they will index the url but they go no further
than that(if they
| use the robots.txt file)
|
|
|
|
|
| Thursday, July 17, 2003, 8:57:55 AM
oi Randell!!
yeah, I don't even know if they will index the url but they go no further than
that(if they
use the robots.txt file)
Thursday, July 17, 2003, 8:57:55 AM, you wrote:
RBA> Thanks also by using:
RBA> User-agent: *
RBA> Disallow: /
//www.forta.com/cf/isp/isp.cfm?isp_id=772
==
If you are not satisfied with my service, my job isn't done!
- Original Message -
From: "Randell B Adkins" <[EMAIL PROTECTED]>
To: "CF-Community" <[EMAIL PROTECTED]>
Sent:
Thanks also by using:
User-agent: *
Disallow: /
That would allow all BOTs to hit your site but not index any pages.
Is that right? so in theory it would index just the URL and the
META tag data?
>>> [EMAIL PROTECTED] 07/17/03 08:49AM >>>
oi Randell!!
http://www.google.com/search?hl=en&ie=UT
Here is one that I have used in the past:
http://www.payperclicksearchengines.com/creating-robots-txt-file.shtml
> -Original Message-
> From: Randell B Adkins [mailto:[EMAIL PROTECTED]
> Sent: Thursday, July 17, 2003 8:44 AM
> To: CF-Community
> Subject: Search Eng
oi Randell!!
http://www.google.com/search?hl=en&ie=UTF-8&oe=UTF-8&q=how+to+build+a+robots.txt&btnG=Google+Search
first return looks like it might splain it...
mine all just say:
User-agent: *
Disallow: /
:)
Thursday, July 17, 2003, 8:43:30 AM, you wrot
Does anyone have a good resource for building a robot.txt
file for search engine bots?
~|
Archives: http://www.houseoffusion.com/cf_lists/index.cfm?forumid=5
Subscription:
http://www.houseoffusion.com/cf_lists/index.cfm?method=s