At 11:42 AM 10/1/99 -0500, Geoff Hutchison wrote:
>At 11:20 AM +0530 10/1/99, B.G. Mahesh wrote:
>>But nothing from food.indiainfo.com is being indexed :-(
>
>What does it look like if you run htdig -vvv?
BAD TAG IN SERIALIZED DATA: 97
BAD TAG IN SERIALIZED DATA: 111
BAD TAG IN SERIALIZED DATA: 32
I am trying to get you the full report. The line keeps getting cut off.
htdig must have an option where the output is stored in file.
> Is there something on that server (like a robots.txt file) that prevents
> it from indexing the server?
I don't have any robots.txt on http://food.indiainfo.com
But I do have one on http://www.indiainfo.com which reads
indiahub.kode.net:india(17)% more robots.txt
# Stop peeking! You know what this file is about..
User-agent: *
Disallow: /wwwboard/messages/
Disallow: /search/
Disallow: /docs/
Disallow: /private/
Disallow: /india/media.html
Disallow: /india/health.html
Disallow: /india/business.html
Disallow: /india/tv.html
Disallow: /india/radio.html
Disallow: /india/bollywood.html
Disallow: /weather.html
Disallow: /india/articles.html
Disallow: /india/vidyasagar.html
-----
B.G. Mahesh
http://www.indiainfo.com/
http://www.CareerIndia.com/
Email: [EMAIL PROTECTED]
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the SUBJECT of the message.
