At 5:13 PM -0600 12/1/01, Geoff Hutchison wrote: >At 2:43 AM -0500 12/1/01, [EMAIL PROTECTED] wrote: >>where do i put robots.txt file(s)? in every url directory i want to search? >>(i cannot put it in apache, et al). > >No, it goes in the root of the server, e.g.: > >http://www.foo.com/robots.txt
what i should have said is: the urls to the students' sites are formed, e.g.,: http://slis.lis.sco.edu/~H765-87 the actual paths on the server to their home dirs is: /usr2/foo/foo/foo/~H765-87/public_html/index.html and each student is incremented, e.g., ~H765-87, -88, -89.... can i put a robots.txt file in each ~homdir or in each public_html dir? Thanks. TR _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
