On Thursday, August 25, 2005, at 03:12 PM, Walter Underwood wrote:
I would call desktop clients "clients" not "robots". The distinction is
how they add feeds to the polling list. Clients add them because of
human decisions. Robots discover them mechanically and add them.
So, clients should act like browsers, and ignore robots.txt.
How could this all be related to aggregators that accept feed URL
submissions? I'd imagine the desired behavior is the same as for
crawlers--should they check for robots.txt at the root of any domain
where a feed is submitted? How about cases where the feed is hosted on
a site other than the website that it's tied to (for example, a service
like FeedBurner) so some other site's robot.txt controls access to the
feed (...or at least tries to)?
We've already rejected the idea of trying to build DRM into feeds--is
there some way to sidestep the legal complexities and problems that
would arise from trying to to that and at the same time enable machine
readable statements about what the publisher wants to allow others to
do with the feed, and things they want to prohibit, into the feed? If
we're not qualified to design an extension to do that, is there someone
else who is qualified, and who cares enough to do it?
