This is a great discussion although some of you can type wa-a-a-y better than I can which is why I use bullets and brevity. Please allow me to digest some of this, and Lisa and I will figure out how to condense it down.
Keep it coming! I hope we can basically get something out the door, then let the marketplace chew on it, and then rest knowing we've done what we can. As stated, we have no enforcement powers but we haven't made it easy to have influence. That is my goal with this document. I will take with me on the plane to work on, so please keep the comments coming! Aloha, Rox On 6/6/07, David Meade <[EMAIL PROTECTED]> wrote: > > > > > > > On 6/6/07, Steve Watkins <[EMAIL PROTECTED]> wrote: > > > > My point about it potentially hampering end-users was wrong because > > Robots.txt is easy to ignore. This also means I assume those bad > > actors who dont care would never bother getting their stuff to look at > > robots.txt in the first place. I completely agree that opt-out sucks > > and people shouldnt be expected to have to do that, I just fear > > robots.txt is too easily ignored and so doesnt really solve the problem. > > True. It is easy to ignore ... but it's also easy to use and very commonly > used already. If only MAs would identify themselves in the user-agent we > could use this as a CO-owned opt-out mechanism. We can basically ask for > them to do this, or ask for them to create a user-account based opt-out > feature on their site. > > My instinct is that if our best practices document cant inspire an MA to > bother to identify themselves in their user-agent ... it isn't going to > inspire them to code up a whole user-opt-out feature. (setting your > user-agent is REALLY easy to do). > > What I was thinking of that would potentially harm end users with > > unusual setups, would be attempts to do something equivalent to > > robots, but that is actually real enforcement, real technological > > measures that the outside party cannot ignore. Eg reconfiguring the > > webserver to block access from certain addresses or those using > > certain clients to read the feed. > > Yeah we want to avoid that sort of thing naturally. I think (Rox?) that > this best practices document is a sort of "guidelines for parties that want > to get along". So yes I agree that we don't want to define some sort of > technical blocking or enforcement scheme ... but rather offer guidelines to > the parties that say "here's how we can best get along". > > Asking MAs to "identify themselves in user-agents and respect robots.txt" > is > merely asking them to abide by best practices that honestly have already > been in place. And I believe most feed spiders do have the functionality if > not the practice of honoring robots.txt. > > To me it seems the mutually-least-painful option. For MAs it should be > REALLY easy to comply with and for COs it prevents us from having to use > the > system we dislike in order to opt-out (each and every time another one > comes > around). > > So that stuff pushes me back towards technology that MAs can ignore, > > but good ones will hopefully read. And I much prefer stuff in the RSS > > feed than the robots idea. > > Yeah, but again now we're getting into licensing rather than opt-out. I > agree that there should totally be an expectation to read and honor the > license information in the RSS feed. But site-based opt-out cant really > happen in the feed. > > When I say its existing technology, I meant > > its v.likely that these MA sites are already reading your feed. Wheras > > its unlikely they read your robots.txt, because they arent generally > > getting your content that way. > > Hmm. Are you sure? I thought most of them operated as a spider ... and most > of those now-a-days have robots.txt check out-of-the-box. In anycase it > would be very simple to implement. > > I have been disappointed to date with how many companies bother to > > read & do something with the creative commons or copyrigt license info > > that can be put into RSS feeds. But the battle must be to improve > > this, I think theres more chance of it than getting them to go to the > > sites & read robots files, and which site should they go to anyway - > > the site that hosts the blog or the site that hosts the video, if > > different? > > Yeah ... but I'm not sure the robots.txt file can really help in the "how > do > we get people to respect our license" discussion .. thats different than > just "how can we ensure COs can opt-out" from any given site. (regardless > as > to how good they are at license compliance). > > -- > http://www.DavidMeade.com > > [Non-text portions of this message have been removed] > > -- Roxanne Darling "o ke kai" means "of the sea" in hawaiian 808-384-5554 http://www.twitter.com/roxannedarling http://www.beachwalks.tv http://www.barefeetshop.com http://www.barefeetstudios.com
