Matt, far from getting into RFC debates, but really concerned for the non-server apps out there, which may not have full control over the network infrastructure they run on. If I set up my own server(s) at a data center, I sure can take care of sending you the right referrer and user-agent, but unfortunately that's not the case in many environments behind firewalls and / or proxies.
What's your point on that? I fully understand your intention and the need for getting some identification - so happy to discuss anything that'll also work through restricted network access. Thanks, Marco 2009/6/16 Matt Sanford <m...@twitter.com> > Hi there, > While all of this flame is keeping my feet warm it's not really > productive. This isn't Slashdot comments, let's try and remain on topic > rather the getting into RFC debates. To be even more explicit than my > previous email: Use the user-agent. Referrer will be taken care of by > browsers and I see as a fallback for client-side JSON > users rather than a replacement for a user-agent. > > The subsequent reply from Michael Ivey about how this helps is dead on. > With no context at all I'm forced to block all of ECS/AppEngine/Yahoo Pipes > is one person misbehaves. Nobody likes that. Since search is not > authenticated OAuth does not really help here. We may be forced to make > search authenticated if we can't find a reasonable way to sort the good from > the bad. This is a first attempt at helping us cut out poorly build spam > scripts and shorten the time I spend researching each abuser. It saves time > and lets me fix more bugs, assuming I don't spend the newly saved time in > RFC debates, that is :) > > Thanks; > – Matt Sanford / @mzsanford > Twitter Dev > > On Jun 16, 2009, at 12:39 PM, Stuart wrote: > > 2009/6/16 Naveen Kohli <naveenko...@gmail.com> > >> Redefining HTTP spec, eh :-) >> Whatever makes twitter boat float. Lets hope for the best. Just concerned >> that some firewalls or proxies tend to remove "referrer". > > > What a completely ridiculous thing to say. It's not "redefining" anything. > If Twitter want to require something in order to access their service they > absolutely have that right. It's not like they're saying every HTTP server > should start requiring these headers. > > It's true that some firewalls and proxies remove the referrer header, and > some also remove the user agent header. > > I'm somewhat unclear on exactly how this stuff is supposed to help. If an > application sets out to abuse the system they'll simply set the headers so > they look like a normal browser. I don't see what purpose requiring these > headers to be something useful will actually serve. IMHO you might as well > "require" the source parameter for all API requests that use basic auth > which is simple for all apps to implement; OAuth clearly carries > identification with it already. > > -Stuart > > -- > http://stut.net/projects/twitter > > On Tue, Jun 16, 2009 at 1:05 PM, Stuart <stut...@gmail.com> wrote: >> >>> >>> It's optional in the HTTP spec, but mandatory for the Twitter Search >>> API. I don't see a problem with that. >>> >>> Doug: Presumably the body of the 403 response will contain a suitable >>> descriptive error message in the usual format? >>> >>> -Stuart >>> >>> -- >>> http://stut.net/projects/twitter >>> >>> 2009/6/16 Naveen Kohli <naveenko...@gmail.com>: >>> > Why would you make decision based on "Referrer" which is an OPTIONAL >>> header >>> > field in HTTP protocol? Making decision based on something that is >>> > "REQUIRED" may be more appropriate. >>> > >>> > >>> > On Tue, Jun 16, 2009 at 12:33 PM, Doug Williams <d...@twitter.com> >>> wrote: >>> >> >>> >> Hi all, >>> >> The Search API will begin to require a valid HTTP Referrer, or at the >>> very >>> >> least, a meaningful and unique user agent with each request. Any >>> request not >>> >> including this information will be returned a 403 Forbidden response >>> code by >>> >> our web server. >>> >> >>> >> This change will be effective within the next few days, so please >>> check >>> >> your applications using the Search API and make any necessary code >>> changes. >>> >> >>> >> Thanks, >>> >> Doug >>> > >>> > >>> > >>> > -- >>> > Naveen K Kohli >>> > http://www.netomatix.com >>> > >>> >> >> >> >> -- >> Naveen K Kohli >> http://www.netomatix.com >> > > >
