I don't have a domain to point back to. I'm doing data-mining and analysis on a server that isn't public. I have set the User-Agent to something unique (I thought you were saying to change it for every request?). Yet I'm still getting rate limited and told to back off a lot. Ryan S said it might still have to do something with the DDoS attack, which is still going on apparently.
dave On Aug 11, 4:13 pm, Chad Etzel <c...@twitter.com> wrote: > The referrer is not as important as the user-agent. You can also put > your URL in the user-agent instead. > -Chad > > On Tue, Aug 11, 2009 at 4:09 PM, Larry Wright<larrywri...@gmail.com> wrote: > > In addition to setting a unique user-agent, I believe it was requested that > > we set a referrer header that pointed back to a domain. > > > On Tue, Aug 11, 2009 at 9:30 AM, David Fisher <tib...@gmail.com> wrote: > > >> While i haven't done scientific testing of this, I was able to run up > >> to 3-4 instances of my search script prior at a time before it told me > >> to enhance my calm. Now I'm barely able to run one without hitting the > >> limit. I can put delays in my code to slow it down, but I'm wondering > >> if this is just a symptom of the aftermath of the DDoS attack or > >> something else? My server has a dedicated IP and no one else runs code > >> from it, so it isn't other people on my IP hitting the Search API. > > >> Maybe I need to talk about Search API whitelisting... > > >> dave > >>http://webecologyproject.org > > > -- > > Larry Wright
