Re: [twitter-dev] users/lookup returns duplicates, missing records for valid users

[Nischal Shetty]

Anything on this? It's been quite some time and user lookup I believe would
be used by a lot of apps.

-N

On 16 March 2011 05:46, Taylor Singletary wrote:

> Still working on it, unfortunately. No ETA for a fix yet. I know it's an
> aggravating bug for anyone who runs into it. Thanks for being patient.
>
> Taylor
>
>
> On Mar 15, 2011, at 5:09 PM, Adrian Petrescu  wrote:
>
> Hi Taylor,
>
> Not trying to be pushy or anything, but have you guys uncovered anything
> related to this issue? It's still happening quite regularly for a couple of
> weeks now.
>
> Thanks,
> Adrian
>
> --
> Twitter developer documentation and resources:
> http://dev.twitter.com/doc
> API updates via Twitter: 
> http://twitter.com/twitterapi
> Issues/Enhancements Tracker:
> 
> http://code.google.com/p/twitter-api/issues/list
> Change your membership to this group:
> 
> http://groups.google.com/group/twitter-development-talk
>
>  --
> Twitter developer documentation and resources: http://dev.twitter.com/doc
> API updates via Twitter: http://twitter.com/twitterapi
> Issues/Enhancements Tracker:
> http://code.google.com/p/twitter-api/issues/list
> Change your membership to this group:
> http://groups.google.com/group/twitter-development-talk
>



-- 
-Nischal
twitter: NischalShetty 
facebook: Nischal 

-- 
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: 
http://groups.google.com/group/twitter-development-talk

Re: [twitter-dev] Re: Getting too many login attempts error though my app uses per user auth apis

[Nischal Shetty]

Thank you so much for getting back. I'm using twitter4j version 2.2.1 , it
uses all the latest api endpoints.

I could not get the raw response data as there's no easy way to do that
using the API and I've been busy  trying to work around the errors my app
has been getting.

Here's the message that I get in my log:

420:Returned by the Search and Trends API when you are being rate limited (
http://dev.twitter.com/pages/rate-limiting).
Returned by the Streaming API:
Too many login attempts in a short period of time.
Running too many copies of the same application authenticating with the same
account name.


This is the URL being used :


http://api.twitter.com/1/friendships/destroy.json?include_entities=false&user_id=userIdOfUserToUnfollow


-N

On 8 April 2011 00:37, Matt Harris  wrote:

> Does the error you get back have any message in the response body?
> Also, just so we are absolutely clear which API method you are calling, can
> you share the exact URL you are hitting and parameters you are sending
> (obscuring any secure information)
>
> On Thu, Apr 7, 2011 at 10:58 AM, nischalshetty 
> wrote:
>
>> It's just not working. No matter how much I wait, as soon as unfollow
>> requests start I get the Error Code 420. I have no clue what to do.
>>
>> Can you please check on your end if my app JustUnfollow.com is being
>> rate limited for some reason. I've been trying from more than 24
>> hours. Thousands of users use the app everyday. This is causing a lot
>> of problem and making me lose users.
>>
>> I request you, please have a check, I have not changed any code, nor
>> am I making any new API calls and my API calls did not increase
>> drastically. I've been running this app over a year now.
>>
>> It's only the unfollow requests that get the 420 error. My app also
>> has follow requests which seem to be working fine.
>>
>> -N
>>
>> On Apr 7, 11:10 am, nischalshetty  wrote:
>> > I'm implementing queuing right now. That will hopefully settle things
>> > a little. Once that is done I will provide whatever info I can to get
>> > this thing sorted.
>> >
>> > @Taylor I would like to specifically thank you for getting back to me.
>> > Considering the amount of distress calls you guys get, a response time
>> > this quick is awesome. I'm hoping queuing would help me stop all the
>> > 420 error codes.
>> >
>> > The way I'll be implementing this is :
>> >
>> > 1. First 420 error occurs, start adding unfollows to Queue
>> > 2. Queue would try to unfollow, if error, defer for 10s
>> > 3. Keep doubling the check time till it reaches say 10 minutes
>> > 4. Keep trying every 10 minutes
>> >
>> > I hope this will ease things out and make my app look good to your
>> > abuse detection algorithm :)
>> >
>> > I tried searching but did not get any definite write up. Is it
>> > advisable to back off even when there are 502/503/500 error? The
>> > reason I ask is because these errors are quite common and if an app
>> > really starts backing off on these errors then there would be a lot of
>> > lag. But I have a feeling not backing off on these errors is one of
>> > the reasons for the 420 error that my apps encountering, though you
>> > would be the right person to answer this.
>> >
>> > -Nischal
>> >
>> > On Apr 7, 6:14 am, nischalshetty  wrote:
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > > Another thing I would like to add is along with these errors I also am
>> > > getting a lot of 502 and 503 requests. Do they have to do something
>> > > with this?
>> >
>> > > -N
>> >
>> > > On Apr 7, 5:58 am, nischalshetty  wrote:
>> >
>> > > > Even as I wrote this and got my servers back up, the 420 errors
>> > > > started all over again :( Please, can you check on your end using my
>> > > > app code or something. I can mail you the app id if you want. The
>> app
>> > > > is justunfollow.com
>> >
>> > > > -N
>> >
>> > > > On Apr 7, 5:55 am, nischalshetty  wrote:
>> >
>> > > > > > There may be others on the IP address you are using abusing the
>> system and
>> > > > > > making you suffer as a result.
>> >
>> > > > > Probably but then this had happened twice as I said earlier and
>> both
>> > > > > the times none of the api's worked until one of you guys removed
>> the
>> > > > > ips from the blacklist.
>> >
>> > > > > > Do you get a HTTP status code 420 just for unfollow requests or
>> is it for
>> > > > > > all requests? Have you attempted to perform the operation in
>> isolation from
>> > > > > > your servers (not tied to your application business logic,
>> perhaps using
>> > > > > > command line tools or Curl, Twurl, etc.)?
>> >
>> > > > > I'm getting it for Unfollow requests only. It does not throw
>> errors
>> > > > > when I try it individually even on my own server. I had a test url
>> set
>> > > > > up and it did not throw error on unfollowing around 200 users.
>> That's
>> > > > > the time I made it live for everyone but it started giving the
>> errors
>> > > > > in a few minutes of going live again.
>> >
>> > > > > > Can you do the foll

Re: [twitter-dev] Getting too many login attempts error though my app uses per user auth apis

[Nischal Shetty]

Oh, so if I send in unfollows from a different IP this would work! Nice,
I'll do that.

I've been monitoring my logs, and as you said the blocks are happening at
intervals in bursts. Alright, what I will do is set up some service on aws
which would do the unfollowing and at the same time I will contact GAE too.

Though I guess a good long term solution is to keep these intensive api
calls to a dedicated IP?

Thank you so much for the support, I really appreciate it.

-Nischal

On 8 April 2011 06:40, Taylor Singletary wrote:

> We've tested this scenario thoroughly and the limiting ceases after a
> short period of time when the maximum levels are reached. We couldn't
> reproduce any out of the ordinary behavior.
>
> The most likely culprit in your case is another application performing
> unfollows on the same GAE IP address you are on; this happens.
> Consider contacting GAE support to see if your app can be moved to a
> different IP or some other solution on their side. Consider moving to
> a hosting environment that allows you to have your own dedicated IP
> addresss.
>
> Taylor
>
> On Thursday, April 7, 2011, nischalshetty 
> wrote:
> > I'm making around 50-60 unfollow requests per minute and it's still
> > throwing the error. Did you have a chance to look at it? My site is
> > unsable :(
> >
> > -N
> >
> > On Apr 8, 1:11 am, Nischal Shetty  wrote:
> >> Thank you so much for getting back. I'm using twitter4j version 2.2.1 ,
> it
> >> uses all the latest api endpoints.
> >>
> >> I could not get the raw response data as there's no easy way to do that
> >> using the API and I've been busy  trying to work around the errors my
> app
> >> has been getting.
> >>
> >> Here's the message that I get in my log:
> >>
> >> 420:Returned by the Search and Trends API when you are being rate
> limited (http://dev.twitter.com/pages/rate-limiting).
> >> Returned by the Streaming API:
> >> Too many login attempts in a short period of time.
> >> Running too many copies of the same application authenticating with the
> same
> >> account name.
> >>
> >> This is the URL being used :
> >>
> >> http://api.twitter.com/1/friendships/destroy.json?include_entities=fa.
> ..
> >>
> >> -N
> >>
> >> On 8 April 2011 00:37, Matt Harris  wrote:
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> > Does the error you get back have any message in the response body?
> >> > Also, just so we are absolutely clear which API method you are
> calling, can
> >> > you share the exact URL you are hitting and parameters you are sending
> >> > (obscuring any secure information)
> >>
> >> > On Thu, Apr 7, 2011 at 10:58 AM, nischalshetty <
> nischalshett...@gmail.com>wrote:
> >>
> >> >> It's just not working. No matter how much I wait, as soon as unfollow
> >> >> requests start I get the Error Code 420. I have no clue what to do.
> >>
> >> >> Can you please check on your end if my app JustUnfollow.com is being
> >> >> rate limited for some reason. I've been trying from more than 24
> >> >> hours. Thousands of users use the app everyday. This is causing a lot
> >> >> of problem and making me lose users.
> >>
> >> >> I request you, please have a check, I have not changed any code, nor
> >> >> am I making any new API calls and my API calls did not increase
> >> >> drastically. I've been running this app over a year now.
> >>
> >> >> It's only the unfollow requests that get the 420 error. My app also
> >> >> has follow requests which seem to be working fine.
> >>
> >> >> -N
> >>
> >> >> On Apr 7, 11:10 am, nischalshetty  wrote:
> >> >> > I'm implementing queuing right now. That will hopefully settle
> things
> >> >> > a little. Once that is done I will provide whatever info I can to
> get
> >> >> > this thing sorted.
> >>
> >> >> > @Taylor I would like to specifically thank you for getting back to
> me.
> >> >> > Considering the amount of distress calls you guys get, a response
> time
> >> >> > this quick is awesome. I'm hoping queuing would help me stop all
> the
> >> >> > 420 error codes.
> >>
> >> >> > The way I'll be implementing 

Re: [twitter-dev] Getting too many login attempts error though my app uses per user auth apis

[Nischal Shetty]

Moved the unfollow tasks out of GAE. Hoping things go smooth now. Allowing
only a subset of users to unfollow. Will gradually allow everyone. Keeping
fingers crossed.

-Nischal

On 8 April 2011 06:52, Nischal Shetty  wrote:

> Oh, so if I send in unfollows from a different IP this would work! Nice,
> I'll do that.
>
> I've been monitoring my logs, and as you said the blocks are happening at
> intervals in bursts. Alright, what I will do is set up some service on aws
> which would do the unfollowing and at the same time I will contact GAE too.
>
> Though I guess a good long term solution is to keep these intensive api
> calls to a dedicated IP?
>
> Thank you so much for the support, I really appreciate it.
>
> -Nischal
>
> On 8 April 2011 06:40, Taylor Singletary wrote:
>
>> We've tested this scenario thoroughly and the limiting ceases after a
>> short period of time when the maximum levels are reached. We couldn't
>> reproduce any out of the ordinary behavior.
>>
>> The most likely culprit in your case is another application performing
>> unfollows on the same GAE IP address you are on; this happens.
>> Consider contacting GAE support to see if your app can be moved to a
>> different IP or some other solution on their side. Consider moving to
>> a hosting environment that allows you to have your own dedicated IP
>> addresss.
>>
>> Taylor
>>
>> On Thursday, April 7, 2011, nischalshetty 
>> wrote:
>> > I'm making around 50-60 unfollow requests per minute and it's still
>> > throwing the error. Did you have a chance to look at it? My site is
>> > unsable :(
>> >
>> > -N
>> >
>> > On Apr 8, 1:11 am, Nischal Shetty  wrote:
>> >> Thank you so much for getting back. I'm using twitter4j version 2.2.1 ,
>> it
>> >> uses all the latest api endpoints.
>> >>
>> >> I could not get the raw response data as there's no easy way to do that
>> >> using the API and I've been busy  trying to work around the errors my
>> app
>> >> has been getting.
>> >>
>> >> Here's the message that I get in my log:
>> >>
>> >> 420:Returned by the Search and Trends API when you are being rate
>> limited (http://dev.twitter.com/pages/rate-limiting).
>> >> Returned by the Streaming API:
>> >> Too many login attempts in a short period of time.
>> >> Running too many copies of the same application authenticating with the
>> same
>> >> account name.
>> >>
>> >> This is the URL being used :
>> >>
>> >> http://api.twitter.com/1/friendships/destroy.json?include_entities=fa.
>> ..
>> >>
>> >> -N
>> >>
>> >> On 8 April 2011 00:37, Matt Harris  wrote:
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> > Does the error you get back have any message in the response body?
>> >> > Also, just so we are absolutely clear which API method you are
>> calling, can
>> >> > you share the exact URL you are hitting and parameters you are
>> sending
>> >> > (obscuring any secure information)
>> >>
>> >> > On Thu, Apr 7, 2011 at 10:58 AM, nischalshetty <
>> nischalshett...@gmail.com>wrote:
>> >>
>> >> >> It's just not working. No matter how much I wait, as soon as
>> unfollow
>> >> >> requests start I get the Error Code 420. I have no clue what to do.
>> >>
>> >> >> Can you please check on your end if my app JustUnfollow.com is being
>> >> >> rate limited for some reason. I've been trying from more than 24
>> >> >> hours. Thousands of users use the app everyday. This is causing a
>> lot
>> >> >> of problem and making me lose users.
>> >>
>> >> >> I request you, please have a check, I have not changed any code, nor
>> >> >> am I making any new API calls and my API calls did not increase
>> >> >> drastically. I've been running this app over a year now.
>> >>
>> >> >> It's only the unfollow requests that get the 420 error. My app also
>> >> >> has follow requests which seem to be working fine.
>> >>
>> >> >> -N
>> >>
>> >> >> On Apr 7, 11:10 am, nischalshetty 
>> wrote:
>> >> >> > I

Re: [twitter-dev] Getting too many login attempts error though my app uses per user auth apis

[Nischal Shetty]

Help!

I guess my ordeals are not over yet :( It's painful. I created a new module
and hosted it on aws. I opened it up to a subset of followers and bang, the
rate limit error appears again! Now I'm confident it's my app that is being
rate limited and rate limited ONLY for Unfollow calls.

I haven't had any peace from the last 48 hours due to this. I request you to
please look into this. Should I  try resetting the twitter keys of my app?
Will doing that help?

Please help me out, things look all gloomy right now. It's the same Error
code 420.



-N

On 8 April 2011 15:09, Nischal Shetty  wrote:

> Moved the unfollow tasks out of GAE. Hoping things go smooth now. Allowing
> only a subset of users to unfollow. Will gradually allow everyone. Keeping
> fingers crossed.
>
> -Nischal
>
>
> On 8 April 2011 06:52, Nischal Shetty  wrote:
>
>> Oh, so if I send in unfollows from a different IP this would work! Nice,
>> I'll do that.
>>
>> I've been monitoring my logs, and as you said the blocks are happening at
>> intervals in bursts. Alright, what I will do is set up some service on aws
>> which would do the unfollowing and at the same time I will contact GAE too.
>>
>> Though I guess a good long term solution is to keep these intensive api
>> calls to a dedicated IP?
>>
>> Thank you so much for the support, I really appreciate it.
>>
>> -Nischal
>>
>> On 8 April 2011 06:40, Taylor Singletary wrote:
>>
>>> We've tested this scenario thoroughly and the limiting ceases after a
>>> short period of time when the maximum levels are reached. We couldn't
>>> reproduce any out of the ordinary behavior.
>>>
>>> The most likely culprit in your case is another application performing
>>> unfollows on the same GAE IP address you are on; this happens.
>>> Consider contacting GAE support to see if your app can be moved to a
>>> different IP or some other solution on their side. Consider moving to
>>> a hosting environment that allows you to have your own dedicated IP
>>> addresss.
>>>
>>> Taylor
>>>
>>> On Thursday, April 7, 2011, nischalshetty 
>>> wrote:
>>> > I'm making around 50-60 unfollow requests per minute and it's still
>>> > throwing the error. Did you have a chance to look at it? My site is
>>> > unsable :(
>>> >
>>> > -N
>>> >
>>> > On Apr 8, 1:11 am, Nischal Shetty  wrote:
>>> >> Thank you so much for getting back. I'm using twitter4j version 2.2.1
>>> , it
>>> >> uses all the latest api endpoints.
>>> >>
>>> >> I could not get the raw response data as there's no easy way to do
>>> that
>>> >> using the API and I've been busy  trying to work around the errors my
>>> app
>>> >> has been getting.
>>> >>
>>> >> Here's the message that I get in my log:
>>> >>
>>> >> 420:Returned by the Search and Trends API when you are being rate
>>> limited (http://dev.twitter.com/pages/rate-limiting).
>>> >> Returned by the Streaming API:
>>> >> Too many login attempts in a short period of time.
>>> >> Running too many copies of the same application authenticating with
>>> the same
>>> >> account name.
>>> >>
>>> >> This is the URL being used :
>>> >>
>>> >>
>>> http://api.twitter.com/1/friendships/destroy.json?include_entities=fa...
>>> >>
>>> >> -N
>>> >>
>>> >> On 8 April 2011 00:37, Matt Harris  wrote:
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> > Does the error you get back have any message in the response body?
>>> >> > Also, just so we are absolutely clear which API method you are
>>> calling, can
>>> >> > you share the exact URL you are hitting and parameters you are
>>> sending
>>> >> > (obscuring any secure information)
>>> >>
>>> >> > On Thu, Apr 7, 2011 at 10:58 AM, nischalshetty <
>>> nischalshett...@gmail.com>wrote:
>>> >>
>>> >> >> It's just not working. No matter how much I wait, as soon as
>>> unfollow
>>> >> >> requests start I get the Error Code 420. I have no clue what to do.
>>> >>
&g

Re: [twitter-dev] Getting too many login attempts error though my app uses per user auth apis

[Nischal Shetty]

Ok wait. I just gave a brand new static ip to my instance on EC2. Sorry, I
don't know how these things work, it's been a few hours since I started with
this. I'll update soon on what happens now. Though if you find time then
having a look again just to confirm that JustUnfollow isn't being blocked
would be a big relief.

-Nischal

On 8 April 2011 19:56, Nischal Shetty  wrote:

> Help!
>
> I guess my ordeals are not over yet :( It's painful. I created a new module
> and hosted it on aws. I opened it up to a subset of followers and bang, the
> rate limit error appears again! Now I'm confident it's my app that is being
> rate limited and rate limited ONLY for Unfollow calls.
>
> I haven't had any peace from the last 48 hours due to this. I request you
> to please look into this. Should I  try resetting the twitter keys of my
> app? Will doing that help?
>
> Please help me out, things look all gloomy right now. It's the same Error
> code 420.
>
>
>
> -N
>
>
> On 8 April 2011 15:09, Nischal Shetty  wrote:
>
>> Moved the unfollow tasks out of GAE. Hoping things go smooth now. Allowing
>> only a subset of users to unfollow. Will gradually allow everyone. Keeping
>> fingers crossed.
>>
>> -Nischal
>>
>>
>> On 8 April 2011 06:52, Nischal Shetty  wrote:
>>
>>> Oh, so if I send in unfollows from a different IP this would work! Nice,
>>> I'll do that.
>>>
>>> I've been monitoring my logs, and as you said the blocks are happening at
>>> intervals in bursts. Alright, what I will do is set up some service on aws
>>> which would do the unfollowing and at the same time I will contact GAE too.
>>>
>>> Though I guess a good long term solution is to keep these intensive api
>>> calls to a dedicated IP?
>>>
>>> Thank you so much for the support, I really appreciate it.
>>>
>>> -Nischal
>>>
>>> On 8 April 2011 06:40, Taylor Singletary 
>>> wrote:
>>>
>>>> We've tested this scenario thoroughly and the limiting ceases after a
>>>> short period of time when the maximum levels are reached. We couldn't
>>>> reproduce any out of the ordinary behavior.
>>>>
>>>> The most likely culprit in your case is another application performing
>>>> unfollows on the same GAE IP address you are on; this happens.
>>>> Consider contacting GAE support to see if your app can be moved to a
>>>> different IP or some other solution on their side. Consider moving to
>>>> a hosting environment that allows you to have your own dedicated IP
>>>> addresss.
>>>>
>>>> Taylor
>>>>
>>>> On Thursday, April 7, 2011, nischalshetty 
>>>> wrote:
>>>> > I'm making around 50-60 unfollow requests per minute and it's still
>>>> > throwing the error. Did you have a chance to look at it? My site is
>>>> > unsable :(
>>>> >
>>>> > -N
>>>> >
>>>> > On Apr 8, 1:11 am, Nischal Shetty  wrote:
>>>> >> Thank you so much for getting back. I'm using twitter4j version 2.2.1
>>>> , it
>>>> >> uses all the latest api endpoints.
>>>> >>
>>>> >> I could not get the raw response data as there's no easy way to do
>>>> that
>>>> >> using the API and I've been busy  trying to work around the errors my
>>>> app
>>>> >> has been getting.
>>>> >>
>>>> >> Here's the message that I get in my log:
>>>> >>
>>>> >> 420:Returned by the Search and Trends API when you are being rate
>>>> limited (http://dev.twitter.com/pages/rate-limiting).
>>>> >> Returned by the Streaming API:
>>>> >> Too many login attempts in a short period of time.
>>>> >> Running too many copies of the same application authenticating with
>>>> the same
>>>> >> account name.
>>>> >>
>>>> >> This is the URL being used :
>>>> >>
>>>> >>
>>>> http://api.twitter.com/1/friendships/destroy.json?include_entities=fa.
>>>> ..
>>>> >>
>>>> >> -N
>>>> >>
>>>> >> On 8 April 2011 00:37, Matt Harris 
>>>> wrote:
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>