Re: [twitter-dev] Encrypted data over Twitter
On Wed, May 26, 2010 at 10:16:40AM -0700, John Adams wrote: I think you're referring to ITAR, most of which was repealed in 1997. Until 1996?1997, ITAR classified strong cryptography as arms and prohibited their export from the U.S. Times have changed quite a bit since then. Also, as I understood the matter at the time, ITAR only restricted international distribution of encryption technology (i.e., crypto algorithms and implementations of those algorithms) and didn't care at all about what might be done with the resulting ciphertext or where it might be sent. However... Amateur radio regulations in the US do include (or at least did include last I heard) an absolute ban on transmitting encrypted information. It used to be fairly common for ham radio operators to use packet radio technologies to transmit TCP/IP data over the amateur bands in order to get free roaming internet access; I expect this practice is less common today (thanks to widespread cellular data access), but not extinct. When using packet radio on amateur bands, ssh/ssl/etc. are not legal due to the crypto ban on those bands. I would expect encrypted tweets to be illgal under the same regulations, although that may be dependent on whether the receiver has the means to decrypt them. -- Dave Sherohman
Re: [twitter-dev] Basic authentication
On Mon, May 17, 2010 at 11:22:56AM -0700, Jef Poskanzer wrote: Have you considered keeping basic auth enabled, but only for https? This would be secure against packet sniffing and would probably use less resources than OAuth. The issue with Basic Auth isn't packet sniffing. The issue with Basic Auth is that you're giving your Twitter login credentials to a third party (i.e., some application that is not Twitter) and, in most cases, allowing that third party to store your Twitter login credentials for future use. This is Very Bad. General security best practice states that you should *never* give your login credentials for *any* system to *any* third party for *any* reason. To put it another way, how about if you just give me the username and password for your bank's website so that I can deposit some money for you? I won't use it to transfer money to my account, or to lock you out of your bank account, and I'll forget them as soon as the deposit has been made. Really. I promise. *That* is the problem with Basic Auth, regardless of whether I use https when I log in to your bank account or not. -- Dave Sherohman
Re: [twitter-dev] Re: Can our twitter app call /oauth/revoke?
On Sun, May 02, 2010 at 07:12:00PM -0700, Mr-Yellow wrote: oAuth Core 1.0 Service Providers SHOULD allow Users to revoke Access Tokens. Without this end-point it's impossible for users to disconnect a twitter account. If a user links the wrong account then wishes to remove this link they their only option is a lot of navigation to twitters controls. Your app can still have a 'logout' button which causes the app to forget the user's oauth credentials. As far as your app is concerned, this is the same as if the credentials had been revoked. It's not the ideal situation (if a third party had intercepted the user's credentials and also had access to your app's credentials, they could still use them to impersonate the user), but it *is* possible for a user to disconnect one twitter account from an app and link another without having to go to twitter.com and find the 'revoke credentials' page. -- Dave Sherohman
Re: [twitter-dev] Re: countdown to OAuth / basic auth removal / OAuthcalypse
On Mon, Apr 26, 2010 at 11:09:38AM -0400, Dean Collins wrote: Yeh but John, who is going to install MySQL for a desktop client? 1) John was responding to someone who said a database wouldn't work for [him] since I do not have a desktop app, end I do not store anything in a database. If he doesn't have a desktop app, then he's talking about a web app and, as John said, pretty much every web host out there gives you MySQL, no installation required. 2) MySQL isn't the only RDBMS out there... If I were writing a desktop app which needed to handle significant amounts of data, I'd use SQLite. The SQLite libraries in every language I'm aware of aren't just interfaces to the database, but rather contain the entire database engine, so there's no external application to install. This is a well- proven approach, with Firefox probably being the biggest-name application using an embedded SQLite database engine to handle its data. http://en.wikipedia.org/wiki/SQLite -- Dave Sherohman -- Subscription settings: http://groups.google.com/group/twitter-development-talk/subscribe?hl=en
Re: [twitter-dev] Re: Upcoming changes to the way status IDs are sequenced
On Thu, Apr 08, 2010 at 05:03:29PM -0700, Naveen wrote: However, I wanted to be clear and feel it should be made obvious that with this change, there is a possibility that a tweet may not be delivered to client if the implementation of how since_id is currently used is not updated to cover the case. I still envision the situation as more likely than you seem to believe and figure as tweet velocity increases, the likelihood will also increase; But I am assuming have better data to support your viewpoint than I and shall defer. Maybe I'm just missing something here, but it seems trivial to fix on Twitter's side (enough so that I assume it's what they've been planning from the start to do): Only return tweets from closed buckets. We are guaranteed that the buckets will be properly ordered. The order will only be randomized within a bucket. Therefore, by only returning tweets from buckets which are no longer receiving new tweets, since_id works and will never miss a tweet. And, yes, this does mean a slight delay in getting the tweets out because they have to wait a few milliseconds for their bucket to close before being exposed to calls which can use since_id, plus maybe a little longer for the contents of that bucket to be distributed to multiple servers. That's still going to only take time comparable to round-trip times for an HTTP request to fetch the data for display to a user and be far, far less than the average refresh delay required by those clients which fall under the API rate limit. I submit, therefore, that any such delay caused by waiting for buckets to close will be inconsequential. -- Dave Sherohman
Re: [twitter-dev] Whats the best practice for caching data?
On Wed, Feb 17, 2010 at 05:52:28PM -0800, Dmitri Snytkine wrote: I have 2 choices: store the data in the database and put cookie in user's browser and next time user visits, I can just pull the username, name, etc from my database Or I can use user's access token/secret that I also store in database to get the fresh data from Twitter. Getting fresh data will guarantee that I have user's latest color settings, background, avatar, description But I may run over 150 requests per hour very easily. How is this usually done by other app developers? What's the best practice for synchronizing user's settings with Twitter? The way I'm doing it on FishTwits is to cache the most recent profile data for each user. So I'm basically doing your first option (store it in the database), but refreshing it with the latest fresh profile data whenever I send or retrieve a status update for the user - that information is already coming back with the status, so I can update it without having to wait for any extra requests to complete. -- Dave Sherohman
Re: [twitter-dev] Re: More OAuth Change Please - Simple Stuff
On Tue, Feb 09, 2010 at 04:07:29PM -0800, Dewald Pretorius wrote: I don't see how Allow being the default can be a security issue. Authorizing an app that you don't intend to has the potential to result in significant harm - forged status updates, follow list vandalism, etc. It *is* a security issue. In contrast, accidentally denying an app that you intend to authorize (as would happen in the current setup) merely results in minor inconvenience by requiring you to go back and try again. Unless you happen to be an automated tool which lacks the flexibility to allow the actual user to specify which button you should hit, but I'd say that's evidence that RoboForm is broken, not that there's a problem with Twitter's oAuth form. The user is specifically sent to that page for the purpose of granting access. Only a minuscule number of users will need to click the Deny button. The user is specifically sent to that page because the app wants them to grant access, which does not necessarily mean that the user wishes to do so. Most of us on this list are trivially capable of creating an app that displays a click here to win $1,000,000 and a pony link which takes the user to the Twitter oAuth page without giving them any prior indication that Twitter would be involved. In that case, the vast majority of users would (or at least should) click Deny because they were sent there under false pretenses. Most frequent case does not imply secure. If it did, we wouldn't bother with passwords or authentication at all because, hey, it's pretty much only ever me who uses my Twitter account. Only a miniscule number of attempts to update @DaveSherohman's status are made by people other than me. -- Dave Sherohman
Re: [twitter-dev] Source parameter request for mobile Twitter app ignored (and issues with Twitter's policy toward oAuth on mobile/desktop)
On Mon, Feb 01, 2010 at 08:29:18PM +, Aral Balkan wrote: I would really love to have a comment on from you guys for the blog post I'm writing: is Twitter actively discouraging the creation of new mobile and desktop apps? I'm not Raffi. I don't even work for Twitter. But I am very confident that the purpose of their policy regarding source params has nothing to do with penalizing anyone or actively discouraging the creation of new applications. I _really_ hope you can reconsider this as I see no logic whatsoever behind this policy. The logic is very simple: OAuth provides Twitter with the ability to identify the sending application. Basic Auth does not. Therefore, Basic Auth source params are easily forged, allowing apps to trivially impersonate each other, which is clearly undesirable. (Unfortunately, this logic is not watertight, in that desktop/mobile apps are vulnerable to having their OAuth keys extracted from them, in which case they could still be impersonated, but that's the reasoning I've seen given previously for the policy.) -- Dave Sherohman
Re: [twitter-dev] Re: A New API For Browserless Apps?
On Fri, Dec 11, 2009 at 04:03:01AM -0800, Duane Roelands wrote: It seems clear to me from Raffi's comments on it that this third oauth flow is intended solely to enable Twitter use from embedded applications or in other environments in which it is not possible to use the existing oauth flows because there is no way to bring up a browser. And this will be enforced...how? The same way that oauth usage is enforced today: By attempting to educate users that they should not provide their login credentials to third-party websites. Developers who want the easier implementation and easier user experience will choose those methods, As noted in my earlier message, I dispute your assertion that type in name, remember password, type in password (even without an intermediate try to remember which password was for Twitter, enter a couple incorrect and/or typoed attempts, give up, go find the piece of paper where password is written down phase) is an easier user experience than click, click, click, done. Again, I grant that enter username and password is more familiar from long use, but it is not actually easier. In my experience (both as a developer and as a sysadmin), users prefer authentication methods which do not require them to remember or locate credentials. They often initially resist these alternative methods due to unfamiliarity, but, once they've done it enough times that they no longer have to think about how to carry out each step, they are highly resistant to going back to more traditional authentication methods which require them to do more work to log in. The fact that methods exist which remove this extra work while also being more secure is just gravy for those of us who are concerned with such things. Once users are accustomed to the ease of passwordless oauth login from one site, I fully expect them to want that same convenience on other sites. I further expect that this is already happening as users find that the existing web-based oauth flow is easier for them than basic auth. (The exception to these comments is in the case where oauth fails to work properly, which can be intensely frustrating for users. However, this is much less commmon now than it was even three months ago and, more importantly, by allowing Twitter to focus exclusively on oauth, I expect that replacing basic auth with this third oauth flow will lead to increased reliability of *all* oauth flows.) It in no way prevents or discourages use of the existing oauth flows in scenarios where a browser is available. Prevents? No. Discourages? Absolutely. It provides an incentive for poor security decisions by developers and users. Lazy developers will always take the easy way out, sure, and that's generally going to be less secure. But I'm still not seeing any coherent argument for how the planned third oauth flow will, in any way, be *worse* than the existing basic auth scheme. It may not be an absolutely perfect world in which absolutely nothing except Twitter itself is capable of accepting a Twitter password, but it's still a big improvement on what we have today. -- Dave Sherohman
Re: [twitter-dev] Re: A New API For Browserless Apps?
On Thu, Dec 10, 2009 at 07:33:28AM -0700, John Meyer wrote: okay, forgive me if I'm wrong, but wasn't the whole point of oAuth that the application didn't need to know the username/password? That the user would grant access to the application and then the application would store that rather than the actual username/password. Or am I missing the point of going to an oAuth system? That is *a* point of oauth, not *the* point. It's probably the most obvious point from a naive end-user viewpoint, but there are also other advantages to oauth, even when the keys are obtained by entering your username/password. Just off the top of my head: - The app doesn't need to store your username/password in a retrievable format, so an attacker who compromises the application's data store will not gain access to them. - Your username/password will not be transmitted across the internet or wireless LAN connections on every request made, which greatly reduces the window of vulnerability for an attacker to intercept them in transit. - You can change your Twitter username and/or password without then having to immediately reauthorize the app by re-entering the new username/password. - Conversely, if you wish to revoke a specific app's access, you can do so without affecting any other application. (Unless the app in question has stored your username/password in a retrievable format, in which case it could just get new oauth credentials, of course.) - If oauth is the only allowed authentication method, a rogue app would not be able to gain full access to your account. Perhaps most importantly, it would not be capable of changing your password and locking you out. - On Twitter's side, I'm sure it will simplify things considerably for all API methods to support only a single authentication method. -- Dave Sherohman
Re: [twitter-dev] Re: A New API For Browserless Apps?
On Thu, Dec 10, 2009 at 10:40:20AM -0800, Duane Roelands wrote: Many of us in the developer community have been strongly pushing the point of view that third-party apps should never be asking for user credentials. We did so because we believed that Twitter was firmly committed to the security of the ecosystem and protecting the accounts of its users. It now appears that this belief was in error. I see no evidence that Twitter is anti-security here. This decision is going to actively hurt developers who chose the more secure implementation. Application A just lets me log in with my Twitter credentials, but Application B wants me to go through this harder process. Most users will choose option A, and the more-secure application B loses users. this decision punishes developers who chose the more secure model. It's disappointing, because a lot of developers have worked very hard to bring OAuth implementations to the community that were robust and secure and **didn't require a user to hand over their Twitter credentials**. 1) Given that third-party apps are currently able to request username/ password and use them for Basic Auth, I do not see how apps being able to request username/password and use them for oauth makes things any worse. I can see an (IMO weak) argument that this may prevent things from getting better, but it will not make them worse. 2) I seem to be the odd man out here, but I consider the current oauth web application auth method (which I'm assuming is the more-secure application B) to be the easier method. Click-click-click, done. No typing in my username or remembering my password required. Username/ password is definitely the more familiar method for most users, since it's what they've been using all their (online) lives, but I reject your assertion that it's easier to complete. There was a great opportunity here for Twitter to be a security leader in the social network space by saying We don't want our users giving their Twitter credentials to anyone except Twitter. It's a shame they didn't stick to their gun; the result is going to be a less- secure ecosystem. They still can stick to their guns and say We don't want our users giving their Twitter credentials to any *web site or desktop application* except Twitter. It seems clear to me from Raffi's comments on it that this third oauth flow is intended solely to enable Twitter use from embedded applications or in other environments in which it is not possible to use the existing oauth flows because there is no way to bring up a browser. It in no way prevents or discourages use of the existing oauth flows in scenarios where a browser is available. Really, the current lack of oauth delegation is a far bigger obstacle to being able to say don't give your Twitter password to anyone else than the ability to turn a username/password into oauth credentials will ever be. -- Dave Sherohman
Re: [twitter-dev] Unexplored Dark Underbelly of OAuth
On Thu, Dec 10, 2009 at 06:09:10PM -0800, Dewald Pretorius wrote: Now, will there be hackers and scammers that will decompile TweetDeck's code to get their grubby hands on its Consumer Key Secret? You bet your bottom dollar there will be. Many. Once they have that, they can spam the living shits out of Twitter, and there is nothing, NOTHING, Twitter can do to stop it. Twitter or anyone else. This is the same basic issue as that faced by DRM manufacturers: It is fundamentally impossible for me to allow you to cryptographically authenticate yourself without providing the necessary crypto keys and algorithms to you in some form. Once I have done so, there is no way to prevent you from using them in ways other than those which I intended. -- Dave Sherohman
Re: [twitter-dev] Re: A New API For Browserless Apps?
On Fri, Dec 11, 2009 at 01:27:48AM -0600, Abraham Williams wrote: On Fri, Dec 11, 2009 at 00:53, Dave Sherohman d...@fishtwits.com wrote: - If oauth is the only allowed authentication method, a rogue app would not be able to gain full access to your account. Perhaps most importantly, it would not be capable of changing your password and locking you out. Currently this is not true. http://code.google.com/p/twitter-api/issues/detail?id=1012 Hrm. Didn't know about that one. In that case, amend my quoted statement to replace both occurrences of would not with should not. :p -- Dave Sherohman
Re: [twitter-dev] Re: How to detect 'Duplicate text' failure to send tweet
On Tue, Dec 08, 2009 at 10:08:14AM -0800, Stuart Smith wrote: Hi Abraham, so I should keep a list of all of the tweets I send and check the id of each and every status I send to see if it matches a previously sent id? That's quite a lot of additional processing. Not necessarily... What I've been doing is just keeping a record of the highest status ID my application has seen and checking whether the returned ID is larger than that highest-seen ID or not. Very quick, simple, and effective. Is this what I have heard called 'recurring tweets'? If so, I read somewhere that they are going to be allowed to re-occur at some point soon - is this correct? Related, but distinct. What you've described is repeated tweets sent manually for a game, probably in short bursts. The recurring tweets (mis-)feature offered by some sites is automatically tweet this text once every three hours for the next month. Either way, though, it sounds pretty spammy to me (would your users' friends really want to see their timelines clogged up with N, N, E, S...?) and I'm all for Twitter doing what they can to suppress it. It just would have been nice if rejected updates had been reported as errors (which they are) from square one. -- Dave Sherohman
Re: [twitter-dev] Twitter Photo service for oauth clients
On Sat, Dec 05, 2009 at 10:00:30PM -0800, Michael Steuer wrote: Are any of you aware of a photo service that allows picture uploads from my oauth based Twitter client? The Api for all the usual suspects (twitpic, yfrog, tweetphoto, etc) requires passing of the user's Twitter username and password, but being an oauth client, I don't have the user's password available to me. How are other oauth based clients dealing with this? It seems my only option is building my own, but I'd rather hook into an existing service. You're going to have to roll your own, I expect. The Twitter API doesn't currently support OAuth delegation, so a user wanting to use your site to upload a photo to one third-party site and tweet about it would have to authenticate twice, once from your site to the photo site (to do the upload) and a second time from your site to Twitter (to send the tweet, since it's not possible for the photo site to use the credentials from their login on your site to tweet on their behalf). Offhand, I do know of one site which takes OAuth logins and accepts photo uploads, but: a) It's a fishing niche site, so non-fishing photos wouldn't be entirely welcome there, b) it does not provide an API, OAuth-based or otherwise, and c) I had to write it myself. -- Dave Sherohman
Re: [twitter-dev] Re: How does twitter recognize @username in tweets?
On Thu, Dec 03, 2009 at 03:37:51AM -0800, yegle wrote: But I still want to know what regex twitter is used to recognize @username , because the regex used to recognize @username in dabr doesn't work exactly same as twitter.com. You mean the list that appears when you click the @yegle link on your twitter.com homepage? I'm pretty sure that's not a regex at all. Try the statuses/mentions API method (http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-statuses-mentions) to retrieve that list. If you're implementing your own search through a local database of tweets, my first thought to emulate this would be to check the tweets' in_reply_to_user_id properties. -- Dave Sherohman
Re: [twitter-dev] Hashtag widget
That sounds very similar to the original design of fishtwits.com (and I submitted Twitter::TagGrep to CPAN, which handles the filtering by hashtag part in Perl), although we eventually decided not to worry about hashtags and just filter on any occurence of the words of interest, tagged or not. I haven't put it into widget form and I did it in Perl rather than PHP, but feel free to contact me if you think I might be able to help with what you're doing. On Thu, Dec 03, 2009 at 04:15:37AM -0800, BenR wrote: Hi all just wondering if anyone has done anything with hashtags. I'd like to be able to display only posts from selected accounts using a specific hashtag. So I need to have an array of accounts and search these posts for a hashtag and if the hashtag is used pull the post into a widget. Has anyone done this using PHP or know of anything like it? Thanks all -- Dave Sherohman
Re: [twitter-dev] Re: Track streaming : how to match tweets?
On Thu, Dec 03, 2009 at 03:12:05PM -0800, Julien wrote:
Well, then I'd need some help with that...
Again, it's easy with single search keywords, but I haven't found a
solution for combined searches like twitter+stream or photo+Paris...
because I would have to compare each combination of tokens in the
tweet...
Can someone give more details.
I don't mean to be flogging my site today, but take a look at
http://fishtwits.com for the results I'm producing (just click the logo
at the top of the page to view the full site without logging in): Any
tweets from users followed by FishTwits are scanned for fishing-related
terms and all such terms found in the tweet are displayed below it. At
this moment, for instance, the first displayed tweet shows matches for
both Fly Fishing and Sole.
This is accomplished with the following Perl code (edited to remove
parts which aren't directly relevant):
sub load_from_text {
my ($class, $text) = @_;
unless($topic_regex) {
require Regexp::Assemble;
my $ra = Regexp::Assemble-new(
chomp = 0,
anchor_word_begin = 1,
anchor_word_end = 1,
);
for my $topic (@topic_list) {
$ra-add(lc $topic);
}
$topic_regex = $ra-re;
}
$text = lc $text;
my @topics = $text =~ /$topic_regex/g;
return sort @topics;
}
It first uses Regexp::Assemble to build a $topic_regex[1] which will
match any of the words/phrases found in the topic table, then does a
global match of $text (the body of the tweet being examined) against
$topic_regex, capturing all matches into the array @topics, which is
then sorted and returned to the caller.
After the match is performed, @topics contains every search term which
is matched, no matter how many there may be, which should fill your
requirement for combined searches, unless I'm misunderstanding it.
If you mean you would want that Fly Fishing, Sole tweet to return
three hits rather than two (Fly Fishing, Sole, Fly Fishing+Sole),
that's easy enough to create from @topics, just generate every
permutation of the terms which the individual tweet matched.
[1] If you're only dealing with 10 or so keywords, you'd probably be
just as well off building the regex by hand. The main reason I'm using
Regexp::Assemble to do it on the fly is because manually creating and
then maintaining a regex that will efficiently match any of 1300 terms
would be a nightmare.
--
Dave Sherohman
Re: [twitter-dev] Perl script for retrieving twits
On Tue, Dec 01, 2009 at 06:20:05AM -0800, Oreste wrote:
I'm using Net::Twitter module which is working nicely. I'm getting
back a lot of variables, I need to grab just those variables giving me
the message and date. Could you help me to find them out?
This is the script I'm using:
use Net::Twitter;
use Data::Dumper;
my $nt = Net::Twitter-new(
traits = [qw/API::REST/],
username = 'user',
password = 'pass'
);
print Dumper $nt;
Your $nt is just the interface to the API. If you want to, say, get the
latest status updates to your/your friends' accounts, you need to use
that interface to retrieve them:
my $timeline = $nt-friends_timeline;
for my $tweet (@$timeline) {
print $tweet-{created_at} . ': ' . $tweet-{text};
}
(Code is untested, but should work. Note that the maintainer of
Net::Twitter was talking a couple months ago about wrapping the
retrieved messages up into objects, so $tweet may no longer be a bare
hashref; I just haven't needed to update my direct Twitter-interfacing
functions since then so I haven't stayed on top of it. Check the
Net::Twitter documentation for full details.)
--
Dave Sherohman
Re: [twitter-dev] How does twitter recognize @username in tweets?
On Tue, Dec 01, 2009 at 07:14:25PM -0800, yegle wrote: I have a saved search which is yegle -...@yegle to track tweets which intends to mention me, I found some tweets with format described below can be found using yegle -...@yegle but also appear in my reply_timeline: Here is sometext and then w...@yegle (no space before the @ ) So my question is, what regex does twitter use to recognize @username in tweets? You're imagining that this is being handled in a more complex (more intelligent?) way than it actually is. The search function operates on complete words only, with no special-case handling for @usernames. w...@yegle doesn't match @yegle for the same reason it doesn't match it: they're substrings embedded within the word, not the complete word. -- Dave Sherohman
Re: [twitter-dev] Call for action #StopBritneyBots
On Mon, Nov 30, 2009 at 04:23:40PM -0500, TJ Luoma wrote: On Mon, Nov 30, 2009 at 4:19 PM, M. Edward (Ed) Borasky zzn...@gmail.com wrote: Twitter, what say you? Developer community, what say you? Twitter, Inc. can't even keep up with porn spammers reported manually using the Report As Spam links, what makes you think they would be able to keep up with an automated version? As Ed demonstrated in his original message, this particular class of spambots can be detected and auto-blocked quite easily. With the information he provided, I (and I expect many others on this list) could create an anti-BritneyBot-bot to do so in very short order, if not for the potential legal/TOS issues which could arise from doing so without Twitter's official sanction. It's not a question of keeping up with them; Twitter could use Ed's suggested technique to shut them all down en masse at the cost of less than one day of a single employee's time (and they may have other techniques they could use which would be more effective and/or even quicker to implement). It's a question of will and of policy. -- Dave Sherohman
Re: [twitter-dev] Re: Retweets - where are they placed on timeline
On Tue, Nov 17, 2009 at 12:37:32PM -0800, Dewald Pretorius wrote: This collapsing behavior is far from ideal and will cause people with busy timelines to completely miss retweets. I disagree completely. If you have 57 retweets of the same status in your timeline, then all this duplicated, redundant noise forces fresh, new content off the end prematurely. It gets bad enough at times that I'm forced to tell TweetDeck to filter out everything containing the text RT @. One of the first features that got me thinking about building a custom Twitter client was that I wanted to be able to identify and filter out duplicate updates so that I would see each status update *once*, no matter how many people were repeating it. Thankfully, Twitter is now (or will shortly be) providing that function automatically. (Although, if I were to have done it, I would go a step further and suppress *all* retweets if the original had appeared in my timeline.) With old-style retweets, if I saw ten retweets of the same thing, I knew to check it out because obviously a lot of people felt it was something worth sharing with their followers. And if I saw them, I would think Oh, good god... I wasn't interested in it the first nine times, I'm not going to be interested this time either. How long do I have to wait before people just shut up about that already? In my irrelevant opinion, the new retweet feature is trying to fix something that was not broken. As another response mentioned, clients have the ability to un-collapse retweets, so those of us who view constant retweets of the same thing as worthless noise can get our timelines uncluttered and those of you who use them as a means to judge social approval and/or importance have the tools to do that as well. Give it a couple weeks, and I bet someone will be working on a way to do that which works even better than the old-style retweets, perhaps based on preiodically polling the retweets timeline and presenting its contents sorted by most retweets in the last hour/day. This change sets the stage for serving both of us better. -- Dave Sherohman
[twitter-dev] Re: PHP/cURL statuses/update results in 200 OK, but no actual update
On Fri, Nov 13, 2009 at 01:56:40PM -0800, EvilJordan wrote: Very simple PHP/cURL script to update a status. With the curl output set to verbose, I see the connection being made and a response code of 200 OK. Unfortunately, my status isn't actually updating and I can't figure out why. The status message is urlencoded, basic authentication is successful, and using curl from the command line works just fine. For the last month or so, Twitter has been silently rejecting status updates which either exceed 140 characters or are duplicates of any previous status for that user (not just the most recent). When this occurs, the response is a 200 OK and carries the user's most recent successful status as its body. You can easily avoid the silent failures for exceeding 140 characters by double-checking the length of the text before sending it, but predicting whether an update will fail due to duplication is not such a simple task... Twitter has stated that providing a clear indication of when this has occurred (I don't recall whether it was stated whether it will be an error code per se or not) is on the short list, but no ETA on a fix has been given. Until this is resolved, you basically have two options for trying to detect it: 1) Keep track of the highest status ID you've encountered. When a status update is submitted, check the status ID in the response. If the update succeeded, the new ID will be higher than any old ID. If it failed, you'll get an older status (with an older, lower ID) instead. 2) Compare the text that you sent with the text you got back. If they're different, it failed - or Twitter did automatic URL shortening or modified the text in some other manner. If the new status is an exact duplicate of the previous status, this would also falsely lead you to believe that it had succeeded when it actually failed; depending on your application, this may or may not be relevant. Obviously, I consider method 1 to be the more reliable (although there are certain race conditions in which it would fail, too), but it does require you to keep that latest seen ID around and relatively up-to-date. -- Dave Sherohman
[twitter-dev] Re: the new retweet feature might break my app
On Thu, Nov 12, 2009 at 08:00:56AM -0800, Cameron Kaiser wrote: Speaking for TTYtter only, while I'll support receiving retweets, I am unhappy with the API as it currently exists and retweets received will be canonized into the older format (and retweets sent will be done programmatically in the older fashion instead of through the retweet methods). I suspect there are other app authors who will also do something similar. I haven't looked closely at the RT API (it's not currently relevant to FishTwits, so I figure I'll let it stabilize before concerning myself with it), but would you mind sharing your issues with it, either here or off-list if you think that would be more appropriate? -- Dave Sherohman
[twitter-dev] Re: Oauth API not updating statuses consistently
On Mon, Nov 02, 2009 at 09:38:48PM -0800, Enlai wrote: Sorry if this is a dumb question but I can't find an answer online. This is happening right now so I thought I'd see if something is up with the API. Sometimes, I get no response from the status update post and other times, I get a response with a successful post result. However, the text in the post shows the previous successful post rather than the one I just posted. What does this mean? Has some rate been tripped? This has been noted by a few of us since mid-October. Twitter changed the API behaviour such that a status update will fail silently and return the user's most recent prior status in two (known) cases: - If the text of the update exceeds 140 characters. (This is a change from the documented behaviour of truncating the text to 140 characters.) - If the text of the update is a duplicate of *any* previous status update within an as-yet-unknown time period. I know from my own experience that the time period is at least an hour and I suspect that it it much longer, perhaps even unlimited. (This is a change from the documented behaviour of only rejecting duplicates when consecutive.) There has been a comment made here on the list to the effect that Twitter intends to respond with an error code in these cases rather than failing silently, but no indication was given regarding an expected timeframe. Until Twitter does get around to telling us when updates have been rejected, two basic approaches have been suggested for detecting these silent failures: - Store the highest status ID which has been seen by your application. When you submit an update, check the ID of the returned status. If the returned ID is less than the stored ID, you know the update was rejected and an old status was returned. - Compare the text of the sent status to the text of the returned status to see if they match. I prefer the ID-based method, as the text-based method is subject to both false positives (if Twitter applies bit.ly shortening, you'll think it failed unless you remember to ignore URLs when comparing the text) and false negatives (if the update is a duplicate of the user's most- recent status, the text will match even though the update failed), but YMMV. -- Dave Sherohman
[twitter-dev] Re: Twitter Lists Need Optional Hashtag binding with list subscription
This sounds like what I've already done (for fishing) at http://FishTwits.com ... With my codebase, creating similar sites for sports, food, tech, whatever would mostly just be a matter of getting a new layout designed and new lists of keywords made up. Feel free to contact me off-list if you'd like to talk about doing so. On Thu, Oct 29, 2009 at 07:14:58PM -0400, Chad Etzel wrote: This sounds like a 3rd party API application opportunity to me... -Chad On Thu, Oct 29, 2009 at 7:11 PM, leonspencer spencer_l...@yahoo.com wrote: The new lists are a great way reduce the clutter in timeline. However, I immediately notice management problems with the lists where tweets are duplicate in timeline and list for tweets that have nothing to do with the list. For example, let's say I have a sports list: http://twitter.com/#/list/leonspencer/sports I add userA that tweets about sports. But userA also tweets about food. And being a friend, userA will tweet about personal stuff. If a user tweets about sports, dining, and personal stuff, ALL of these tweets are shown under the sports list as well as my timeline. Furthermore, if I create a second list for food/dining and add userA, ALL of the tweets regardless of food or sports will display under both lists. So what would be great is for Twitter to allow the optional specification of one or more hashtags when associating an account with a list, which would indicate tweets from that account should only appear under this list IF these hashtags are part of the tweet. Leon -- Dave Sherohman
[twitter-dev] Re: Automated Tweets
On Thu, Oct 29, 2009 at 12:48:02PM +0700, Dwi Sasongko Supriyadi wrote: On Wed, Oct 28, 2009 at 9:40 PM, Jai jaishank...@gmail.com wrote: What is the criteria for dupliate tweets? then how the polling applications works? they may have several tweets with similar text? Duplicate tweets = consecutively, exactly same tweet. tweet1 = foo; tweet2 = bar; rv1 = send_tweet(tweet1); rv2 = send_tweet(tweet1); rv3 = send_tweet(tweet2); above sequence, only rv1 and rv3 are succeed. That is not the full definition, or at least it wasn't around two weeks ago when I first noticed it. Once tweet1 has been sent, any attempt to send tweet1 again within the next hour (probably longer, but I have not attempted to determine the upper bound) will fail, regardless of how many additional updates have been sent during that time. rv1 = send_tweet(tweet1); # OK rv2 = send_tweet(tweet2); # OK ... rv100 = send_tweet(tweet100); # OK rv101 = send_tweet(tweet1); # Fails, if within some unknown time limit -- Dave Sherohman
[twitter-dev] Re: API 140 character truncation change?
On Fri, Oct 23, 2009 at 04:20:36PM -0700, Dewald Pretorius wrote: Instead of all of us having to do fancy tap-dances, the proper solution is for Twitter to issue an error response when a sent tweet is rejected for whatever reason. Yep, exactly. That's my big gripe about the whole thing. Failing silently is *never* the right answer in an API or other library code. -- Dave Sherohman
[twitter-dev] Re: New behaviour for statuses/update API call for 141+ char sized messages and duplicates?
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/cd95ce07be341223/66c66de585383868#66c66de585383868 http://groups.google.com/group/twitter-development-talk/browse_thread/thread/3d6a727892710d5e# These are deliberate changes on Twitter's part, so they are not bugs. Whether they are features depends on who you ask... As it currently stands, you must either check the returned status ID to see if it's higher than previous IDs or compare the submitted status to the returned status (ignoring URLs) to determine whether the update was actually successful or if it was silently rejected by Twitter. Hopefully, in the (very near) future, Twitter will start providing some indication in the response that will make it simple and reliable to determine when an update has been rejected without requiring app developers to try to figure that out on our own, but, so far as I am aware, Twitter has not yet made any statement regarding this. On Fri, Oct 23, 2009 at 08:53:16AM -0700, Kevin Menard wrote: Hi, I'm seeing the same thing that Ole is. Twitter is not truncating the status, but rather returning the last correctly updated status. -- Kevin On Oct 16, 4:58 am, janole s...@mobileways.de wrote: According to my tests, messages will not be truncated anymore! Instead, you will get the most recentstatusupdateas a reply. Is this a bug or feature? Also, it seems as if the API now checks for duplicates in your backlog of statuses and not just you most recent tweet. Previously, only the last tweet was checked: - Last tweet test - Send new tweet with status=test will return the oldstatus(with the old status_id) but if you had something like this: Last tweet Hello, world. Second last tweet test Then you were able to create a new tweet with status=test! This is not possible at the moment. Bug or feature? I'm getting a lot of complaints from my Twitter client users who apparently experience both of these new behaviours or bugs (long tweets fail, duplicates fail.) Ole @ mobileways.de On Twitter:http://twitter.com/janole On Oct 15, 8:26 pm, Josh Roesslein jroessl...@gmail.com wrote: If you send a message longer than 140 twitter will truncate it and set the truncate value on thestatusto True. For duplicates it will just ignore thestatus. Josh On Thu, Oct 15, 2009 at 1:20 PM, janole s...@mobileways.de wrote: Hi, I just figured out that when calling statuses/updatewith a text longer than 140 chars, the reply of that API call will be 200 OK with the laststatusof the user. Wouldn't it be better to return some sort of error message? The same seems to be happening when sending a duplicate tweet. Ole -- Jan Ole Suhr s...@mobileways.de On Twitter:http://twitter.com/janole -- Dave Sherohman
[twitter-dev] Re: API 140 character truncation change?
On Fri, Oct 23, 2009 at 02:24:53PM -0700, AJ Chen wrote: I noticed this behavior a long time ago (may be a month) and reported the problem on this list, but it did not get any response from the api team. I thought it was a bug, but just realized yesterday that the api probably ignores 140+ chars status update intentionally. but' I'm not sure this is the policy or temporary tactic to reduce workload on api. it would be good that api team can clasify on this issue. to check if this happens or not, you can compare the status sent to api and the status returned from api in your application code. Note that duplicate statuses are silently failing in the same manner as those over 140 characters. Aside from the already-mentioned issue of Twitter doing automatic bit.ly shortening on URLs, comparing the submitted status to the received status will also produce incorrect results when the submitted status is rejected for being identical to the user's most-recent previous status. Depending on your application, failure in this scenario may or may not be relevant. -- Dave Sherohman
[twitter-dev] Re: Comparing Posted Tweet to Current Status in PHP
On Wed, Oct 21, 2009 at 11:22:44AM -0700, JB wrote: Does anyone have a way to compare these two strings in PHP? I have tried 3 different ways of comparing strings and they all fail. Even more so recently. Looking for a solid way to see if a tweet was accepted by the API or not. Remember that every new status will be assigned a higher ID than any previously-existing ID. Therefore, when you get/store the user's current statusprior to sending the update (which you must be doing if you have its text), you can also remember its ID. Comparing the ID against the returned ID from the attempted update will tell you whether it was accepted or not. Alternately, if this is the only reason you're checking/storing the user's previous status, you may be able to save yourself a few API calls by saving the last seen ID globally instead of keeping it updated for each user individually. Using the ID rather than the status text should be more reliable, not only because numeric comparison is much more straightforward than string comparison, but also because the IDs are guaranteed to always be unique. This method could fail if the user updates his status using another application, then comes back and submits an update from yours before yours checks in with Twitter and is made aware of the new update, but there's really no way to reliably detect whether the update from your app has been rejected in that scenario. Which is why Twitter really needs to stop rejecting updates silently and add some kind of indicator (a non-200 HTTP status, an extra flag in the response payload, whatever) to tell us when the update had been rejected (and, ideally, why). -- Dave Sherohman
[twitter-dev] Re: API 140 character truncation change?
On Tue, Oct 20, 2009 at 07:37:03AM -0700, James Tymann wrote: Has anyone else noticed a change in the way that the 140 character limit is enforced via the API? I noticed a change sometime between the 13th and the 16th that is now causing all my 140+ character posts to be rejected by the API. Also a side note is that the api is not returning errors, they return proper responses however they are the proper response for the current status of the account, not the new status that was just attempted to be posted. My users first reported issues arising from this on the 15th, although I didn't identify the cause until the 17th, at which point I asked about it in #Net::Twitter and Marc Mims brought the question here under the subject line Bug? Updates 140 characters return success with prior update payload. See the discussion under that thread for more on it, but the overall upshot is: - This is an intentional (if poorly-announced) change, not a bug. - Status updates are known to be getting silently rejected in this manner both due to exceeding 140 characters and due to violation of the expanded no duplicates policy. - Twitter has not stated whether there are any additional circumstances beyond those two cases in which updates will be silently rejected. - Twitter has not stated any plans regarding adding an indicator for when a 200 OK status update has, in fact, been rejected. I am attempting to compensate for this change by checking the returned status ID against the previous highest-seen ID to determine whether the status returned with the 200 OK response is a new one or the user's pre-existing status. This seems to work, but does not indicate the reason for the silent failure, so I can't report the cause to my users. Andy Freeman has mentioned that, in the case of rejection due to duplication, this is also unsatisfactory in that it does not allow him to identify the original status which was duplicated. -- Dave Sherohman
[twitter-dev] Re: Bug? Updates 140 characters return success with prior update payload
On Sat, Oct 17, 2009 at 10:41 AM, Marc Mims marc.m...@gmail.com wrote: Updates longer than 140 characters should be forcibly truncated according to the documentation. Instead, the update call returns with a 200 status and the payload contains the prior update. Has there been a change to the API or is this a bug. On Sat, Oct 17, 2009 at 05:15:42PM -0500, Josh Roesslein wrote: This is a change in the API confirmed by one of twitter's API members. The docs should be updated soon. In that case, is there any change (planned or current) which will indicate when this has happened, or if the update has been rejected for any other reason? Failing silently does not seem appropriate, particularly when the failure returns the user's previous status. -- Dave Sherohman
