[twitter-dev] Re: link wrapping on the API

[Gil]

Hi,

Is it possible to get those links when using the user_timeline REST
API:
http://twitter.com/status/user_timeline/{user}.json?count=5&callback=?

The returned result doesn't give you any links - the http/https and
the links to other twitter accounts are just as plain text.

I need a simple way to identify those links and transform them into
html anchors, without having to start parsing the text and searching
for stuff like '@'.


On Jun 9, 1:57 am, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a best practices
> document on how to use these t.co links.
>
> what's the timeline?  "soon" we'll enable this on @twitterapi, @rsarver,
> @raffi, and a few other test accounts so you all have live data to play
> with.  on the timescale of weeks (to potentially a mon

[twitter-dev] Re: link wrapping on the API

[Rich]

I've noticed on Raffi's timeline a field called expanded_url not
display_url for a t.co entity.  Is the expanded_url element to be
treated differently, or is it as it says it is, the expanded version
of any url and not necessarily just t.co urls?

On Jun 10, 3:38 pm, Rich  wrote:
> Another question will you HAVE to use include_entities=true to see the
> display_url or will it always be included?
>
> On Jun 10, 4:21 am, ASK  wrote:
>
> > How disruptive - and not in the "good way", for the most part.
>
> > For example, I've recently been developing a link shortening platform
> > with some unique aspects (similar to Twitter annotations). Here is a
> > mashup that leverages my platform in conjunction with 
> > Twitter:http://mvtweets.com/tweetmap. Just like Twitter intends, I parse
> > shortened URL's to display a truncated destination domain (or a
> > TwitPic thumbnail, or a YouTube embed), but the href in the anchor tag
> > is the shortened mv2.me link, so the click-through can be tracked.
>
> > Part of the data feeding this map-mashup comes from the @mvtweets
> > Twitter account. Another part comes from the mv2.me platform and API,
> > which provides the calendar and GPS metadata. Now I'll have to rewrite
> > it to accommodate the new link wrapping scenario. Will we be seeing
> > changes to the data Twitter API returns, so that the task of adjusting
> > our code is made easier? And will I have to make the hrefs all t.co's?
> > The line is very unclear to me - I'm using a whole bunch of API's
> > mashed together, 3rd party and my own - why should I have to make
> > analyzing my map's click-throughs more difficult for myself?
>
> > I fear innovations such as the one I have shared with you above will
> > be fewer and farther in between due to such policy changes. We will
> > soon be seeing how fragile the Twitter ecosystem can be.
>
> > (on a side note, regarding my map: isn't it funny how Chrome, Google's
> > browser, is the one technology that can't seem to handle Google's
> > YouTube technology ebmedded inside Google's map technology? sorry
> > Chrome users!)
>
> > On Jun 9, 10:24 pm, John Meyer  wrote:
>
> > > On 6/9/2010 7:00 PM, Bernd Stramm wrote:
>
> > > > On Wed, 09 Jun 2010 17:13:04 -0700
> > > > "M. Edward (Ed) Borasky"  wrote:
>
> > > >> Quoting Ken:
>
> > > >>> Not exactly spyware, but deceptive. Don't expect the public to
> > > >>> appreciate this.
>
> > > >> How is this deceptive? Who is being deceived, and how?
>
> > > > How? There is text that is marked as a link, for example
> > > > "http://nasa.gov";, and it does not go to nasa.gov.
>
> > > > If a user clicks on the link saying nasa.gov, it  goes to t.co,
> > > > which does business with a third party, not telling the user anything
> > > > about it.
>
> > > > How is that *not* deceptive?
>
> > > As long as the terms are clearly laid out and Twitter is open about
> > > where the user is being sent I see no problem with it in terms of
> > > openness.  However, what I am wondering is why Twitter would feel the
> > > need to wrap other URL shorteners.  Won't that increase the time needed
> > > to get to the final destination?

[twitter-dev] Re: link wrapping on the API

[Rich]

Another question will you HAVE to use include_entities=true to see the
display_url or will it always be included?

On Jun 10, 4:21 am, ASK  wrote:
> How disruptive - and not in the "good way", for the most part.
>
> For example, I've recently been developing a link shortening platform
> with some unique aspects (similar to Twitter annotations). Here is a
> mashup that leverages my platform in conjunction with 
> Twitter:http://mvtweets.com/tweetmap. Just like Twitter intends, I parse
> shortened URL's to display a truncated destination domain (or a
> TwitPic thumbnail, or a YouTube embed), but the href in the anchor tag
> is the shortened mv2.me link, so the click-through can be tracked.
>
> Part of the data feeding this map-mashup comes from the @mvtweets
> Twitter account. Another part comes from the mv2.me platform and API,
> which provides the calendar and GPS metadata. Now I'll have to rewrite
> it to accommodate the new link wrapping scenario. Will we be seeing
> changes to the data Twitter API returns, so that the task of adjusting
> our code is made easier? And will I have to make the hrefs all t.co's?
> The line is very unclear to me - I'm using a whole bunch of API's
> mashed together, 3rd party and my own - why should I have to make
> analyzing my map's click-throughs more difficult for myself?
>
> I fear innovations such as the one I have shared with you above will
> be fewer and farther in between due to such policy changes. We will
> soon be seeing how fragile the Twitter ecosystem can be.
>
> (on a side note, regarding my map: isn't it funny how Chrome, Google's
> browser, is the one technology that can't seem to handle Google's
> YouTube technology ebmedded inside Google's map technology? sorry
> Chrome users!)
>
> On Jun 9, 10:24 pm, John Meyer  wrote:
>
> > On 6/9/2010 7:00 PM, Bernd Stramm wrote:
>
> > > On Wed, 09 Jun 2010 17:13:04 -0700
> > > "M. Edward (Ed) Borasky"  wrote:
>
> > >> Quoting Ken:
>
> > >>> Not exactly spyware, but deceptive. Don't expect the public to
> > >>> appreciate this.
>
> > >> How is this deceptive? Who is being deceived, and how?
>
> > > How? There is text that is marked as a link, for example
> > > "http://nasa.gov";, and it does not go to nasa.gov.
>
> > > If a user clicks on the link saying nasa.gov, it  goes to t.co,
> > > which does business with a third party, not telling the user anything
> > > about it.
>
> > > How is that *not* deceptive?
>
> > As long as the terms are clearly laid out and Twitter is open about
> > where the user is being sent I see no problem with it in terms of
> > openness.  However, what I am wondering is why Twitter would feel the
> > need to wrap other URL shorteners.  Won't that increase the time needed
> > to get to the final destination?

[twitter-dev] Re: link wrapping on the API

[ASK]

How disruptive - and not in the "good way", for the most part.

For example, I've recently been developing a link shortening platform
with some unique aspects (similar to Twitter annotations). Here is a
mashup that leverages my platform in conjunction with Twitter:
http://mvtweets.com/tweetmap. Just like Twitter intends, I parse
shortened URL's to display a truncated destination domain (or a
TwitPic thumbnail, or a YouTube embed), but the href in the anchor tag
is the shortened mv2.me link, so the click-through can be tracked.

Part of the data feeding this map-mashup comes from the @mvtweets
Twitter account. Another part comes from the mv2.me platform and API,
which provides the calendar and GPS metadata. Now I'll have to rewrite
it to accommodate the new link wrapping scenario. Will we be seeing
changes to the data Twitter API returns, so that the task of adjusting
our code is made easier? And will I have to make the hrefs all t.co's?
The line is very unclear to me - I'm using a whole bunch of API's
mashed together, 3rd party and my own - why should I have to make
analyzing my map's click-throughs more difficult for myself?

I fear innovations such as the one I have shared with you above will
be fewer and farther in between due to such policy changes. We will
soon be seeing how fragile the Twitter ecosystem can be.

(on a side note, regarding my map: isn't it funny how Chrome, Google's
browser, is the one technology that can't seem to handle Google's
YouTube technology ebmedded inside Google's map technology? sorry
Chrome users!)

On Jun 9, 10:24 pm, John Meyer  wrote:
> On 6/9/2010 7:00 PM, Bernd Stramm wrote:
>
>
>
>
>
> > On Wed, 09 Jun 2010 17:13:04 -0700
> > "M. Edward (Ed) Borasky"  wrote:
>
> >> Quoting Ken:
>
> >>> Not exactly spyware, but deceptive. Don't expect the public to
> >>> appreciate this.
>
> >> How is this deceptive? Who is being deceived, and how?
>
> > How? There is text that is marked as a link, for example
> > "http://nasa.gov";, and it does not go to nasa.gov.
>
> > If a user clicks on the link saying nasa.gov, it  goes to t.co,
> > which does business with a third party, not telling the user anything
> > about it.
>
> > How is that *not* deceptive?
>
> As long as the terms are clearly laid out and Twitter is open about
> where the user is being sent I see no problem with it in terms of
> openness.  However, what I am wondering is why Twitter would feel the
> need to wrap other URL shorteners.  Won't that increase the time needed
> to get to the final destination?

Re: [twitter-dev] Re: link wrapping on the API

[John Meyer]

On 6/9/2010 7:00 PM, Bernd Stramm wrote:

On Wed, 09 Jun 2010 17:13:04 -0700
"M. Edward (Ed) Borasky"  wrote:


Quoting Ken:


Not exactly spyware, but deceptive. Don't expect the public to
appreciate this.


How is this deceptive? Who is being deceived, and how?


How? There is text that is marked as a link, for example
"http://nasa.gov";, and it does not go to nasa.gov.

If a user clicks on the link saying nasa.gov, it  goes to t.co,
which does business with a third party, not telling the user anything
about it.

How is that *not* deceptive?



As long as the terms are clearly laid out and Twitter is open about 
where the user is being sent I see no problem with it in terms of 
openness.  However, what I am wondering is why Twitter would feel the 
need to wrap other URL shorteners.  Won't that increase the time needed 
to get to the final destination?

Re: [twitter-dev] Re: link wrapping on the API

[Bernd Stramm]

On Wed, 09 Jun 2010 17:13:04 -0700
"M. Edward (Ed) Borasky"  wrote:

> Quoting Ken :
> 
> > Not exactly spyware, but deceptive. Don't expect the public to
> > appreciate this.
> 
> How is this deceptive? Who is being deceived, and how?

How? There is text that is marked as a link, for example
"http://nasa.gov";, and it does not go to nasa.gov. 

If a user clicks on the link saying nasa.gov, it  goes to t.co,
which does business with a third party, not telling the user anything
about it.

How is that *not* deceptive?

> >
> > On Jun 9, 9:45 pm, Bernd Stramm  wrote:
> >
> >> If an application wants to provide the original intent of the
> >> user, it is forced (by ToS), to present a link that doesn't go to
> >> where it says it does. That is problematic, the application acts
> >> as spyware.
> >>
> >> --
> >> Bernd Stramm
> >> 
> >
> 
> 
> 



-- 
Bernd Stramm

Re: [twitter-dev] Re: link wrapping on the API

[M. Edward (Ed) Borasky]

Quoting Ken :


Not exactly spyware, but deceptive. Don't expect the public to
appreciate this.


How is this deceptive? Who is being deceived, and how?


On Jun 9, 9:45 pm, Bernd Stramm  wrote:


If an application wants to provide the original intent of the user, it
is forced (by ToS), to present a link that doesn't go to where it says
it does. That is problematic, the application acts as spyware.

--
Bernd Stramm

[twitter-dev] Re: link wrapping on the API

[Ken]

Not exactly spyware, but deceptive. Don't expect the public to
appreciate this.

On Jun 9, 9:45 pm, Bernd Stramm  wrote:

> If an application wants to provide the original intent of the user, it
> is forced (by ToS), to present a link that doesn't go to where it says
> it does. That is problematic, the application acts as spyware.
>
> --
> Bernd Stramm
> 

Re: [twitter-dev] Re: link wrapping on the API

[M. Edward (Ed) Borasky]

Quoting Bernd Stramm :


On Wed, 09 Jun 2010 12:22:03 -0700
"M. Edward (Ed) Borasky"  wrote:


Quoting Ron B :

> I just had one of those *rough edges* brought to my attention that I
> think may also have already been alluded to on this thread.  Some
> users use long URLs as a portion or even their entire tweet.  This
> tweet technique is being used significantly in tweets about the Gulf
> oil spill crisis, for example. (i.e.
>   
http://www.grist.org/aritcle/2010-06-08-does-obama-need-more-not-fewer-oilmen-in-his-administration).

> The author of such a tweet fully expects that this message will be
> conveyed in the tweet itself, by virtue of the message conveyed by
> the long URL - not just the URL destination's content.  It doesn't
> sound like this interesting and useful melding of tweet text and
> tweet attachments will be possible any longer after the t.co wrapper
> initiative is implemented.  That would be a real shame...

That particular example looks like a blatant "search engine
optimization" ploy and not the "human-to-human communication" that
is, IIRC, the "Spirit of Twitter". ;-)


Really now, what is wrong with a person expressing themselves by making
human readable links?

If an application wants to provide the original intent of the user, it
is forced (by ToS), to present a link that doesn't go to where it says
it does. That is problematic, the application acts as spyware.

--
Bernd Stramm





I make my WordPress blog posts SEO-friendly by having the URLs  
constructed from the title via the WordPress setting that does that.  
And I use a sitemap plugin and a few other gizmos to alert the  
crawlers when I make a post. But I *tweet* links to the posts via  
bit.ly with the title text, or maybe a quotation from the post.


When Twitter's link shortener is activated *and* I can get Twitter  
analytics, I'll use Twitter's link shortener for links in tweets and  
kick bit.ly to the curb. Hell, if the price is right, I'll get a  
Twitter "commercial account". ;-)

[twitter-dev] Re: link wrapping on the API

[janole]

Hi Raffi,

my Twitter client has an option to post links to del.ico.us, Read-it
Later and Instapaper. Do I have to post the t.co links there or will I
be allowed to use the expanded links?

ole @ mobileways.de / @janole on Twitter ( Gravity Twitter Client for
Symbian )

On 9 Jun., 00:57, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a best practices
> document on how to use these t.co links.
>
> what's the timeline?  "soon" we'll enable this on @twitterapi, @rsarver,
> @raffi, and a few other test accounts so you all have live data to play
> with.  on the timescale of weeks (to potentially a month or two), we'll roll
> this out to everybody.
>
> of course, if there are any questions, just feel free to direct them to
> @twitterapi!
>
> --
> Raffi Krikorian
> Tw

Re: [twitter-dev] Re: link wrapping on the API

[Bernd Stramm]

On Wed, 09 Jun 2010 12:22:03 -0700
"M. Edward (Ed) Borasky"  wrote:

> Quoting Ron B :
> 
> > I just had one of those *rough edges* brought to my attention that I
> > think may also have already been alluded to on this thread.  Some
> > users use long URLs as a portion or even their entire tweet.  This
> > tweet technique is being used significantly in tweets about the Gulf
> > oil spill crisis, for example. (i.e.
> > http://www.grist.org/aritcle/2010-06-08-does-obama-need-more-not-fewer-oilmen-in-his-administration).
> > The author of such a tweet fully expects that this message will be
> > conveyed in the tweet itself, by virtue of the message conveyed by
> > the long URL - not just the URL destination's content.  It doesn't
> > sound like this interesting and useful melding of tweet text and
> > tweet attachments will be possible any longer after the t.co wrapper
> > initiative is implemented.  That would be a real shame...
> 
> That particular example looks like a blatant "search engine  
> optimization" ploy and not the "human-to-human communication" that
> is, IIRC, the "Spirit of Twitter". ;-)

Really now, what is wrong with a person expressing themselves by making
human readable links? 

If an application wants to provide the original intent of the user, it
is forced (by ToS), to present a link that doesn't go to where it says
it does. That is problematic, the application acts as spyware. 

-- 
Bernd Stramm

Re: [twitter-dev] Re: link wrapping on the API

[M. Edward (Ed) Borasky]

Quoting Ron B :


I just had one of those *rough edges* brought to my attention that I
think may also have already been alluded to on this thread.  Some
users use long URLs as a portion or even their entire tweet.  This
tweet technique is being used significantly in tweets about the Gulf
oil spill crisis, for example. (i.e.
http://www.grist.org/aritcle/2010-06-08-does-obama-need-more-not-fewer-oilmen-in-his-administration).
The author of such a tweet fully expects that this message will be
conveyed in the tweet itself, by virtue of the message conveyed by the
long URL - not just the URL destination's content.  It doesn't sound
like this interesting and useful melding of tweet text and tweet
attachments will be possible any longer after the t.co wrapper
initiative is implemented.  That would be a real shame...


That particular example looks like a blatant "search engine  
optimization" ploy and not the "human-to-human communication" that is,  
IIRC, the "Spirit of Twitter". ;-)


Bonus points for correcting the spelling of "article" in the link? ;-)

[twitter-dev] Re: link wrapping on the API

[Ken]

Chris, I am not worried about that or any of this, but agree that it's
unfortunate to lose the choice. And it feels wrong to be obfuscating
links to my own website...

For apps that display tweets, I understand that the t.co link must be
used and not the display link. But what does it mean, "require you to
check t.co and register the click" ? How do we "check" and
"register" ?

On Jun 9, 7:03 pm, Chris Barr  wrote:
> My 2 pence:
>
> The difference with bit.ly is that I choose to use it. If I don't want
> to use it I'm not forced to.
>
> Additionally, what happens if the t.co service goes down? All links
> will be temporarily broken until the service goes back up.
>
> On Jun 9, 4:17 pm, Harshad RJ  wrote:
>
> > On Wed, Jun 9, 2010 at 6:48 PM, Dewald Pretorius  wrote:
>
> > > I don't buy the click tracking privacy argument. Twitter will have no
> > > more insight into clicks than what bit.ly or any other shortening
> > > service has,
>
> > The difference being that the user who clicks the links in Twitter will have
> > most probably logged into Twitter. Thus, Twitter can directly associate a
> > click with a user.
>
> > When clicking on bit.ly shortened URLs it is very very unlikely that the
> > user is logged into bit.ly. That is because only people who shorten URLs
> > need a bit.ly account (which is a very small percentage).
>
> > --
> > Harshad RJhttp://hrj.wikidot.com

[twitter-dev] Re: link wrapping on the API

[Ron B]

I just had one of those *rough edges* brought to my attention that I
think may also have already been alluded to on this thread.  Some
users use long URLs as a portion or even their entire tweet.  This
tweet technique is being used significantly in tweets about the Gulf
oil spill crisis, for example. (i.e.
http://www.grist.org/aritcle/2010-06-08-does-obama-need-more-not-fewer-oilmen-in-his-administration).
The author of such a tweet fully expects that this message will be
conveyed in the tweet itself, by virtue of the message conveyed by the
long URL - not just the URL destination's content.  It doesn't sound
like this interesting and useful melding of tweet text and tweet
attachments will be possible any longer after the t.co wrapper
initiative is implemented.  That would be a real shame...

Good point Ed, about staying in the game.  Fortunately, we can row and
talk at the same time. :)

On Jun 9, 11:47 am, "M. Edward (Ed) Borasky"  wrote:
> Quoting Ron B :
>
> > Users are accustomed to the fact that use of *free* services is
> > entirely *as is* and at their own risk, so none of us should feel we
> > have to protect their privacy or their security beyond this original
> > expectation.  If they don't like the performance, security, or privacy
> > implications of this change, they'll leave Twitter, just like they
> > left Facebook.
>
> Ah, but they *aren't* leaving Facebook! They're continuing to join and  
> share. Facebook and its hundreds of millions of users are continuing  
> to co-evolve. And *when* - not *if* - the FUD brigade realizes it  
> can't take Facebook out, it will turn its energies on Twitter. I hope  
> as a community / ecosystem / whatever, we're ready for that.
>
> > But that is ultimately Twitter's responsibility to
> > control and react to.  Our responsibility is to keep rowing
> > collectively in the same direction, or get off the boat.  :)
>
> Well, to the extent that it's possible, sure. But "groupthink" can  
> kill a product / service too. There are clearly some *technical* rough  
> edges on link wrapping that need to be discussed / debated.

[twitter-dev] Re: link wrapping on the API

[Chris Barr]

My 2 pence:

The difference with bit.ly is that I choose to use it. If I don't want
to use it I'm not forced to.

Additionally, what happens if the t.co service goes down? All links
will be temporarily broken until the service goes back up.


On Jun 9, 4:17 pm, Harshad RJ  wrote:
> On Wed, Jun 9, 2010 at 6:48 PM, Dewald Pretorius  wrote:
>
> > I don't buy the click tracking privacy argument. Twitter will have no
> > more insight into clicks than what bit.ly or any other shortening
> > service has,
>
> The difference being that the user who clicks the links in Twitter will have
> most probably logged into Twitter. Thus, Twitter can directly associate a
> click with a user.
>
> When clicking on bit.ly shortened URLs it is very very unlikely that the
> user is logged into bit.ly. That is because only people who shorten URLs
> need a bit.ly account (which is a very small percentage).
>
> --
> Harshad RJhttp://hrj.wikidot.com

Re: [twitter-dev] Re: link wrapping on the API

[M. Edward (Ed) Borasky]

Quoting Ron B :


Users are accustomed to the fact that use of *free* services is
entirely *as is* and at their own risk, so none of us should feel we
have to protect their privacy or their security beyond this original
expectation.  If they don't like the performance, security, or privacy
implications of this change, they'll leave Twitter, just like they
left Facebook.


Ah, but they *aren't* leaving Facebook! They're continuing to join and  
share. Facebook and its hundreds of millions of users are continuing  
to co-evolve. And *when* - not *if* - the FUD brigade realizes it  
can't take Facebook out, it will turn its energies on Twitter. I hope  
as a community / ecosystem / whatever, we're ready for that.



But that is ultimately Twitter's responsibility to
control and react to.  Our responsibility is to keep rowing
collectively in the same direction, or get off the boat.  :)


Well, to the extent that it's possible, sure. But "groupthink" can  
kill a product / service too. There are clearly some *technical* rough  
edges on link wrapping that need to be discussed / debated.

[twitter-dev] Re: link wrapping on the API

[Ron B]

Users are accustomed to the fact that use of *free* services is
entirely *as is* and at their own risk, so none of us should feel we
have to protect their privacy or their security beyond this original
expectation.  If they don't like the performance, security, or privacy
implications of this change, they'll leave Twitter, just like they
left Facebook.  But that is ultimately Twitter's responsibility to
control and react to.  Our responsibility is to keep rowing
collectively in the same direction, or get off the boat.  :)

On Jun 9, 10:17 am, Harshad RJ  wrote:
> On Wed, Jun 9, 2010 at 6:48 PM, Dewald Pretorius  wrote:
>
> > I don't buy the click tracking privacy argument. Twitter will have no
> > more insight into clicks than what bit.ly or any other shortening
> > service has,
>
> The difference being that the user who clicks the links in Twitter will have
> most probably logged into Twitter. Thus, Twitter can directly associate a
> click with a user.
>
> When clicking on bit.ly shortened URLs it is very very unlikely that the
> user is logged into bit.ly. That is because only people who shorten URLs
> need a bit.ly account (which is a very small percentage).
>
> --
> Harshad RJhttp://hrj.wikidot.com

[twitter-dev] Re: link wrapping on the API

[janole]

I think it pretty much depends on what Twitter will require us to do
when following t.co links. On their own website and in their own
Twitter clients I could imagine that they will track link usage from
people clicking on the link (sooner or later.)

If they don't require third party clients to send any user related
info when accessing t.co links, they only have the IP and possibly the
user agent header.

Cheers,
ole @ mobileways.de / @janole on Twitter

On 9 Jun., 15:30, "Brian Smith"  wrote:
> Dewald Pretorius wrote:
> > StatusNet, blog, etc.). If the privacy argument carried any water, then
> bit.ly
> > would be a far greater privacy threat than Twitter.
>
> I agree, and that's why I (and others) were working on solutions to get
> around bit.ly and other link shorteners. And, Twitter has already developed
> the ultimate solution to that privacy issue, but they just are refusing to
> let API developers use it. That is what is so incredibly frustrating.
>
> Regards,
> Brian

[twitter-dev] Re: link wrapping on the API

[Scott Mills]

I, like others, have problems with this new "service".  My main
concern is this.  In the past, we had an expectation that what we post
to twitter will be shown verbatim.  Some people will want to post a
url and know that it will be shown that way across all sites/apps.
Like it or not, urls have meaning.

A person will no longer be able to post something like "This is/was
the best site ever: http://twitter.com"; and have the same intent/
meaning.

This could lead to other things like twitter replacing bad words in
the interest of "protecting users" or s/coke/pepsi/ for the right
price.

I know this is probably paranoid talk.  I just hope this is not a
shark jump moment for Twitter.

Also...

On Jun 8, 11:18 pm, John Kalucki  wrote:
> Apps that don't update will continue to work, they will just display
> something different than they do now.

This is completely false with one of my apps.  It will completely
break it and it will show nothing.  The app (peephole for iphone)
provides an alternate view of Twitter by mostly showing the photos
people are posting.

As someone else asked, will the search api still return results for
searches like "twitpic"?


Most of the twitter photo services have a documented way to get
thumbnail or full size images from the photo url.
ex. with this url http://twitpic.com/1e10q you know that:
 - http://twitpic.com/show/thumb/1e10q will give you the thumbnail

Most twitter apps don't use the posted photo url.  Does inferring a
link to show a photo to a user violate the TOS?


A time scale of weeks is a very short time frame for apps that go
through Apple's app store.  It seems like you would have a period of
at least a few months where all tweets have the url in the entities
before you start replacing the urls in the text.


Thanks for considering my comments and questions.

Scott Mills
IFWORLD Inc

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

-totally-

the timeline on this is, i think (and i'll check to be sure), sometime like
6-8 weeks.

On Wed, Jun 9, 2010 at 1:45 AM, Rich  wrote:

> Please make sure the timeline for this is LONGER than 2 weeks please,
> some of us have to code this and then wait at least a week to get
> through Apple's approval system.  This is especially important when it
> comes to detecting images, etc.
>
> Richard
>
> On Jun 9, 4:27 am, Raffi Krikorian  wrote:
> > yeah - its definitely case that counting characters will become a bit
> more
> > subtle.  i hope that we can provide a really good and easy way to help
> you
> > all out.  at the very least we are going to update documentation, but i
> know
> > we can do better than that.
> >
> > On Tue, Jun 8, 2010 at 8:17 PM, Andy Matsubara  >wrote:
> >
> >
> >
> >
> >
> > > Raffi wrote:
> > > > related to this: the way the Twitter API counts characters is going
> to
> > > change ever so slightly. our 140
> > > > characters is now going to be defined as 140 characters after link
> > > wrapping. t.co links are of a
> > > > predictable length -- they will always be 20 characters. after we
> make
> > > this live, it will be feasible to
> > > > send in the text for a status that is greater than 140 characters.
> the
> > > rule is after the link wrapping,
> > > > the text transforms to 140 characters or fewer. we'll be using the
> same
> > > logic that is in twitter-text-rb
> > > > to figure out what is a URL.
> > > I guess this change will make frontend text handling more difficult.
> > > Counting characters in a text box must figure out what is a URL. I
> > > hope Twitter will publish JavaScript library for realtime character
> > > counts. I also want APIs to make shortened URL.
> >
> > > Andy Matsubara
> >
> > --
> > Raffi Krikorian
> > Twitter Platform Teamhttp://twitter.com/raffi
>



-- 
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi

[twitter-dev] Re: link wrapping on the API

[Dewald Pretorius]

Rich,

That of course would be the right thing for Twitter to do. In other
words, if they want this change, then they must take the bulk of the
pain, instead of creating a problem and dumping it on the developers
to solve.

On Jun 9, 11:46 am, Rich  wrote:
> My proposal is you need to keep the 140 limit as it is now, if your
> own link shortener makes it over 140 then so be it, that's your
> decision, not ours... but the original posted text was 140.

Re: [twitter-dev] Re: link wrapping on the API

[Harshad RJ]

On Wed, Jun 9, 2010 at 6:48 PM, Dewald Pretorius  wrote:

>
> I don't buy the click tracking privacy argument. Twitter will have no
> more insight into clicks than what bit.ly or any other shortening
> service has,
>

The difference being that the user who clicks the links in Twitter will have
most probably logged into Twitter. Thus, Twitter can directly associate a
click with a user.

When clicking on bit.ly shortened URLs it is very very unlikely that the
user is logged into bit.ly. That is because only people who shorten URLs
need a bit.ly account (which is a very small percentage).

-- 
Harshad RJ
http://hrj.wikidot.com

Re: [twitter-dev] Re: link wrapping on the API

[Thomas Woolway]

As well as everything that others have asked, I'm also concerned about the
following:

1) Does this mean that tweets can effectively be infinite length - say I
write a tweet with 120 chars of text and then
http://www.a-really-really-really-long-url-maybe-with-a-message-encoded.com/in-the-middle-of-it-
that will then get counted as 140 chars after the link has been
wrapped.
When we come to display that, we have to be able to correctly format with
realistically, anything up to 200 char statuses.

2) What will be coming out of the search API? Will those tweets have
t.colinks in them? If so, will we get the additional entity data?

Tom

On Wed, Jun 9, 2010 at 3:46 PM, Rich  wrote:

> I'm a +1 one for the huge can of worms this is going to cause with
> realtime character counting.  Here are my concerns
>
> 1) An app realtime character counts what is entered in a text box and
> that could end up being wrong when posted (and in the mobile space
> making ANOTHER http call just get yet another shortener is not
> realistic)
>
> 2) If the user has a bit.ly link, for example, and it's only say 15
> characters long and they post at 136 characters, it'll get rejected
> from Twitter whilst the client app said it was ok.
>
> 3) After the system has launch we could dynamically look at urls,
> ignore their character count and substitute it for 20 characters,
> however for iPhone developers we have AT LEAST a week's wait to get
> these things through Apple's approval process.
>
> But it's not just iPhone developers, all developers can't just push
> out code that quickly without a decent amount of testing
>
> We can't realistically do that until the system is live... and yet the
> apps will be 'broken' in the time between 'go live' and end of testing/
> approval.
>
> Do you see the chicken and the egg scenario here?
>
> My proposal is you need to keep the 140 limit as it is now, if your
> own link shortener makes it over 140 then so be it, that's your
> decision, not ours... but the original posted text was 140.
>
> On Jun 9, 2:30 pm, "Brian Smith"  wrote:
> > Dewald Pretorius wrote:
> > > StatusNet, blog, etc.). If the privacy argument carried any water, then
> > bit.ly
> > > would be a far greater privacy threat than Twitter.
> >
> > I agree, and that's why I (and others) were working on solutions to get
> > around bit.ly and other link shorteners. And, Twitter has already
> developed
> > the ultimate solution to that privacy issue, but they just are refusing
> to
> > let API developers use it. That is what is so incredibly frustrating.
> >
> > Regards,
> > Brian
>

[twitter-dev] Re: link wrapping on the API

[Rich]

I'm a +1 one for the huge can of worms this is going to cause with
realtime character counting.  Here are my concerns

1) An app realtime character counts what is entered in a text box and
that could end up being wrong when posted (and in the mobile space
making ANOTHER http call just get yet another shortener is not
realistic)

2) If the user has a bit.ly link, for example, and it's only say 15
characters long and they post at 136 characters, it'll get rejected
from Twitter whilst the client app said it was ok.

3) After the system has launch we could dynamically look at urls,
ignore their character count and substitute it for 20 characters,
however for iPhone developers we have AT LEAST a week's wait to get
these things through Apple's approval process.

But it's not just iPhone developers, all developers can't just push
out code that quickly without a decent amount of testing

We can't realistically do that until the system is live... and yet the
apps will be 'broken' in the time between 'go live' and end of testing/
approval.

Do you see the chicken and the egg scenario here?

My proposal is you need to keep the 140 limit as it is now, if your
own link shortener makes it over 140 then so be it, that's your
decision, not ours... but the original posted text was 140.

On Jun 9, 2:30 pm, "Brian Smith"  wrote:
> Dewald Pretorius wrote:
> > StatusNet, blog, etc.). If the privacy argument carried any water, then
> bit.ly
> > would be a far greater privacy threat than Twitter.
>
> I agree, and that's why I (and others) were working on solutions to get
> around bit.ly and other link shorteners. And, Twitter has already developed
> the ultimate solution to that privacy issue, but they just are refusing to
> let API developers use it. That is what is so incredibly frustrating.
>
> Regards,
> Brian

RE: [twitter-dev] Re: link wrapping on the API

[Brian Smith]

Dewald Pretorius wrote:
> StatusNet, blog, etc.). If the privacy argument carried any water, then
bit.ly
> would be a far greater privacy threat than Twitter.

I agree, and that's why I (and others) were working on solutions to get
around bit.ly and other link shorteners. And, Twitter has already developed
the ultimate solution to that privacy issue, but they just are refusing to
let API developers use it. That is what is so incredibly frustrating.

Regards,
Brian

[twitter-dev] Re: link wrapping on the API

[Dewald Pretorius]

This thing really is a can of worms, in terms of:

1) The impact on the ecosystem if characters are counted after link
wrapping.
2) The impact on Twitter's core system and SMS communication if
characters are counted before link wrapping.
3) The perception that there is no guaranteed stability in the TOS,
which makes developing and investing in the ecosystem increasingly
unattractive.

I don't buy the click tracking privacy argument. Twitter will have no
more insight into clicks than what bit.ly or any other shortening
service has, and they won't have any insight into clicks on links that
are not in tweets. For example, if I tweet something with a bit.ly
link and I use that same bit.ly link in other places, Twitter will
track only the clicks on the link in the tweet. They are not able to
track the clicks on that bit.ly link in the other places I've used it
(Facebook, Buzz, StatusNet, blog, etc.). If the privacy argument
carried any water, then bit.ly would be a far greater privacy threat
than Twitter.

[twitter-dev] Re: link wrapping on the API

[Terence Eden]

Would it be possible to use a 3rd party URL lengthener (like
http://longurlplease.com/) then use that?

Alice posts bit.ly/foo
Twitter changes it to t.co/bar
LongUrl returns bit.ly/foo resolves to example.com
App shows http://t.co/bar";>example.com
When Bob clicks on example.com, he hits t.co, which then redirects to
bit.ly, which then redirects to example.

Aside from the annoyance / speed on a mobile of seeing the page reload
several times (assuming example.com doesn't redirect) does that seems
to be ok?

Terence

On Jun 9, 10:04 am, artesea  wrote:
> At present my code reads a tweet in plain text format. It then adds
>  tags to any links, the @username and the #tags. It then looks for
> twitpic/flickr/youtube etc urls it will add a clickable thumbnail to
> the top of the tweet.
> As I can see it, my code will need to put the t.co in the  tag, do
> a lookup on the full url and replace the t.co in the text of the
> tweet, check for twitpic links, add the thumbnail, BUT ensure that the
> link on the thumbnail goes via t.co else it will be a breach of the
> TOS.
> Will not parsing the image url and displaying the thumbnail in the
> first place be a breach of the TOS as I haven't sent the users browser
> via t.co?
>
> To repeat another request will you be providing search results with
> words inside the URLs redirects like you already do for tinyurl
> bit.ly?
>
> Also two weeks, with just a couple of feeds to test against and no
> sample code???
>
> On Jun 9, 9:45 am, Rich  wrote:
>
> > Please make sure the timeline for this is LONGER than 2 weeks please,
> > some of us have to code this and then wait at least a week to get
> > through Apple's approval system.  This is especially important when it
> > comes to detecting images, etc.
>
> > Richard
>
> > On Jun 9, 4:27 am, Raffi Krikorian  wrote:
>
> > > yeah - its definitely case that counting characters will become a bit more
> > > subtle.  i hope that we can provide a really good and easy way to help you
> > > all out.  at the very least we are going to update documentation, but i 
> > > know
> > > we can do better than that.
>
> > > On Tue, Jun 8, 2010 at 8:17 PM, Andy Matsubara 
> > > wrote:
>
> > > > Raffi wrote:
> > > > > related to this: the way the Twitter API counts characters is going to
> > > > change ever so slightly. our 140
> > > > > characters is now going to be defined as 140 characters after link
> > > > wrapping. t.co links are of a
> > > > > predictable length -- they will always be 20 characters. after we make
> > > > this live, it will be feasible to
> > > > > send in the text for a status that is greater than 140 characters. the
> > > > rule is after the link wrapping,
> > > > > the text transforms to 140 characters or fewer. we'll be using the 
> > > > > same
> > > > logic that is in twitter-text-rb
> > > > > to figure out what is a URL.
> > > > I guess this change will make frontend text handling more difficult.
> > > > Counting characters in a text box must figure out what is a URL. I
> > > > hope Twitter will publish JavaScript library for realtime character
> > > > counts. I also want APIs to make shortened URL.
>
> > > > Andy Matsubara
>
> > > --
> > > Raffi Krikorian
> > > Twitter Platform Teamhttp://twitter.com/raffi

Re: [twitter-dev] Re: link wrapping on the API

[Cameron Kaiser]

> But if apps don't update and user sends a tweet which is just below
> 140 characters say, 139, and which contain a link(s) shorter than 19
> (or is it 20) characters will mysteriously fail. The user will wonder
> why the app doesn't let them send the tweet when their app clearly
> says it's still within 140 characters, because Twitter is now counting
> the longer 19/20 character t.co link.
> 
> Is this considered a rare scenario?

I wouldn't consider this a rare scenario; in fact, I think this might be
common with some clients. The proper way to solve this is to offer a
shortener API that a front end can talk to. *cough*

-- 
 personal: http://www.cameronkaiser.com/ --
  Cameron Kaiser * Floodgap Systems * www.floodgap.com * ckai...@floodgap.com
-- We aren't your science project! -- Akane, "Ranma 1/2" (OAV 9) --

[twitter-dev] Re: link wrapping on the API

[artesea]

At present my code reads a tweet in plain text format. It then adds
 tags to any links, the @username and the #tags. It then looks for
twitpic/flickr/youtube etc urls it will add a clickable thumbnail to
the top of the tweet.
As I can see it, my code will need to put the t.co in the  tag, do
a lookup on the full url and replace the t.co in the text of the
tweet, check for twitpic links, add the thumbnail, BUT ensure that the
link on the thumbnail goes via t.co else it will be a breach of the
TOS.
Will not parsing the image url and displaying the thumbnail in the
first place be a breach of the TOS as I haven't sent the users browser
via t.co?

To repeat another request will you be providing search results with
words inside the URLs redirects like you already do for tinyurl
bit.ly?

Also two weeks, with just a couple of feeds to test against and no
sample code???

On Jun 9, 9:45 am, Rich  wrote:
> Please make sure the timeline for this is LONGER than 2 weeks please,
> some of us have to code this and then wait at least a week to get
> through Apple's approval system.  This is especially important when it
> comes to detecting images, etc.
>
> Richard
>
> On Jun 9, 4:27 am, Raffi Krikorian  wrote:
>
> > yeah - its definitely case that counting characters will become a bit more
> > subtle.  i hope that we can provide a really good and easy way to help you
> > all out.  at the very least we are going to update documentation, but i know
> > we can do better than that.
>
> > On Tue, Jun 8, 2010 at 8:17 PM, Andy Matsubara 
> > wrote:
>
> > > Raffi wrote:
> > > > related to this: the way the Twitter API counts characters is going to
> > > change ever so slightly. our 140
> > > > characters is now going to be defined as 140 characters after link
> > > wrapping. t.co links are of a
> > > > predictable length -- they will always be 20 characters. after we make
> > > this live, it will be feasible to
> > > > send in the text for a status that is greater than 140 characters. the
> > > rule is after the link wrapping,
> > > > the text transforms to 140 characters or fewer. we'll be using the same
> > > logic that is in twitter-text-rb
> > > > to figure out what is a URL.
> > > I guess this change will make frontend text handling more difficult.
> > > Counting characters in a text box must figure out what is a URL. I
> > > hope Twitter will publish JavaScript library for realtime character
> > > counts. I also want APIs to make shortened URL.
>
> > > Andy Matsubara
>
> > --
> > Raffi Krikorian
> > Twitter Platform Teamhttp://twitter.com/raffi

[twitter-dev] Re: link wrapping on the API

[Rich]

Please make sure the timeline for this is LONGER than 2 weeks please,
some of us have to code this and then wait at least a week to get
through Apple's approval system.  This is especially important when it
comes to detecting images, etc.

Richard

On Jun 9, 4:27 am, Raffi Krikorian  wrote:
> yeah - its definitely case that counting characters will become a bit more
> subtle.  i hope that we can provide a really good and easy way to help you
> all out.  at the very least we are going to update documentation, but i know
> we can do better than that.
>
> On Tue, Jun 8, 2010 at 8:17 PM, Andy Matsubara wrote:
>
>
>
>
>
> > Raffi wrote:
> > > related to this: the way the Twitter API counts characters is going to
> > change ever so slightly. our 140
> > > characters is now going to be defined as 140 characters after link
> > wrapping. t.co links are of a
> > > predictable length -- they will always be 20 characters. after we make
> > this live, it will be feasible to
> > > send in the text for a status that is greater than 140 characters. the
> > rule is after the link wrapping,
> > > the text transforms to 140 characters or fewer. we'll be using the same
> > logic that is in twitter-text-rb
> > > to figure out what is a URL.
> > I guess this change will make frontend text handling more difficult.
> > Counting characters in a text box must figure out what is a URL. I
> > hope Twitter will publish JavaScript library for realtime character
> > counts. I also want APIs to make shortened URL.
>
> > Andy Matsubara
>
> --
> Raffi Krikorian
> Twitter Platform Teamhttp://twitter.com/raffi

[twitter-dev] Re: link wrapping on the API

[Ken]

Sorry if this is pedantic, but can you point to Twitter's definition
of "malicious" ?

Obviously, viruses, phishing etc. Presumably, "fraudulent" or
"illegal" would be included, but this might vary depending on the
jurisdiction.

Also, if a site is banned in country x, can the government of x
request that Twitter disable redirection for users in its
jurisdiction? Twitter might prefer this to a complete ban on Twitter
itself...

Re: [twitter-dev] Re: link wrapping on the API

[Bernd Stramm]

On Tue, 8 Jun 2010 22:19:04 -0700 (PDT)
Hwee-Boon Yar  wrote:

> But if apps don't update and user sends a tweet which is just below
> 140 characters say, 139, and which contain a link(s) shorter than 19
> (or is it 20) characters will mysteriously fail. The user will wonder
> why the app doesn't let them send the tweet when their app clearly
> says it's still within 140 characters, because Twitter is now counting
> the longer 19/20 character t.co link.
> 
> Is this considered a rare scenario?

The right way to do this for twitter would be to count the characters
submitted to the API, before twitter changes the content.

That way, the API acceptance of a well formed post is predictable.
Otherwise it's not, since an application really doesn't know what
twitter will do with the content.

> 
> --
> Hwee-Boon
> 
> On Jun 9, 12:18 pm, John Kalucki  wrote:
> > Apps that don't update will continue to work, they will just display
> > something different than they do now.



-- 
Bernd Stramm

[twitter-dev] Re: link wrapping on the API

[Hwee-Boon Yar]

But if apps don't update and user sends a tweet which is just below
140 characters say, 139, and which contain a link(s) shorter than 19
(or is it 20) characters will mysteriously fail. The user will wonder
why the app doesn't let them send the tweet when their app clearly
says it's still within 140 characters, because Twitter is now counting
the longer 19/20 character t.co link.

Is this considered a rare scenario?

--
Hwee-Boon

On Jun 9, 12:18 pm, John Kalucki  wrote:
> Apps that don't update will continue to work, they will just display
> something different than they do now.

Re: [twitter-dev] Re: link wrapping on the API

[John Kalucki]

Existing url shortners will continue to work just fine. We're not
going to resolve them to their final link and remove them from the
chain.

By redirecting all links, we can protect all users and the entire
ecosystem much faster. The adoption via opt-in would be slower, and
might never reach critical mass.

Apps that don't update will continue to work, they will just display
something different than they do now.


On Tue, Jun 8, 2010 at 9:06 PM, Alex B  wrote:
> OK, it's a little confusing naming for display URL, as that implies
> that is what clients should show directly to the users, as most of the
> time I would imagine that field should be cut for brevity.
>
> The difference between having a ping service that can help twitter
> track clicks and a redirect service that can help twitter protect
> users, and having twitter simply force-edit everyone's tweet texts, is
> that instead of providing a new service that developers and users can
> opt to use, you are providing a service that everyone _must_ use or
> add code to work around.
>
> You could have simply provided an extension to posting new tweets that
> identified the real urls of shortened urls, that would have protected
> short url providers who invested in your platform and allowed
> developers to add the improvement on their own time frames.
>
> I like the general idea of embedding real links in the twitter
> metadata even if it adds to an already bloated tweet data payload, but
> Twitter isn't respecting its ecosystem here by forcing complexity on
> all developers and giving a time frame of weeks to change established
> code developed over years.
>
> On Jun 9, 11:18 am, Raffi Krikorian  wrote:
>> > What's the algorithm for the display url? Ideally it will be a
>> > predictable length, to aid predictability in tweet display code.
>>
>> i'm not sure why the display_url would be of predictable length?  the
>> display_url is -exactly- the URL that the user has sent into the system.
>>  so, that may be of varying length.
>>
>> > If the motive is really to protect us from malicious URLs, what about
>> > giving a service we can call to route links through your protective
>> > redirect servers? Then we can give users the option to be protected by
>> > your malicious detection algorithms if they want.
>>
>> If you want to click track every URL for whatever reason, ask client
>>
>> > developers to hit a ping URL if the user clicks? I'm not sure
>> > otherwise how you will tell in a software client if it's the user
>> > requesting the t.co URL or the software.
>>
>> i guess i'm confused on this as well?  isn't that what t.co does?
>>
>>
>>
>>
>>
>> > On Jun 9, 6:57 am, Raffi Krikorian  wrote:
>> > > hi all.
>>
>> > > twitter has been wrapping links in e-mailed DMs for a couple months
>> > > now.
>> > > with that feature, we're trying to protect users against phishing and
>> > other
>> > > malicious attacks. the way that we're doing this is that any URL that
>> > comes
>> > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
>> > turns
>> > > out to be malicious, Twitter can simply shut it down, and whoever follows
>> > > that link will be presented with a page that warns them of potentially
>> > > malicious content. in a few weeks, we're going to start slowly enabling
>> > this
>> > > throughout the API for all statuses as well, but instead of twt.tl, we
>> > will
>> > > be using t.co.
>>
>> > > practically, any tweet that is sent through statuses/update that has a
>> > link
>> > > on it will have the link automatically converted to a t.co link on its
>> > way
>> > > through the Twitter platform. if you fetch any tweet created after this
>> > > change goes live, then its text field will have all its links
>> > automatically
>> > > wrapped with t.co links. when a user clicks on that link, Twitter will
>> > > redirect them to the original URL after first confirming with our
>> > database
>> > > that that URL is not malicious.  on top of the end-user benefit, we hope
>> > to
>> > > eventually provide all developers with aggregate usage data around your
>> > > applications such as the number of clicks people make on URLs you display
>> > > (it will, of course, be in aggregate and not identifiable manner).
>> > > additionally, we want to be able to build services and APIs that can make
>> > > algorithmic recommendations to users based on the content they are
>> > > consuming. gathering the data from t.co will help make these possible.
>>
>> > > our current plan is that no user will see a t.co URL on twitter.com but
>> > we
>> > > still have some details to work through. the links will still be
>> > displayed
>> > > as they were sent in, but the target of the link will be the t.co link
>> > > instead. and, we want to provide the same ability to display original
>> > links
>> > > to developers. we're going to use the entities attribute to make this
>> > > possible.
>>
>> > > let's say i send out the following tweet

Re: [twitter-dev] Re: link wrapping on the API

[John Kalucki]

All links will be wrapped. It's not about length.


On Tue, Jun 8, 2010 at 9:06 PM, Alex B  wrote:
> OK, it's a little confusing naming for display URL, as that implies
> that is what clients should show directly to the users, as most of the
> time I would imagine that field should be cut for brevity.
>
> The difference between having a ping service that can help twitter
> track clicks and a redirect service that can help twitter protect
> users, and having twitter simply force-edit everyone's tweet texts, is
> that instead of providing a new service that developers and users can
> opt to use, you are providing a service that everyone _must_ use or
> add code to work around.
>
> You could have simply provided an extension to posting new tweets that
> identified the real urls of shortened urls, that would have protected
> short url providers who invested in your platform and allowed
> developers to add the improvement on their own time frames.
>
> I like the general idea of embedding real links in the twitter
> metadata even if it adds to an already bloated tweet data payload, but
> Twitter isn't respecting its ecosystem here by forcing complexity on
> all developers and giving a time frame of weeks to change established
> code developed over years.
>
> On Jun 9, 11:18 am, Raffi Krikorian  wrote:
>> > What's the algorithm for the display url? Ideally it will be a
>> > predictable length, to aid predictability in tweet display code.
>>
>> i'm not sure why the display_url would be of predictable length?  the
>> display_url is -exactly- the URL that the user has sent into the system.
>>  so, that may be of varying length.
>>
>> > If the motive is really to protect us from malicious URLs, what about
>> > giving a service we can call to route links through your protective
>> > redirect servers? Then we can give users the option to be protected by
>> > your malicious detection algorithms if they want.
>>
>> If you want to click track every URL for whatever reason, ask client
>>
>> > developers to hit a ping URL if the user clicks? I'm not sure
>> > otherwise how you will tell in a software client if it's the user
>> > requesting the t.co URL or the software.
>>
>> i guess i'm confused on this as well?  isn't that what t.co does?
>>
>>
>>
>>
>>
>> > On Jun 9, 6:57 am, Raffi Krikorian  wrote:
>> > > hi all.
>>
>> > > twitter has been wrapping links in e-mailed DMs for a couple months
>> > > now.
>> > > with that feature, we're trying to protect users against phishing and
>> > other
>> > > malicious attacks. the way that we're doing this is that any URL that
>> > comes
>> > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
>> > turns
>> > > out to be malicious, Twitter can simply shut it down, and whoever follows
>> > > that link will be presented with a page that warns them of potentially
>> > > malicious content. in a few weeks, we're going to start slowly enabling
>> > this
>> > > throughout the API for all statuses as well, but instead of twt.tl, we
>> > will
>> > > be using t.co.
>>
>> > > practically, any tweet that is sent through statuses/update that has a
>> > link
>> > > on it will have the link automatically converted to a t.co link on its
>> > way
>> > > through the Twitter platform. if you fetch any tweet created after this
>> > > change goes live, then its text field will have all its links
>> > automatically
>> > > wrapped with t.co links. when a user clicks on that link, Twitter will
>> > > redirect them to the original URL after first confirming with our
>> > database
>> > > that that URL is not malicious.  on top of the end-user benefit, we hope
>> > to
>> > > eventually provide all developers with aggregate usage data around your
>> > > applications such as the number of clicks people make on URLs you display
>> > > (it will, of course, be in aggregate and not identifiable manner).
>> > > additionally, we want to be able to build services and APIs that can make
>> > > algorithmic recommendations to users based on the content they are
>> > > consuming. gathering the data from t.co will help make these possible.
>>
>> > > our current plan is that no user will see a t.co URL on twitter.com but
>> > we
>> > > still have some details to work through. the links will still be
>> > displayed
>> > > as they were sent in, but the target of the link will be the t.co link
>> > > instead. and, we want to provide the same ability to display original
>> > links
>> > > to developers. we're going to use the entities attribute to make this
>> > > possible.
>>
>> > > let's say i send out the following tweet: "you have to check outhttp://
>> > dev.twitter.com!"
>>
>> > > a returned (and truncated) status object may look like:
>>
>> > > {
>> > >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>> > >   ...
>> > >   "user" : {
>> > >     "screen_name" : "raffi",
>> > >     ...
>> > >   },
>> > >   ...
>> > >   "entities" : {
>> > >     "urls" : [
>> > >       

[twitter-dev] Re: link wrapping on the API

[Alex B]

OK, it's a little confusing naming for display URL, as that implies
that is what clients should show directly to the users, as most of the
time I would imagine that field should be cut for brevity.

The difference between having a ping service that can help twitter
track clicks and a redirect service that can help twitter protect
users, and having twitter simply force-edit everyone's tweet texts, is
that instead of providing a new service that developers and users can
opt to use, you are providing a service that everyone _must_ use or
add code to work around.

You could have simply provided an extension to posting new tweets that
identified the real urls of shortened urls, that would have protected
short url providers who invested in your platform and allowed
developers to add the improvement on their own time frames.

I like the general idea of embedding real links in the twitter
metadata even if it adds to an already bloated tweet data payload, but
Twitter isn't respecting its ecosystem here by forcing complexity on
all developers and giving a time frame of weeks to change established
code developed over years.

On Jun 9, 11:18 am, Raffi Krikorian  wrote:
> > What's the algorithm for the display url? Ideally it will be a
> > predictable length, to aid predictability in tweet display code.
>
> i'm not sure why the display_url would be of predictable length?  the
> display_url is -exactly- the URL that the user has sent into the system.
>  so, that may be of varying length.
>
> > If the motive is really to protect us from malicious URLs, what about
> > giving a service we can call to route links through your protective
> > redirect servers? Then we can give users the option to be protected by
> > your malicious detection algorithms if they want.
>
> If you want to click track every URL for whatever reason, ask client
>
> > developers to hit a ping URL if the user clicks? I'm not sure
> > otherwise how you will tell in a software client if it's the user
> > requesting the t.co URL or the software.
>
> i guess i'm confused on this as well?  isn't that what t.co does?
>
>
>
>
>
> > On Jun 9, 6:57 am, Raffi Krikorian  wrote:
> > > hi all.
>
> > > twitter has been wrapping links in e-mailed DMs for a couple months
> > > now.
> > > with that feature, we're trying to protect users against phishing and
> > other
> > > malicious attacks. the way that we're doing this is that any URL that
> > comes
> > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> > turns
> > > out to be malicious, Twitter can simply shut it down, and whoever follows
> > > that link will be presented with a page that warns them of potentially
> > > malicious content. in a few weeks, we're going to start slowly enabling
> > this
> > > throughout the API for all statuses as well, but instead of twt.tl, we
> > will
> > > be using t.co.
>
> > > practically, any tweet that is sent through statuses/update that has a
> > link
> > > on it will have the link automatically converted to a t.co link on its
> > way
> > > through the Twitter platform. if you fetch any tweet created after this
> > > change goes live, then its text field will have all its links
> > automatically
> > > wrapped with t.co links. when a user clicks on that link, Twitter will
> > > redirect them to the original URL after first confirming with our
> > database
> > > that that URL is not malicious.  on top of the end-user benefit, we hope
> > to
> > > eventually provide all developers with aggregate usage data around your
> > > applications such as the number of clicks people make on URLs you display
> > > (it will, of course, be in aggregate and not identifiable manner).
> > > additionally, we want to be able to build services and APIs that can make
> > > algorithmic recommendations to users based on the content they are
> > > consuming. gathering the data from t.co will help make these possible.
>
> > > our current plan is that no user will see a t.co URL on twitter.com but
> > we
> > > still have some details to work through. the links will still be
> > displayed
> > > as they were sent in, but the target of the link will be the t.co link
> > > instead. and, we want to provide the same ability to display original
> > links
> > > to developers. we're going to use the entities attribute to make this
> > > possible.
>
> > > let's say i send out the following tweet: "you have to check outhttp://
> > dev.twitter.com!"
>
> > > a returned (and truncated) status object may look like:
>
> > > {
> > >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> > >   ...
> > >   "user" : {
> > >     "screen_name" : "raffi",
> > >     ...
> > >   },
> > >   ...
> > >   "entities" : {
> > >     "urls" : [
> > >       {
> > >         "url" : "http://t.co/s9gfk2d4";,
> > >         "display_url" : "http://dev.twitter.com";,
> > >         "indices" : [23, 43]
> > >       }
> > >     ],
> > >     ...
> > >   },
> > >   ...
>
> > > }
>
> > > two th

[twitter-dev] Re: link wrapping on the API

[Hwee-Boon Yar]

Are all links going to be wrapped or only long links? If it's the
latter, what's the definition?

1. This affects how we count characters before sending and has quite a
potential to go wrong, since we'll now need to know exactly which
links are going to be wrapped in a tweet.

2. It's also going to be tricky for apps that currently show a live
character count (like SimplyTweet and many other iPhone Twitter apps
and possibly web sites) as users type.

--
Hwee-Boon

On Jun 9, 6:57 am, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a best practices
> document on how to use these t.co links.
>
> what's the timeline?  "soon" we'll enable this on @twitterapi, @rsarver,
> @raffi, and a few other test accounts so you all have live data to play
> with.  on the timescale

RE: [twitter-dev] Re: link wrapping on the API

[Brian Smith]

I was basing my statement on the blog post, which indicated that at least
some "display URLs" will be truncated:

 

http://blog.twitter.com/2010/06/links-and-twitter-length-shouldnt.html

 

"A really long link such as
http://www.amazon.com/Delivering-Happiness-Profits-Passion-Purpose/dp/044656
3048 might be wrapped as http://t.co/DRo0trj for display on SMS, but it
could be displayed to web or application users as amazon.com/Delivering-."

 

Will the application be doing the truncation from the full URL to the
truncated one ("amazon.com/Delivering-"), or will the API?

 

And, if the application really will get the full destination URL, then it is
even more ridiculous to prevent them (through the ToS) from using it to
improve the user's privacy, don't you think?

 

Regards,

Brian

 

 

 

Right now, Twitter can see all the links that users *post*, but they don't
see which links users *click*.

In order to implement this feature, Twitter has already built the framework
that does all the hard work that applications need to protect users' privacy
against (link-shortener) click-tracking. Twitter will be withholding that
final URL from applications, preventing us (through the ToS) from
implementing our own anti-click-tracking privacy measures. If, instead, they
gave the final URL to the application and let the application use that URL,
then applications could implement anti-click-tracking privacy measures with
an even greater degree of privacy than they could by using a third-party
service.

 

hey brian - just wanted to point out - the "Twitter will be withholding that
final URL from applications" is NOT true at all.  we are providing, as part
of the "entities" the original URL back to the developers.  stated another
way - we are giving all the data back and we are not withholding the data.

 

-- 
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

yeah - its definitely case that counting characters will become a bit more
subtle.  i hope that we can provide a really good and easy way to help you
all out.  at the very least we are going to update documentation, but i know
we can do better than that.

On Tue, Jun 8, 2010 at 8:17 PM, Andy Matsubara wrote:

> Raffi wrote:
> > related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140
> > characters is now going to be defined as 140 characters after link
> wrapping. t.co links are of a
> > predictable length -- they will always be 20 characters. after we make
> this live, it will be feasible to
> > send in the text for a status that is greater than 140 characters. the
> rule is after the link wrapping,
> > the text transforms to 140 characters or fewer. we'll be using the same
> logic that is in twitter-text-rb
> > to figure out what is a URL.
> I guess this change will make frontend text handling more difficult.
> Counting characters in a text box must figure out what is a URL. I
> hope Twitter will publish JavaScript library for realtime character
> counts. I also want APIs to make shortened URL.
>
> Andy Matsubara
>



-- 
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

>
> Right now, Twitter can see all the links that users *post*, but they don't
> see which links users *click*.
>
> In order to implement this feature, Twitter has already built the framework
> that does all the hard work that applications need to protect users'
> privacy
> against (link-shortener) click-tracking. Twitter will be withholding that
> final URL from applications, preventing us (through the ToS) from
> implementing our own anti-click-tracking privacy measures. If, instead,
> they
> gave the final URL to the application and let the application use that URL,
> then applications could implement anti-click-tracking privacy measures with
> an even greater degree of privacy than they could by using a third-party
> service.
>

hey brian - just wanted to point out - the "Twitter will be withholding
that final URL from applications" is NOT true at all.  we are providing, as
part of the "entities" the original URL back to the developers.  stated
another way - we are giving all the data back and we are not withholding the
data.

-- 
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

>
> What's the algorithm for the display url? Ideally it will be a
> predictable length, to aid predictability in tweet display code.
>

i'm not sure why the display_url would be of predictable length?  the
display_url is -exactly- the URL that the user has sent into the system.
 so, that may be of varying length.


> If the motive is really to protect us from malicious URLs, what about
> giving a service we can call to route links through your protective
> redirect servers? Then we can give users the option to be protected by
> your malicious detection algorithms if they want.
>

If you want to click track every URL for whatever reason, ask client
> developers to hit a ping URL if the user clicks? I'm not sure
> otherwise how you will tell in a software client if it's the user
> requesting the t.co URL or the software.
>

i guess i'm confused on this as well?  isn't that what t.co does?




>
> On Jun 9, 6:57 am, Raffi Krikorian  wrote:
> > hi all.
> >
> > twitter has been wrapping links in e-mailed DMs for a couple months
> > now.
> > with that feature, we're trying to protect users against phishing and
> other
> > malicious attacks. the way that we're doing this is that any URL that
> comes
> > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> turns
> > out to be malicious, Twitter can simply shut it down, and whoever follows
> > that link will be presented with a page that warns them of potentially
> > malicious content. in a few weeks, we're going to start slowly enabling
> this
> > throughout the API for all statuses as well, but instead of twt.tl, we
> will
> > be using t.co.
> >
> > practically, any tweet that is sent through statuses/update that has a
> link
> > on it will have the link automatically converted to a t.co link on its
> way
> > through the Twitter platform. if you fetch any tweet created after this
> > change goes live, then its text field will have all its links
> automatically
> > wrapped with t.co links. when a user clicks on that link, Twitter will
> > redirect them to the original URL after first confirming with our
> database
> > that that URL is not malicious.  on top of the end-user benefit, we hope
> to
> > eventually provide all developers with aggregate usage data around your
> > applications such as the number of clicks people make on URLs you display
> > (it will, of course, be in aggregate and not identifiable manner).
> > additionally, we want to be able to build services and APIs that can make
> > algorithmic recommendations to users based on the content they are
> > consuming. gathering the data from t.co will help make these possible.
> >
> > our current plan is that no user will see a t.co URL on twitter.com but
> we
> > still have some details to work through. the links will still be
> displayed
> > as they were sent in, but the target of the link will be the t.co link
> > instead. and, we want to provide the same ability to display original
> links
> > to developers. we're going to use the entities attribute to make this
> > possible.
> >
> > let's say i send out the following tweet: "you have to check outhttp://
> dev.twitter.com!"
> >
> > a returned (and truncated) status object may look like:
> >
> > {
> >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> >   ...
> >   "user" : {
> > "screen_name" : "raffi",
> > ...
> >   },
> >   ...
> >   "entities" : {
> > "urls" : [
> >   {
> > "url" : "http://t.co/s9gfk2d4";,
> > "display_url" : "http://dev.twitter.com";,
> > "indices" : [23, 43]
> >   }
> > ],
> > ...
> >   },
> >   ...
> >
> > }
> >
> > two things to note: the text of the returned status object doesn't have
> the
> > original URL and instead it has a t.co URL, and the entities block now
> has a
> > display_url attribute associated with it. what we're hoping is that with
> > this data, it should be relatively easy to create a UI where you replace
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
> >
> > http://t.co/s9gfk2d4";>http://dev.twitter.com
> >
> > this means the user would not see the t.co link, but we all can still
> > provide the protection and gather data from the wrapped link. for the
> > applications that don't choose to update, the t.co link will be shown
> (and
> > the goal to protect users will be met). i just want to emphasize -- we
> > really do hope that you all render the original URL, but please send the
> > user through the t.co link.   if you do choose to prefetch all the URLs
> on a
> > timeline, then, when a user actually clicks on one of the links, please
> > still send him or her through t.co. We will be updating the TOS to
> require
> > you to check t.co and register the click.
> >
> > related to this: the way the Twitter API counts characters is going to
> > change ever so slightly. our 140 characters is now going to be defined as
> > 140 characters after link wrapping. t.co links are of a predictable
> 

Re: [twitter-dev] Re: link wrapping on the API

[Andy Matsubara]

Raffi wrote:
> related to this: the way the Twitter API counts characters is going to change 
> ever so slightly. our 140
> characters is now going to be defined as 140 characters after link wrapping. 
> t.co links are of a
> predictable length -- they will always be 20 characters. after we make this 
> live, it will be feasible to
> send in the text for a status that is greater than 140 characters. the rule 
> is after the link wrapping,
> the text transforms to 140 characters or fewer. we'll be using the same logic 
> that is in twitter-text-rb
> to figure out what is a URL.
I guess this change will make frontend text handling more difficult.
Counting characters in a text box must figure out what is a URL. I
hope Twitter will publish JavaScript library for realtime character
counts. I also want APIs to make shortened URL.

Andy Matsubara

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

our hope is to eventually provide this analytics.

On Tue, Jun 8, 2010 at 7:14 PM, Sami  wrote:

> I don't see how this feature could impact user privacy more than what
> it is right now. Today Twitter stores all links for all users and they
> can spy on them and the t.co shortner is not changing that :)
>
> My question is, will developers have access to analytics from t.co
> through API?
>
> Thanks
>

-- 
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi

RE: [twitter-dev] Re: link wrapping on the API

[Brian Smith]

Sami wrote:
> I don't see how this feature could impact user privacy more than what it
is right
> now. Today Twitter stores all links for all users and they can spy on them
and the
> t.co shortner is not changing that :)

Right now, Twitter can see all the links that users *post*, but they don't
see which links users *click*.

In order to implement this feature, Twitter has already built the framework
that does all the hard work that applications need to protect users' privacy
against (link-shortener) click-tracking. Twitter will be withholding that
final URL from applications, preventing us (through the ToS) from
implementing our own anti-click-tracking privacy measures. If, instead, they
gave the final URL to the application and let the application use that URL,
then applications could implement anti-click-tracking privacy measures with
an even greater degree of privacy than they could by using a third-party
service.

In other words, instead of Twitter using their existing link-unshortening
technology to let applications tell *fewer* companies what links your users
are clicking on, they are using it to force applications to tell *more*
companies what your users are clicking on.

Only advertisers could build a privacy-improving technology and using it for
the exact opposite purpose. :(

http://mashable.com/2010/06/03/alex-payne-twitter-interview/

Regards,
Brian Smith
@BRIAN_

[twitter-dev] Re: link wrapping on the API

[Alex B]

What's the algorithm for the display url? Ideally it will be a
predictable length, to aid predictability in tweet display code.

If the motive is really to protect us from malicious URLs, what about
giving a service we can call to route links through your protective
redirect servers? Then we can give users the option to be protected by
your malicious detection algorithms if they want.

If you want to click track every URL for whatever reason, ask client
developers to hit a ping URL if the user clicks? I'm not sure
otherwise how you will tell in a software client if it's the user
requesting the t.co URL or the software.

On Jun 9, 6:57 am, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a best practices
> document on how to use these t.co links.
>
> what's the ti

[twitter-dev] Re: link wrapping on the API

[Sami]

I don't see how this feature could impact user privacy more than what
it is right now. Today Twitter stores all links for all users and they
can spy on them and the t.co shortner is not changing that :)

My question is, will developers have access to analytics from t.co
through API?

Thanks

[twitter-dev] Re: link wrapping on the API

[Damon C]

+1 on this, I'd like to know the answer as well.

Damon/@dacort

On Jun 8, 4:43 pm, Jim Gilliam  wrote:
> Will we be able to get matches on the original URL through the streaming
> API?
>
> For example, I'm tracking "act" so I can match tweets that link to 
> 'http://act.ly'.  Will I still be able to do that?
>
> Jim Gilliamhttp://act.ly/http://twitter.com/jgilliam
>
>
>
> On Tue, Jun 8, 2010 at 4:33 PM, Dewald Pretorius  wrote:
> > Raffi,
>
> > I'm fine with everything up to the new 140 character count.
>
> > If you count the characters *after* link wrapping, you are seriously
> > going to mess up my system. My short URLs are currently 18 characters
> > long, and they will be 18 long for quite some time to come. After that
> > they will be 19 for a very long time to come.
>
> > If you implement this change, a ton, and I mean a *huge* number of my
> > system's updates are going to be rejected for being over 140
> > characters.
>
> > On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
> > > hi all.
>
> > > twitter has been wrapping links in e-mailed DMs for a couple months
> > > now.
> > > with that feature, we're trying to protect users against phishing and
> > other
> > > malicious attacks. the way that we're doing this is that any URL that
> > comes
> > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> > turns
> > > out to be malicious, Twitter can simply shut it down, and whoever follows
> > > that link will be presented with a page that warns them of potentially
> > > malicious content. in a few weeks, we're going to start slowly enabling
> > this
> > > throughout the API for all statuses as well, but instead of twt.tl, we
> > will
> > > be using t.co.
>
> > > practically, any tweet that is sent through statuses/update that has a
> > link
> > > on it will have the link automatically converted to a t.co link on its
> > way
> > > through the Twitter platform. if you fetch any tweet created after this
> > > change goes live, then its text field will have all its links
> > automatically
> > > wrapped with t.co links. when a user clicks on that link, Twitter will
> > > redirect them to the original URL after first confirming with our
> > database
> > > that that URL is not malicious.  on top of the end-user benefit, we hope
> > to
> > > eventually provide all developers with aggregate usage data around your
> > > applications such as the number of clicks people make on URLs you display
> > > (it will, of course, be in aggregate and not identifiable manner).
> > > additionally, we want to be able to build services and APIs that can make
> > > algorithmic recommendations to users based on the content they are
> > > consuming. gathering the data from t.co will help make these possible.
>
> > > our current plan is that no user will see a t.co URL on twitter.com but
> > we
> > > still have some details to work through. the links will still be
> > displayed
> > > as they were sent in, but the target of the link will be the t.co link
> > > instead. and, we want to provide the same ability to display original
> > links
> > > to developers. we're going to use the entities attribute to make this
> > > possible.
>
> > > let's say i send out the following tweet: "you have to check outhttp://
> > dev.twitter.com!"
>
> > > a returned (and truncated) status object may look like:
>
> > > {
> > >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> > >   ...
> > >   "user" : {
> > >     "screen_name" : "raffi",
> > >     ...
> > >   },
> > >   ...
> > >   "entities" : {
> > >     "urls" : [
> > >       {
> > >         "url" : "http://t.co/s9gfk2d4";,
> > >         "display_url" : "http://dev.twitter.com";,
> > >         "indices" : [23, 43]
> > >       }
> > >     ],
> > >     ...
> > >   },
> > >   ...
>
> > > }
>
> > > two things to note: the text of the returned status object doesn't have
> > the
> > > original URL and instead it has a t.co URL, and the entities block now
> > has a
> > > display_url attribute associated with it. what we're hoping is that with
> > > this data, it should be relatively easy to create a UI where you replace
> > thehttp://t.co/s9gfk2d4inthe text with the equivalent of
>
> > > http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> > > this means the user would not see the t.co link, but we all can still
> > > provide the protection and gather data from the wrapped link. for the
> > > applications that don't choose to update, the t.co link will be shown
> > (and
> > > the goal to protect users will be met). i just want to emphasize -- we
> > > really do hope that you all render the original URL, but please send the
> > > user through the t.co link.   if you do choose to prefetch all the URLs
> > on a
> > > timeline, then, when a user actually clicks on one of the links, please
> > > still send him or her through t.co. We will be updating the TOS to
> > require
> > > you to check t.co and register the click.
>
> > > related to this: the way the Twitt

RE: [twitter-dev] Re: link wrapping on the API

[Brian Smith]

If you do this, you will literally be forcing app developers to waste users
time and money, especially over metered GRPS/3G connections. 

 

If the user can see the full URL, then why do they need to be "protected"
any more than they are when they use any other service? If anything, you
should be cutting through any and all redirects (shorteners) so that the
application can show the final URL to the user and avoid multiple useless,
latency-inducing redirects that reduce reliability and increase costs for
end-users and network operators. Cutting through all the redirects would
improve security AND improve on the users' privacy, instead of hurting it.

 

And, what about the user's right to privacy? You're basically forcing every
Twitter app to become spyware. Who wants to create spyware? No developers
with a conscience-and I'm sure that includes you guys at Twitter. Please ask
whoever's making these horrible decisions lately to reconsider at least this
one.

 

Sincerely,

Brian Smith

@BRIAN_

 

 

> two things to note: the text of the returned status object doesn't have
the
> original URL and instead it has a t.co URL, and the entities block now has
a
> display_url attribute associated with it. what we're hoping is that with

> this data, it should be relatively easy to create a UI where you replace
the

> http://t.co/s9gfk2d4in the text with the equivalent of

>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on
a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click. 

 

Re: [twitter-dev] Re: link wrapping on the API

[Jim Gilliam]

Will we be able to get matches on the original URL through the streaming
API?

For example, I'm tracking "act" so I can match tweets that link to '
http://act.ly'.  Will I still be able to do that?

Jim Gilliam
http://act.ly/
http://twitter.com/jgilliam

On Tue, Jun 8, 2010 at 4:33 PM, Dewald Pretorius  wrote:

> Raffi,
>
> I'm fine with everything up to the new 140 character count.
>
> If you count the characters *after* link wrapping, you are seriously
> going to mess up my system. My short URLs are currently 18 characters
> long, and they will be 18 long for quite some time to come. After that
> they will be 19 for a very long time to come.
>
> If you implement this change, a ton, and I mean a *huge* number of my
> system's updates are going to be rejected for being over 140
> characters.
>
> On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
> > hi all.
> >
> > twitter has been wrapping links in e-mailed DMs for a couple months
> > now.
> > with that feature, we're trying to protect users against phishing and
> other
> > malicious attacks. the way that we're doing this is that any URL that
> comes
> > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> turns
> > out to be malicious, Twitter can simply shut it down, and whoever follows
> > that link will be presented with a page that warns them of potentially
> > malicious content. in a few weeks, we're going to start slowly enabling
> this
> > throughout the API for all statuses as well, but instead of twt.tl, we
> will
> > be using t.co.
> >
> > practically, any tweet that is sent through statuses/update that has a
> link
> > on it will have the link automatically converted to a t.co link on its
> way
> > through the Twitter platform. if you fetch any tweet created after this
> > change goes live, then its text field will have all its links
> automatically
> > wrapped with t.co links. when a user clicks on that link, Twitter will
> > redirect them to the original URL after first confirming with our
> database
> > that that URL is not malicious.  on top of the end-user benefit, we hope
> to
> > eventually provide all developers with aggregate usage data around your
> > applications such as the number of clicks people make on URLs you display
> > (it will, of course, be in aggregate and not identifiable manner).
> > additionally, we want to be able to build services and APIs that can make
> > algorithmic recommendations to users based on the content they are
> > consuming. gathering the data from t.co will help make these possible.
> >
> > our current plan is that no user will see a t.co URL on twitter.com but
> we
> > still have some details to work through. the links will still be
> displayed
> > as they were sent in, but the target of the link will be the t.co link
> > instead. and, we want to provide the same ability to display original
> links
> > to developers. we're going to use the entities attribute to make this
> > possible.
> >
> > let's say i send out the following tweet: "you have to check outhttp://
> dev.twitter.com!"
> >
> > a returned (and truncated) status object may look like:
> >
> > {
> >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> >   ...
> >   "user" : {
> > "screen_name" : "raffi",
> > ...
> >   },
> >   ...
> >   "entities" : {
> > "urls" : [
> >   {
> > "url" : "http://t.co/s9gfk2d4";,
> > "display_url" : "http://dev.twitter.com";,
> > "indices" : [23, 43]
> >   }
> > ],
> > ...
> >   },
> >   ...
> >
> > }
> >
> > two things to note: the text of the returned status object doesn't have
> the
> > original URL and instead it has a t.co URL, and the entities block now
> has a
> > display_url attribute associated with it. what we're hoping is that with
> > this data, it should be relatively easy to create a UI where you replace
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
> >
> > http://t.co/s9gfk2d4";>http://dev.twitter.com
> >
> > this means the user would not see the t.co link, but we all can still
> > provide the protection and gather data from the wrapped link. for the
> > applications that don't choose to update, the t.co link will be shown
> (and
> > the goal to protect users will be met). i just want to emphasize -- we
> > really do hope that you all render the original URL, but please send the
> > user through the t.co link.   if you do choose to prefetch all the URLs
> on a
> > timeline, then, when a user actually clicks on one of the links, please
> > still send him or her through t.co. We will be updating the TOS to
> require
> > you to check t.co and register the click.
> >
> > related to this: the way the Twitter API counts characters is going to
> > change ever so slightly. our 140 characters is now going to be defined as
> > 140 characters after link wrapping. t.co links are of a predictable
> length
> > -- they will always be 20 characters. after we make this live, it will be
> > feasible to s

[twitter-dev] Re: link wrapping on the API

[Dewald Pretorius]

So what are you saying? Suck it up? That's what I am hearing.

I have a work-around for the problem, in that I can simply adjust my
in-house shortening service to start generating 19-character URLs. But
other developers don't have that option.

On Jun 8, 8:58 pm, Raffi Krikorian  wrote:
> its true, and we understand that.
>
> just to correct my previous post, however -- t.co links are 19 characters.
>
>
>
>
>
> On Tue, Jun 8, 2010 at 4:53 PM, Dewald Pretorius  wrote:
> > This is not unique to me. This will be problematic for anyone who uses
> > a shortening service that shortens URLs to less than 20 characters.
>
> > In these cases, you are basically adding characters to the submitted
> > text, and then rejecting the submitted text as being too long.
>
> > On Jun 8, 8:33 pm, Dewald Pretorius  wrote:
> > > Raffi,
>
> > > I'm fine with everything up to the new 140 character count.
>
> > > If you count the characters *after* link wrapping, you are seriously
> > > going to mess up my system. My short URLs are currently 18 characters
> > > long, and they will be 18 long for quite some time to come. After that
> > > they will be 19 for a very long time to come.
>
> > > If you implement this change, a ton, and I mean a *huge* number of my
> > > system's updates are going to be rejected for being over 140
> > > characters.
>
> > > On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
>
> > > > hi all.
>
> > > > twitter has been wrapping links in e-mailed DMs for a couple months
> > > > now.
> > > > with that feature, we're trying to protect users against phishing and
> > other
> > > > malicious attacks. the way that we're doing this is that any URL that
> > comes
> > > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> > turns
> > > > out to be malicious, Twitter can simply shut it down, and whoever
> > follows
> > > > that link will be presented with a page that warns them of potentially
> > > > malicious content. in a few weeks, we're going to start slowly enabling
> > this
> > > > throughout the API for all statuses as well, but instead of twt.tl, we
> > will
> > > > be using t.co.
>
> > > > practically, any tweet that is sent through statuses/update that has a
> > link
> > > > on it will have the link automatically converted to a t.co link on its
> > way
> > > > through the Twitter platform. if you fetch any tweet created after this
> > > > change goes live, then its text field will have all its links
> > automatically
> > > > wrapped with t.co links. when a user clicks on that link, Twitter will
> > > > redirect them to the original URL after first confirming with our
> > database
> > > > that that URL is not malicious.  on top of the end-user benefit, we
> > hope to
> > > > eventually provide all developers with aggregate usage data around your
> > > > applications such as the number of clicks people make on URLs you
> > display
> > > > (it will, of course, be in aggregate and not identifiable manner).
> > > > additionally, we want to be able to build services and APIs that can
> > make
> > > > algorithmic recommendations to users based on the content they are
> > > > consuming. gathering the data from t.co will help make these possible.
>
> > > > our current plan is that no user will see a t.co URL on twitter.combut 
> > > > we
> > > > still have some details to work through. the links will still be
> > displayed
> > > > as they were sent in, but the target of the link will be the t.co link
> > > > instead. and, we want to provide the same ability to display original
> > links
> > > > to developers. we're going to use the entities attribute to make this
> > > > possible.
>
> > > > let's say i send out the following tweet: "you have to check outhttp://
> > dev.twitter.com!"
>
> > > > a returned (and truncated) status object may look like:
>
> > > > {
> > > >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> > > >   ...
> > > >   "user" : {
> > > >     "screen_name" : "raffi",
> > > >     ...
> > > >   },
> > > >   ...
> > > >   "entities" : {
> > > >     "urls" : [
> > > >       {
> > > >         "url" : "http://t.co/s9gfk2d4";,
> > > >         "display_url" : "http://dev.twitter.com";,
> > > >         "indices" : [23, 43]
> > > >       }
> > > >     ],
> > > >     ...
> > > >   },
> > > >   ...
>
> > > > }
>
> > > > two things to note: the text of the returned status object doesn't have
> > the
> > > > original URL and instead it has a t.co URL, and the entities block now
> > has a
> > > > display_url attribute associated with it. what we're hoping is that
> > with
> > > > this data, it should be relatively easy to create a UI where you
> > replace thehttp://t.co/s9gfk2d4inthetext with the equivalent of
>
> > > > http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> > > > this means the user would not see the t.co link, but we all can still
> > > > provide the protection and gather data from the wrapped link. for the
> > > > applications that don't choose to up

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

its true, and we understand that.

just to correct my previous post, however -- t.co links are 19 characters.

On Tue, Jun 8, 2010 at 4:53 PM, Dewald Pretorius  wrote:

> This is not unique to me. This will be problematic for anyone who uses
> a shortening service that shortens URLs to less than 20 characters.
>
> In these cases, you are basically adding characters to the submitted
> text, and then rejecting the submitted text as being too long.
>
> On Jun 8, 8:33 pm, Dewald Pretorius  wrote:
> > Raffi,
> >
> > I'm fine with everything up to the new 140 character count.
> >
> > If you count the characters *after* link wrapping, you are seriously
> > going to mess up my system. My short URLs are currently 18 characters
> > long, and they will be 18 long for quite some time to come. After that
> > they will be 19 for a very long time to come.
> >
> > If you implement this change, a ton, and I mean a *huge* number of my
> > system's updates are going to be rejected for being over 140
> > characters.
> >
> > On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
> >
> >
> >
> > > hi all.
> >
> > > twitter has been wrapping links in e-mailed DMs for a couple months
> > > now.
> > > with that feature, we're trying to protect users against phishing and
> other
> > > malicious attacks. the way that we're doing this is that any URL that
> comes
> > > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> turns
> > > out to be malicious, Twitter can simply shut it down, and whoever
> follows
> > > that link will be presented with a page that warns them of potentially
> > > malicious content. in a few weeks, we're going to start slowly enabling
> this
> > > throughout the API for all statuses as well, but instead of twt.tl, we
> will
> > > be using t.co.
> >
> > > practically, any tweet that is sent through statuses/update that has a
> link
> > > on it will have the link automatically converted to a t.co link on its
> way
> > > through the Twitter platform. if you fetch any tweet created after this
> > > change goes live, then its text field will have all its links
> automatically
> > > wrapped with t.co links. when a user clicks on that link, Twitter will
> > > redirect them to the original URL after first confirming with our
> database
> > > that that URL is not malicious.  on top of the end-user benefit, we
> hope to
> > > eventually provide all developers with aggregate usage data around your
> > > applications such as the number of clicks people make on URLs you
> display
> > > (it will, of course, be in aggregate and not identifiable manner).
> > > additionally, we want to be able to build services and APIs that can
> make
> > > algorithmic recommendations to users based on the content they are
> > > consuming. gathering the data from t.co will help make these possible.
> >
> > > our current plan is that no user will see a t.co URL on twitter.combut we
> > > still have some details to work through. the links will still be
> displayed
> > > as they were sent in, but the target of the link will be the t.co link
> > > instead. and, we want to provide the same ability to display original
> links
> > > to developers. we're going to use the entities attribute to make this
> > > possible.
> >
> > > let's say i send out the following tweet: "you have to check outhttp://
> dev.twitter.com!"
> >
> > > a returned (and truncated) status object may look like:
> >
> > > {
> > >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> > >   ...
> > >   "user" : {
> > > "screen_name" : "raffi",
> > > ...
> > >   },
> > >   ...
> > >   "entities" : {
> > > "urls" : [
> > >   {
> > > "url" : "http://t.co/s9gfk2d4";,
> > > "display_url" : "http://dev.twitter.com";,
> > > "indices" : [23, 43]
> > >   }
> > > ],
> > > ...
> > >   },
> > >   ...
> >
> > > }
> >
> > > two things to note: the text of the returned status object doesn't have
> the
> > > original URL and instead it has a t.co URL, and the entities block now
> has a
> > > display_url attribute associated with it. what we're hoping is that
> with
> > > this data, it should be relatively easy to create a UI where you
> replace thehttp://t.co/s9gfk2d4inthe text with the equivalent of
> >
> > > http://t.co/s9gfk2d4";>http://dev.twitter.com
> >
> > > this means the user would not see the t.co link, but we all can still
> > > provide the protection and gather data from the wrapped link. for the
> > > applications that don't choose to update, the t.co link will be shown
> (and
> > > the goal to protect users will be met). i just want to emphasize -- we
> > > really do hope that you all render the original URL, but please send
> the
> > > user through the t.co link.   if you do choose to prefetch all the
> URLs on a
> > > timeline, then, when a user actually clicks on one of the links, please
> > > still send him or her through t.co. We will be updating the TOS to
> require
> > > you to check

Re: [twitter-dev] Re: link wrapping on the API

[Raffi Krikorian]

>
> How will this affect links for third party services that clients
> handle natively, such as Twitpic (and obviously TwitLonger, which
> already has shorter dedicated short urls for its posts)?
>

that's why we are providing all the data back out in the API.  while the
tweet itself may have t.co, we do include, in the entities, the original
URL.  our hope, honestly, is that final users never have to see t.co -- we
want to provide enough data back to developers so they can create URLs that
look like

http://t.co/blahblah";>http://mycoolsite.com

all those URLs should still show through.


> What about links through bit.ly etc? Will I still be able to see the
> analytics that they provide for my links? If so, does that mean there
> will be at least two levels of redirection from the ultimate
> destination?
>

yes - we won't be touching the original URL.  all analytics that users want
to see on bit.ly will still be there.  this is what we do on URLs in DMs
right now.


>
> On Jun 8, 11:57 pm, Raffi Krikorian  wrote:
> > hi all.
> >
> > twitter has been wrapping links in e-mailed DMs for a couple months
> > now.
> > with that feature, we're trying to protect users against phishing and
> other
> > malicious attacks. the way that we're doing this is that any URL that
> comes
> > through in a DM gets currently wrapped with a twt.tl URL -- if the URL
> turns
> > out to be malicious, Twitter can simply shut it down, and whoever follows
> > that link will be presented with a page that warns them of potentially
> > malicious content. in a few weeks, we're going to start slowly enabling
> this
> > throughout the API for all statuses as well, but instead of twt.tl, we
> will
> > be using t.co.
> >
> > practically, any tweet that is sent through statuses/update that has a
> link
> > on it will have the link automatically converted to a t.co link on its
> way
> > through the Twitter platform. if you fetch any tweet created after this
> > change goes live, then its text field will have all its links
> automatically
> > wrapped with t.co links. when a user clicks on that link, Twitter will
> > redirect them to the original URL after first confirming with our
> database
> > that that URL is not malicious.  on top of the end-user benefit, we hope
> to
> > eventually provide all developers with aggregate usage data around your
> > applications such as the number of clicks people make on URLs you display
> > (it will, of course, be in aggregate and not identifiable manner).
> > additionally, we want to be able to build services and APIs that can make
> > algorithmic recommendations to users based on the content they are
> > consuming. gathering the data from t.co will help make these possible.
> >
> > our current plan is that no user will see a t.co URL on twitter.com but
> we
> > still have some details to work through. the links will still be
> displayed
> > as they were sent in, but the target of the link will be the t.co link
> > instead. and, we want to provide the same ability to display original
> links
> > to developers. we're going to use the entities attribute to make this
> > possible.
> >
> > let's say i send out the following tweet: "you have to check outhttp://
> dev.twitter.com!"
> >
> > a returned (and truncated) status object may look like:
> >
> > {
> >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> >   ...
> >   "user" : {
> > "screen_name" : "raffi",
> > ...
> >   },
> >   ...
> >   "entities" : {
> > "urls" : [
> >   {
> > "url" : "http://t.co/s9gfk2d4";,
> > "display_url" : "http://dev.twitter.com";,
> > "indices" : [23, 43]
> >   }
> > ],
> > ...
> >   },
> >   ...
> >
> > }
> >
> > two things to note: the text of the returned status object doesn't have
> the
> > original URL and instead it has a t.co URL, and the entities block now
> has a
> > display_url attribute associated with it. what we're hoping is that with
> > this data, it should be relatively easy to create a UI where you replace
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
> >
> > http://t.co/s9gfk2d4";>http://dev.twitter.com
> >
> > this means the user would not see the t.co link, but we all can still
> > provide the protection and gather data from the wrapped link. for the
> > applications that don't choose to update, the t.co link will be shown
> (and
> > the goal to protect users will be met). i just want to emphasize -- we
> > really do hope that you all render the original URL, but please send the
> > user through the t.co link.   if you do choose to prefetch all the URLs
> on a
> > timeline, then, when a user actually clicks on one of the links, please
> > still send him or her through t.co. We will be updating the TOS to
> require
> > you to check t.co and register the click.
> >
> > related to this: the way the Twitter API counts characters is going to
> > change ever so slightly. our 140 characters is now going to be define

[twitter-dev] Re: link wrapping on the API

[Dewald Pretorius]

This is not unique to me. This will be problematic for anyone who uses
a shortening service that shortens URLs to less than 20 characters.

In these cases, you are basically adding characters to the submitted
text, and then rejecting the submitted text as being too long.

On Jun 8, 8:33 pm, Dewald Pretorius  wrote:
> Raffi,
>
> I'm fine with everything up to the new 140 character count.
>
> If you count the characters *after* link wrapping, you are seriously
> going to mess up my system. My short URLs are currently 18 characters
> long, and they will be 18 long for quite some time to come. After that
> they will be 19 for a very long time to come.
>
> If you implement this change, a ton, and I mean a *huge* number of my
> system's updates are going to be rejected for being over 140
> characters.
>
> On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
>
>
>
> > hi all.
>
> > twitter has been wrapping links in e-mailed DMs for a couple months
> > now.
> > with that feature, we're trying to protect users against phishing and other
> > malicious attacks. the way that we're doing this is that any URL that comes
> > through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> > out to be malicious, Twitter can simply shut it down, and whoever follows
> > that link will be presented with a page that warns them of potentially
> > malicious content. in a few weeks, we're going to start slowly enabling this
> > throughout the API for all statuses as well, but instead of twt.tl, we will
> > be using t.co.
>
> > practically, any tweet that is sent through statuses/update that has a link
> > on it will have the link automatically converted to a t.co link on its way
> > through the Twitter platform. if you fetch any tweet created after this
> > change goes live, then its text field will have all its links automatically
> > wrapped with t.co links. when a user clicks on that link, Twitter will
> > redirect them to the original URL after first confirming with our database
> > that that URL is not malicious.  on top of the end-user benefit, we hope to
> > eventually provide all developers with aggregate usage data around your
> > applications such as the number of clicks people make on URLs you display
> > (it will, of course, be in aggregate and not identifiable manner).
> > additionally, we want to be able to build services and APIs that can make
> > algorithmic recommendations to users based on the content they are
> > consuming. gathering the data from t.co will help make these possible.
>
> > our current plan is that no user will see a t.co URL on twitter.com but we
> > still have some details to work through. the links will still be displayed
> > as they were sent in, but the target of the link will be the t.co link
> > instead. and, we want to provide the same ability to display original links
> > to developers. we're going to use the entities attribute to make this
> > possible.
>
> > let's say i send out the following tweet: "you have to check 
> > outhttp://dev.twitter.com!";
>
> > a returned (and truncated) status object may look like:
>
> > {
> >   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
> >   ...
> >   "user" : {
> >     "screen_name" : "raffi",
> >     ...
> >   },
> >   ...
> >   "entities" : {
> >     "urls" : [
> >       {
> >         "url" : "http://t.co/s9gfk2d4";,
> >         "display_url" : "http://dev.twitter.com";,
> >         "indices" : [23, 43]
> >       }
> >     ],
> >     ...
> >   },
> >   ...
>
> > }
>
> > two things to note: the text of the returned status object doesn't have the
> > original URL and instead it has a t.co URL, and the entities block now has a
> > display_url attribute associated with it. what we're hoping is that with
> > this data, it should be relatively easy to create a UI where you replace 
> > thehttp://t.co/s9gfk2d4inthe text with the equivalent of
>
> > http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> > this means the user would not see the t.co link, but we all can still
> > provide the protection and gather data from the wrapped link. for the
> > applications that don't choose to update, the t.co link will be shown (and
> > the goal to protect users will be met). i just want to emphasize -- we
> > really do hope that you all render the original URL, but please send the
> > user through the t.co link.   if you do choose to prefetch all the URLs on a
> > timeline, then, when a user actually clicks on one of the links, please
> > still send him or her through t.co. We will be updating the TOS to require
> > you to check t.co and register the click.
>
> > related to this: the way the Twitter API counts characters is going to
> > change ever so slightly. our 140 characters is now going to be defined as
> > 140 characters after link wrapping. t.co links are of a predictable length
> > -- they will always be 20 characters. after we make this live, it will be
> > feasible to send in the text for a status that is greater 

[twitter-dev] Re: link wrapping on the API

[Twitlonger]

How will this affect links for third party services that clients
handle natively, such as Twitpic (and obviously TwitLonger, which
already has shorter dedicated short urls for its posts)?

I'll also be interested to see how this goes down with the privacy
types who will now be paranoid that Twitter is tracking the sites
they're going to, even if they are going through a third party client.
If I'm clicking a link on a desktop client, should Twitter really be
getting that information?

What about links through bit.ly etc? Will I still be able to see the
analytics that they provide for my links? If so, does that mean there
will be at least two levels of redirection from the ultimate
destination?

On Jun 8, 11:57 pm, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a

[twitter-dev] Re: link wrapping on the API

[Dewald Pretorius]

Raffi,

I'm fine with everything up to the new 140 character count.

If you count the characters *after* link wrapping, you are seriously
going to mess up my system. My short URLs are currently 18 characters
long, and they will be 18 long for quite some time to come. After that
they will be 19 for a very long time to come.

If you implement this change, a ton, and I mean a *huge* number of my
system's updates are going to be rejected for being over 140
characters.

On Jun 8, 7:57 pm, Raffi Krikorian  wrote:
> hi all.
>
> twitter has been wrapping links in e-mailed DMs for a couple months
> now.
> with that feature, we're trying to protect users against phishing and other
> malicious attacks. the way that we're doing this is that any URL that comes
> through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns
> out to be malicious, Twitter can simply shut it down, and whoever follows
> that link will be presented with a page that warns them of potentially
> malicious content. in a few weeks, we're going to start slowly enabling this
> throughout the API for all statuses as well, but instead of twt.tl, we will
> be using t.co.
>
> practically, any tweet that is sent through statuses/update that has a link
> on it will have the link automatically converted to a t.co link on its way
> through the Twitter platform. if you fetch any tweet created after this
> change goes live, then its text field will have all its links automatically
> wrapped with t.co links. when a user clicks on that link, Twitter will
> redirect them to the original URL after first confirming with our database
> that that URL is not malicious.  on top of the end-user benefit, we hope to
> eventually provide all developers with aggregate usage data around your
> applications such as the number of clicks people make on URLs you display
> (it will, of course, be in aggregate and not identifiable manner).
> additionally, we want to be able to build services and APIs that can make
> algorithmic recommendations to users based on the content they are
> consuming. gathering the data from t.co will help make these possible.
>
> our current plan is that no user will see a t.co URL on twitter.com but we
> still have some details to work through. the links will still be displayed
> as they were sent in, but the target of the link will be the t.co link
> instead. and, we want to provide the same ability to display original links
> to developers. we're going to use the entities attribute to make this
> possible.
>
> let's say i send out the following tweet: "you have to check 
> outhttp://dev.twitter.com!";
>
> a returned (and truncated) status object may look like:
>
> {
>   "text" : "you have to check outhttp://t.co/s9gfk2d4!";,
>   ...
>   "user" : {
>     "screen_name" : "raffi",
>     ...
>   },
>   ...
>   "entities" : {
>     "urls" : [
>       {
>         "url" : "http://t.co/s9gfk2d4";,
>         "display_url" : "http://dev.twitter.com";,
>         "indices" : [23, 43]
>       }
>     ],
>     ...
>   },
>   ...
>
> }
>
> two things to note: the text of the returned status object doesn't have the
> original URL and instead it has a t.co URL, and the entities block now has a
> display_url attribute associated with it. what we're hoping is that with
> this data, it should be relatively easy to create a UI where you replace 
> thehttp://t.co/s9gfk2d4in the text with the equivalent of
>
> http://t.co/s9gfk2d4";>http://dev.twitter.com
>
> this means the user would not see the t.co link, but we all can still
> provide the protection and gather data from the wrapped link. for the
> applications that don't choose to update, the t.co link will be shown (and
> the goal to protect users will be met). i just want to emphasize -- we
> really do hope that you all render the original URL, but please send the
> user through the t.co link.   if you do choose to prefetch all the URLs on a
> timeline, then, when a user actually clicks on one of the links, please
> still send him or her through t.co. We will be updating the TOS to require
> you to check t.co and register the click.
>
> related to this: the way the Twitter API counts characters is going to
> change ever so slightly. our 140 characters is now going to be defined as
> 140 characters after link wrapping. t.co links are of a predictable length
> -- they will always be 20 characters. after we make this live, it will be
> feasible to send in the text for a status that is greater than 140
> characters. the rule is after the link wrapping, the text transforms to 140
> characters or fewer. we'll be using the same logic that is in
> twitter-text-rb to figure out what is a URL.
>
> look for an update to dev.twitter.com where we'll have a best practices
> document on how to use these t.co links.
>
> what's the timeline?  "soon" we'll enable this on @twitterapi, @rsarver,
> @raffi, and a few other test accounts so you all have live data to play
> with.  on the timescal