It's pretty simple, but with a few twists.
First of all, remember that everything that Twitter does is done with
simplicity and efficiency in mind.
For the most part its just a frequency count of words over a short
time period, minus stop words, filtering out usernames (notice @foo is
never a tr
Really a database is the way to go. Any modern database should allow
you to check if a value is in there before inserting, so the same
tweet won't go in there twice. Additionally, not every user search has
to use the up to the minute results. They can go back just a little in
time (30 seconds or s
a
single tweet even, that would be somewhat awesome and very useful for me.
Thoughts?
-David Fisher
Web Ecology Project
http://webecologyproject.org
I'll vouch for Laura as being an upright person, and all of the other
oneforty people I've met are as well. They aren't going to try to
screw anyone over and I have faith that whatever they come up with
will be fair for developers and the community both.
david
If you need to search specific users why don't you use the Shadow API
and grab all of their tweets and then search them locally?
On Sep 28, 3:14 pm, Chad Etzel wrote:
> Hello,
>
> The limit is indeed 140 and most likely won't be going up any time
> soon. The reason for the limit is for performan
seful and meaningful data but now we've
gotten mostly past those issues and we are now starting to mine and
analyze other social networks as well.
Thanks,
David Fisher
Web Ecology Project
On Sep 27, 8:11 pm, Stefna wrote:
> Thank you all for the feedback.
>
> My main motivation for
I'm sorry, but the problem isn't Twitter- its your language and JSON
parser. Outputting everything as a string, when it clearly should be a
number, is inefficient and crazy.
Saying that startups can't afford 64 bit processors in systems is
crazy. Most startups I know are running on EC2 or have fa
I've done this this way:
Every time I get a user's data- I store it in my database. Doesn't
matter if its from gardenhose or a REST method. I track various
versions of it and keep all changes (good data warehousing practice).
Then, when I crawl a user's list of friends I ask the database if we
a
The rpp defaults to 15 or something if you don't specify it. Sounds
like you need to mess around and play with things a bit more.
The key to max search results isn't in paging or rpp, but in max_id.
Be careful what you ask for. Retrieval of everything available can
take a long time (hours)
dave
lul. That's like asking the Duke Nukem Forever team when they'll
release.
"When its done"
While I love the twitter dev team, I'm not sure how hard they are able
to stick to deadlines in their development, so its probably just
better to say, "the future"
On Aug 24, 10:15 am, "METROmilwaukee ..."
This is good news. No one will be getting sued over Tweet then.
Yet, keep in mind that Twitter probably *could * shut off access to
the API to any company they choose, as its their playground and their
rules. Not that they'll start doing that at all however.
On Aug 19, 1:57 pm, Sam Johnston wro
Sounds like something you should be able to do in an email to them or
with a message on your website.
On Aug 19, 1:29 pm, arawajy wrote:
> I want to invite them to follow the company on Twitter.
>
> On Aug 19, 8:25 pm, Andrew Badera wrote:
>
> > On Wed, Aug 19, 2009 at 11:07 AM, arawajy wrote:
Unless someone here is a lawyer, we should probably avoid legal
debate- consult with each our own counsels, and move on to doing what
we do best (coding).
I find these debates are often filled with FUD, misinformation,
speculation, a misunderstanding of law, etc
The easiest way to get around it i
How are some of you failing to see the difference between "Powered by
Twitter" being something they want you to do and "http://
TwitterApplication.com" is something they don't want you to do?
Why don't they want the latter? Because someone with the email of
"adultsexdatin...@googlemail.com" regis
I am wondering if this is a case of their legal department getting a
bit heavyhanded and running loose.
What they asked of you seemed fairly reasonable however, and the name
of your application doesn't seem to be the issue.
I'm glad you didn't think you were being sued :)
It seems that Twitter
Jeremy,
The problem with your logic is that you don't feel that a company can
set a ToS for how they want users to use their service. They can.
There are legitimate and non-legitimate uses of Twitter. This guy
screwed up, and overreacted. Case closed. Twitter's got him on the
naming issue and th
@Vincent
No. Do you not understand that Trademark infringements occur between
things that could be mistaken for each other or in the same industry,
diluting a brand?
A Disney film from the 1940's has what to do with a 3rd part
application for a 2006-present social network?
There is a CLEAR conne
Streaming methods are your friend. Start with the follow method, and
email the API team about getting Shadow. If your use is legit (likely)
then they'll give you Shadow. If/when you outstrip Shadow they will
likely upgrade you then.
The only caveat to streaming is that if you miss some time (down
Let's not get into a semantic argument of spam.
Spam, noise, trash followers, whatever you want to call them. They are
all annoying and not what Twitter or most users want in their
service.
I'm unsure if a company HAS to pursue every trademark infringement to
hold their trademark. Otherwise you
My guess is that Twitter will hold both Tweet and Twitter as IP, but
allow general use of Tweet. It seems to really make sense to defend
the name of their company always, and Tweet only when it is misused.
-dave
On Aug 12, 11:43 am, Dewald Pretorius wrote:
> Nick,
>
> TweetLater.com has been us
The OP should have first gotten a lawyer to look over it, instead of
freaking out hysterically here.
He's not being sued. Twitter does own the name Twitter and can
selectively choose to sue/C&D (or not sue) anyone they like who
infringes on it. You defense being "other people have registered the
g 11, 2009 at 4:09 PM, Larry Wright wrote:
> > In addition to setting a unique user-agent, I believe it was requested that
> > we set a referrer header that pointed back to a domain.
>
> > On Tue, Aug 11, 2009 at 9:30 AM, David Fisher wrote:
>
> >> While i haven
Follower churn wouldn't exist, but getting hundreds of spam emails
(about being followed) would still exist.
I've got over 12,000 emails in my inbox about being followed on
Twitter. Dozens of those are from the same users. Some weeks the same
users unfollow and refollow me nonstop to try to get me
The user agent for each search request is the same. I'm using the Ruby
Twitter API wrapper, so sending anything else with search requests
isn't possible unless that is now deprecated.
dave
On Aug 11, 10:36 am, Andrew Badera wrote:
> On Tue, Aug 11, 2009 at 10:30 AM, David
While i haven't done scientific testing of this, I was able to run up
to 3-4 instances of my search script prior at a time before it told me
to enhance my calm. Now I'm barely able to run one without hitting the
limit. I can put delays in my code to slow it down, but I'm wondering
if this is just
IDOLPeeps,
I feel you're being overly alarmist and haven't painted the situation
properly.
You can unfollow anyone you want. The issue is a quick follow and then
unfollowing if not reciprocated. You're *supposed* to follow someone
because you want to hear what they are saying, not because you wa
Yep. In my experience calling a user that doesn't exist does yield a
404. You'll have to handle the errors on this.
I guess 404 does make sense here, as the user is not found.
dave
On Aug 11, 5:40 am, Carlo Zottmann wrote:
> Hi there,
>
> I'm doing statuses/user_timeline requests via JS, and so
Yea, and we all threatened to go to Pownce.
That didn't go so well (seeing that Pownce is now dead)
dave
On Aug 9, 3:05 pm, Dossy Shiobara wrote:
> On 8/9/09 1:53 PM, David Fisher wrote:
>
> > Anyone that was around in 2006/2007 knows that Twitter was excessively
> > un
> It is like a surreal tech soap opera on this list though :-) Hey maybe I
> could pitch this to Fox .
>
> All the best
> Neil
>
DO IT! Maybe Twitter would have made a good reality show. Lots of
super-dramatic angles and queues pointing at sysadmins typing at a
console. :)
The mailing list
It will be fixed when it is. Stop complaining. We're all in the same
boat. The ETA is "when its done".
dave
On Aug 9, 1:08 pm, freefall wrote:
> Decentralisation.
>
> On Aug 9, 5:47 pm, Dewald Pretorius wrote:
>
> > I wonder how many times this weekend has Chad heard, "FO, we're busy,"
> > whe
A few of you are acting like real children and a few of you still have
your heads screwed on right.
I'm confident they are doing everything they can. Chill and enjoy your
weekend. They'll get it sorted out.
What did you guys do in 2007? Twitter was down all the time then. Your
blood pressure mus
ll at once? No, they can't. It doesn't
make sense. Stop bitching and complaining. You're making yourselves
look unprofessional and bad.
-David Fisher
Web Ecology Project
http://webecologyproject.org
On Aug 9, 2:51 am, chinaski007 wrote:
> And, by the way, if you're a deckha
They haven't overreacted. If you think you can do better, then apply
for a job with them. Have you tried your account from multiple IP
blocks?
They have faced a crushing attack. They are working on it and they
have worked hard to communicate with us the status of things.
Anyone that was around i
but if it's John Nunemaker's, I believe it does follow redirects.
> Larry Wright/@larrywright
>
>
>
> On Fri, Aug 7, 2009 at 2:03 PM, David Fisher wrote:
>
> > I can't be sure if my client is following redirects. Probably not. I'm
> > just usin
I can't be sure if my client is following redirects. Probably not. I'm
just using the Ruby Twitter Gem which haven't been updated for a month
or so I think
dave
On Aug 7, 1:15 pm, lucasnicolato wrote:
> im having the same problem. im just lucky my app is still in test.
>
> RT @twitter Due to de
If you're getting that then you're getting more than me.
I'm just doing:
require 'rubygems'
gem 'twitter'
require 'twitter'
Twitter::Search.new('foo').each do |r|
puts r.inspect
end
And I get only this back now:
/var/lib/gems/1.8/gems/httparty-0.4.3/lib/httparty/request.rb:56:in
`setup_raw_
Is the Search API being effected? I thought at first that I had messed
up my code, but I rolled back pretty far and I'm still getting really
odd errors
/var/lib/gems/1.8/gems/httparty-0.4.3/lib/httparty/request.rb:56:in
`setup_raw_request': undefined method `request_uri' for # (NoMethodError)
I don't think that adding more people to the staff at Twitter is the
solution. In one startup I saw a thing posted on the refrigerator that
had the adage, "Adding more people to a project already behind
schedule will only slow it down more". Surely for support and customer
service issues having mo
I would do anything (including paying good amounts of money) to be
able to purchase access to older datasets that I could transfer to my
database through non-rest-api methods. I'm envisioning being able to
download a CSV or SQL file that I could merge with my database easily,
but only have to make
I am a bit concerned. I remember at one point it being between 30-45
days. Now it seems to be getting smaller by about 1-day per month.
Last month it was closer to 10 days.
Is it basically going to keep getting smaller and smaller until we get
V2 of the API, or will we be forced to all use only s
"What gives Twitter the right to dictate who you want to follow or
not?"
Its their service. They can dictate what they want. Their playground,
their rules. The ToS clearly says they can alter their terms at any
time and if you don't want to comply you can leave.
That being said, this is to preve
I can't seem to find a LOLCODE (http://en.wikipedia.org/wiki/LOLCODE)
API wrapper for Twitter. I was hoping to write my next Twitter
application in LOLCODE, but its very hard to do so without one.
Can anyone help me get started with a basic program? My code so far
isn't working
HAI
CAN HAS STDIO
Show me these killer companies doing great NLP with social networks. I
find the ones that are doing stuff right now themselves are far behind
the curve and not really pushing stuff to the edge. They are often
marketing companies that have hired one NLP guy (and underpaid them)
and are just pushing
Why don't you just do the filtering on your end? Twitter's API's job
is to give you data- what you do with filtering it on your end should
be up to you...
On Jul 17, 12:43 pm, Steve Brunton wrote:
> On Fri, Jul 17, 2009 at 12:14 PM, Cameron Kaiser wrote:
>
> > (IDNSOWFT)
> > The problem with thi
Don't feed the trolls. Name a Venture backed startup that DOESNT run
on VC for the first few years. Ok? Right. Stupid topic. Lets move on
and talk about things that matter (ie. development)
On Jul 16, 4:34 pm, Andrew Badera wrote:
> On Thu, Jul 16, 2009 at 4:26 PM, Nick Arnett wrote:
>
> > On T
ing simple paging. I've been able to pull 2M+ results for
a query before. It took 8 hours or so, but it worked. Read up more on
the Search API and you should be able to figure it out.
-David Fisher
http://WebecologyProject.org
On Jul 9, 8:16 pm, John Kalucki wrote:
> First, I wouldn't ex
Thanks for the replies Doug.
btw, as far as I can tell the ~ 52M integer values for user ids aren't
fully sequential, with some amount missing in between, which I'm
assuming is intentionally done to make it slightly more difficult to
know the exact user growth trends/patterns (seems valid to me)
If its within the window that twitter offers via search, you can go
back via a mixture of using the max_id, a rpp of 100, and paging.
However there is almost no way to verify that the oldest one is truely
the first. What you really need is Firehose, and then constantly log
everything to be able t
Topics don't just trend because its something 'important'. Now if it
was of significantly larger volume than another topics (like the
iphone's launch today), then that is rather interesting, but from what
I can tell its mostly the most popular things floating to the top
generally, plus some spam-f
Has anyone had any luck getting Brian Mario's Yajl gem working?
http://github.com/brianmario/yajl-ruby/tree/master
It seems to be breaking for me, but my code was working prior really
well. I tried to patch it, and rebuild the gem, but I was getting
errors installing.
This is critical for me to
Gardenhose is around what... 1.5M tweets/day? I think that's about
right. Yes, that is a LOT of data and going through it then takes a
lot. My server started crying a bit when I switched up to gardenhose.
I'm going to have to get a bigger one if I ever can consume the
firehose
On Jun 11, 5:04 pm,
I'm just using a realtime json parser in Ruby written as a native C
extension (http://github.com/brianmario/yajl-ruby/tree/master)
It's really simple to use and well documented.
I'm just storing everything in a Postgres database, and then using
other scripts to query it. Note: using gardenhose at
If I have Gardenhose access, do I automatically have access to Shadow
or Birddog or do I need to send in a separate application/contract to
gain access?
Thanks,
David
On Jun 11, 12:09 pm, John Kalucki wrote:
> Let me restate: If you are trying to follow many prolific users via
> the REST API,
They had one the other week:
http://twtrcon.com/
-David Fisher
@tibbon
On Jun 8, 2:26 pm, Emrah wrote:
> Hi everybody,
>
> Is there any planned Twitter Meetup in the near future? Is there any
> official schedule available online?
> I heard about #140conf but I was imagining
Can't you get more than 1500 by using Max_id and ID together and
creatively?
-David
@tibbon
On Jun 6, 2:41 pm, Doug Williams wrote:
> See [1] for information on replication. Sometimes there is a lag in the
> replication of the database across multiple machines.
>
> 1.http://en.wikipedia.org/wik
I'm hoping that Twitter counts "users" when reporting their numbers,
and not "accounts". The reason being that I've signed up probably... 5
accounts myself (main, API testing, business, etc, etc). I'm not sure
how many the average user signs up, but it's definitely on average
more than 1.
I'm won
http://snippets.dzone.com/posts/show/3701
Gets it done in Ruby. I haven't tested it, but it looks right and
simple
On Jun 4, 4:04 pm, Mark wrote:
> I would like to take the follower .xml output and convert to csv.
>
> From googling I believe I need to create an XSL or XSD and then use
> MSXML
>
57 matches
Mail list logo
