On Mar 30, 3:32 am, Jesse Stay <jesses...@gmail.com> wrote:
> On Sun, Mar 29, 2009 at 11:41 PM, Damon Clinkscales <sca...@pobox.com>wrote:
>
> > > How often does this cache update? I'm curious how accurate and reliable
> > this would be, since
> > > people are constantly modifying their social graph.
>
> > In the case of the id/screen_name thing, the data wouldn't change
> > much. Ideally, there'd be a way of forcing an update from Twitter in
> > the case of known/suspected stale data. As to keeping up with the
> > social graph, I think the current social graph methods are
> > sufficient/wonderful for that.
>
> Ah, okay - so it's not necessarily a grab of the social graph then, but
> rather a user cache. If that's what it is I have a similar-sized cache,
> assuming Twitter were to start allowing this, I could make available as
> well. I'd be really surprised if they started to allow this though.
>
> Although there is still the problem of keeping the data up-to-date. People
> change their images, location, description, Tweets, number of
> followers/friends, etc. quite often. I think Twitter could provide a cache
> of this data a lot faster than they could provide a way to easily force
> updates on stale data. It sure would be nice though - I wouldn't have to
> make as many calls out to Twitter if they had a better way to get just the
> user updates.
I think that is the point/trade off. What is the real cost to twitter
of developers making more calls for small chunks of data vs. less
calls for a bit more custom set of data? It's less http traffic but a
bigger payload. I guess it also depends on how the data is cached. As
Alex mentioned in the link above "As they are, we fetch data from a
single data store in our architecture to return the lists of IDs. In
order to provide usernames, we'd have to bog down this request by
joining together multiple sources of data." It would require a bit or
rearchitecting on their part before I think we see a compromise being
made. The major difficulty again maintaining the freshness of data
with users changing their screen names among other things. Probably
easier said than done.
It would be great if twitter did start opening up the caching of user
data to other services and perhaps provide web hooks that get fired
when that external services cache should be updated.
>
> Jesse
