Mat, It's like you have a blog and you store every word in a single Entity. Think how quickly your quotas would fill up in a few page views.
I'm considering storing multiple tweets or even the entire stream in a single Entity. This perfectly suits for the main purpose of the site as the focus is streams rather than single tweets. You're right about the RDBMS remark, as I fear for example, it would be harder to add features like 'showing all tweets in January' with this multiple tweets approach. On 9 Ocak, 01:54, Matthew Jaggard <matt...@jaggard.org.uk> wrote: > You're thinking far too much like an RDBMS user. Imagine for a moment that > querying a database is like picking up polystyrene balls. SQL Server or > whatever is like a vacuum cleaner, the datastore is like a pair of hands. > Picking up a million polystyrene balls with your hands is perfectly > possible but hopelessly inefficient. Picking up a really big piece of > polystyrene however is trivial, size being almost irrelevant. Equally, SQL > Server/vacuum cleaner will collect tiny parts from various places with ease. > > So, in your case, I'd definitely combine multiple tweets, I might even put > all of the tweets for a user in the same entity as his other details - > login, etc. See objectify and its load groups for how to do this but > keeping logical separation in Java objects. > > Mat. > On 8 Jan 2012 23:25, "Amy Unruh" <amyu+gro...@google.com> wrote: > > > > > > > > > Serdar, > > > If you are frequently pulling in and storing many users' Twitter streams, > > this might well require you to enable billing for your app eventually. > > However, during initial development and testing, you can probably reduce > > costs enough to avoid that (e.g., try turning down cron frequency). See > > also the documentation regarding billable resources (e.g. > >http://code.google.com/appengine/docs/billing.html#Billable_Resource_...). > >For example, if not all properties in your entities need to be indexed, > > you can reduce your write costs by setting some to unindexed. > > > As you comment, you ought to be able to greatly reduce the number of reads > > by using memcache. You can use appstats ( > >http://code.google.com/appengine/docs/python/tools/appstats.html) to get > > more detail on where your reads and writes are occurring. > > > On Sun, Jan 8, 2012 at 10:09 PM, Serdar <serdar...@gmail.com> wrote: > > >> My app archives and displays tweets in a simple layout, which lets > >> people easily browse older tweets of Twitter users. > > >> This is what happens in a typical user page: > > >> - Get 100 more tweets via Twitter API and save to the datastore. Each > >> tweet is stored in a single Entity. > > >> - Get 1000 (will be 200 in the real case) tweets from the datastore > >> and display. > > >> These datastore reads and writes fill the limits very very quickly. > >> Even a single user (that's me testing) fills the quotas in minutes, > >> checking one or two Twitter user's tweets. > > >> I'll use memcache for the reads and that'll help but I don't see my > >> app could serve more than 10 users a day. > > >> An idea is to save, say, 100 tweets in a single Entity but that just > >> sounds not right in terms of data structure. > > >> How would you store and display tweets (more than 100 a page) in your > >> application? (A typical visitor would like to browse some thousands of > >> tweets.) > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine for Java" group. > >> To post to this group, send email to > >> google-appengine-java@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine-java+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine-java?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-java@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
