The math looks right, but it might be missing a few additional cost items. I think you're assuming 2 writes for index updates (ascending and descending). Most likely you'll also need a composite index that includes the date so you can show the items in a reverse chronological order (assuming your feed is like twitter). So I'd say the cost is $0.06 in index write ops per update rather than $0.04.
Depending on your specific requirements, you might be able to find optimizations over the general case that could save you a lot. A few ideas: - Create your own indexes. Native indexes require 2 ops for each property + 1 op for each composite index. What if you create your own UserStream entity that contains the latest 500 updates for that user? With every new update, you fetch and update each UserStream entity. That's 1 get + 1 write per update per user, as opposed to 3 writes per update per user. Not a huge saving, but it also gives you the option to choose to not update all users streams (i.e. ignore those who haven't logged in for a while). - If you have a limit on how many sources a user can follow (say, 400 max), then you can create an index entity for each source, SourceFeed, that contains the latest, say, 100 updates from that source. Then, to generate the stream of one user, you load the 400 entities with db.get([list]), which loads them in parallel, and then generate the stream in memory. This will work only if you have a small limit on how many sources a user can follow. The cost here will be when the user views the stream rather than when an update is made. The cost will be lower only if your users don't login too often. Otherwise, the previous solutions would be better. Here, you can also choose to store the SourceFeed of popular sources in memcache to further reduce read ops. - You can use a backend instance with a lot of memory to store big parts of recent updates so that for most users you can generate the stream in memory. Waleed On Tue, Nov 8, 2011 at 9:52 PM, Herbert <herber...@gmail.com> wrote: > I'm building a system with news feed style broadcasting. there won't > be millions of users, probably just 100k and interesting feed would go > to inbox of ~10k users. > > everyone's recommend the method to solve fan-out proposed by Brett, > and i really like it, works something like this: > > class Feed(db.Model) > content = db.TextProperty() > > class Receivers(db.Model) > IDs = db.ListProperty(int) > > say an interesting Feed has five children of Receivers each with 2000 > IDs in the list, i like how it solves my problem, but I did the math > on billing: > > that's 2000 * 5 * 2 writes = 20k low-level writes right there on > creation, and cleaning up cost another 20k low-level writes. 40k low- > level writes = $0.04 for a news feed. Am I doing the math wrong, > missing something, missing the best way to handle this problem, or > this seems a reasonable cost? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
