Hi there,
I am a MapReduce rookie, so I have a few questions to ask (sorry if they
are silly questions):
- What is the maximum input processing rate?
- The docs say:
- The aggregate number of entities processed per second by all
mappers. Used to prevent large amounts of quota being used up in a
short
time period.
- Say we use default (1000) - Is this the amount shared by shards?
(so 1000 / 8). Or is it the amount times the number of shards? (1000 * 8).
- What is the maximum shard count? (Default is 8).
I ask because I need to know if multiplying the input processing rate and
shard count by 10 would be a sensible idea for mapping over a couple of
billion entities. (Since using the defaults set by google I think would
take days if my estimates below are correct)
If the input processing rate is shard by shards, (if that makes sense) does
that mean if I have 3 billion entities, it would be:
3,000,000,000 / default input processing rate (1000) = 3,000,000
> 3,000,000 / 60 = 50,000 mins
> 50,000 / 60 = 833.33 hours
> 833.33 / 24 = 34.72 days
Thanks,
Mark
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine-java/-/BPeSvB2Yg5gJ.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
