Thanks Nickolaus for sharing your experience.
I've just come across this simple yet very elegant solution by Nacho Coloma
to break through the 10m time limit of taks queues:
http://blog.extrema-sistemas.com/long-tasks-in-app-engine-for-java/
https://github.com/icoloma/queue4gae
I'm going to pl
We use BigQuery (Python) to analyse visitor and click data on followus.com
pages.
>From what we've seen, BigQuery queries scale really well over large
datasets and complex queries.
However, there is an overhead to each BigQuery query which makes even
simple queries over small datasets take a coup
Hi Gilberto,
quick question: do you think BigQuery could possibly replace the Datastore
for queries?
A big Datastore pain point is the fact for each query requires an index,
while BigQuery doesn't have this restriction.
Do you think it would be feasible for a GAE app to internally redirect
clie
Actually I just migrated my statistics job from MapReduce to BigQuery
(using the Datastore -> Cloud Storage -> BigQuery pattern) =)
I strongly recommend the book "Google BigQuery Analytics" from Jordan
Tigani and Siddartha Naidu if you plan to use or know more about BigQuery.
I got mine at I/O
Thank you very much Gilberto!
It's great to make contact with people out there who are on the same boat.
I've just been watching a series of videos on pipelines, and I'm starting
to get the pattern for big data processing that Google promotes:
Datastore -> Cloud Storage -> BigQuery
The key poin
I've used MapReduce myself for while, and I can say it to you: 100+MB of
keys means A LOT of keys at the shuffle stage. And the real limitations of
MapReduce are:
"The total size of all the instances of Mapper, InputReader, OutputWriter
and Counters must be less than 1MB between slices. This is
It all comes at a cost: increased complexity.
You can't beat the simplicity of task queues and the 10m limit seems
artificially imposed to me. I mean, we pay for CPU time, as we would pay
for 20m, 30m, 1h tasks.
I've got a simple task that takes a long time, looping through hundreds of
thousands
On Sat, Dec 6, 2014 at 5:58 AM, Maneesh Tripathi <
maneesh.tripa...@razaonline.info> wrote:
> I have Created an task queue which stop working after 10 Min.
> I want to increase the timing.
> Please help me on this
>
Task queue requests are limited to 10 minutes of execution time:
https://cloud.g
I have Created an task queue which stop working after 10 Min.
I want to increase the timing.
Please help me on this
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an
