As of today though, what are the advantages to using Pub/Sub over Pull Queues? For example, are there any time or scale limits to using Pull Queues that Pub/Sub resolves?
Thanks, Alex On Friday, February 3, 2017 at 10:30:54 AM UTC-8, Kim Lewandowski wrote: > > Hi Joshua -- > > It’s a good question, and it has a complicated answer. Broader > documentation on when to prefer one over the other is in process, and we > hope that the feature set we expect to evolve in Cloud Tasks (Alpha > signup here <https://goo.gl/Ya0AZd>) will make the distinctions clearer. > > Much of the answer is historical. Task Queues evolved as part of the App > Engine developer libraries (back in 2009) and was (internally) slated for > deprecation and replacement with release of Pub/Sub. As you observed: > Pub/Sub does seem like a strong generalization of Task Queues. When Pub/Sub > launched, however, we found that it did not meet the needs of a significant > number of customers. A number of features that we thought weren’t > particularly import (de-duplication, scheduled delivery, rate management, > concurrency management, tagged tasks, configurable retries, etc) turned out > to be critical and not supported in Pub/Sub. > > In response to that feedback, we took a step back and tried to define the > core differences. The conclusion we arrived at is that Pub/Sub is a > networking and big data product. It is agnostic to both the content and the > target and focuses on delivering very high throughput capabilities to > arbitrary handlers. Task Queues is a managed execution product. It is > intended for running and managing large numbers of explicit commands. We’re > working to repackage Task Queues as Cloud Tasks and open it up for broader > integration with the rest of GCP. For example, we want to make it possible > to enqueue Datastore operations or Cloud Functions for asynchronous > execution. The idea is to make it as easy as possible to spin up large > distributed systems without needing to write handlers and managers. > > An example of what we are trying to enable is the App Engine mapreduce > library (written on top of Datastore and Task Queues). Instructions (in > this case, map or reduce shards) are explicitly encoded and a user-space > library can capitalize on Task Queues infrastructure to avoid needing any > management tools or orchestration services. > > I hope that’s helpful and feel free to send me more details on your use > case and we can provide you with more guidance. > > -Kim > Product Manager of Task Queues > > On Thursday, February 2, 2017 at 7:34:00 AM UTC-8, Joshua Fox wrote: >> >> This Task Queues >> <https://cloud.google.com/appengine/docs/java/taskqueue/> page says "Tip: >> In some cases Google Cloud Pub/Sub >> <https://cloud.google.com/pubsub/overview> is a good alternative to pull >> queues." >> >> Is there a doc that says more about that? >> >> - In what cases are Pub/Sub or Task Queues preferable? >> - Why is PubSub only an alternative for pull task-queues given that both >> push- and pull-style subscriptions are supported in Pub/Sub? >> - Since Pub/Sub appears more general, why do Task Queues exist? Is it >> just the scheduled push feature? Or the specific support for serialization >> of Task objects? >> >> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0317a841-3cfd-4a2b-ac48-ce2c4d163111%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
