Hi Joshua --
It’s a good question, and it has a complicated answer. Broader documentation on when to prefer one over the other is in process, and we hope that the feature set we expect to evolve in Cloud Tasks (Alpha signup here <https://goo.gl/Ya0AZd>) will make the distinctions clearer. Much of the answer is historical. Task Queues evolved as part of the App Engine developer libraries (back in 2009) and was (internally) slated for deprecation and replacement with release of Pub/Sub. As you observed: Pub/Sub does seem like a strong generalization of Task Queues. When Pub/Sub launched, however, we found that it did not meet the needs of a significant number of customers. A number of features that we thought weren’t particularly import (de-duplication, scheduled delivery, rate management, concurrency management, tagged tasks, configurable retries, etc) turned out to be critical and not supported in Pub/Sub. In response to that feedback, we took a step back and tried to define the core differences. The conclusion we arrived at is that Pub/Sub is a networking and big data product. It is agnostic to both the content and the target and focuses on delivering very high throughput capabilities to arbitrary handlers. Task Queues is a managed execution product. It is intended for running and managing large numbers of explicit commands. We’re working to repackage Task Queues as Cloud Tasks and open it up for broader integration with the rest of GCP. For example, we want to make it possible to enqueue Datastore operations or Cloud Functions for asynchronous execution. The idea is to make it as easy as possible to spin up large distributed systems without needing to write handlers and managers. An example of what we are trying to enable is the App Engine mapreduce library (written on top of Datastore and Task Queues). Instructions (in this case, map or reduce shards) are explicitly encoded and a user-space library can capitalize on Task Queues infrastructure to avoid needing any management tools or orchestration services. I hope that’s helpful and feel free to send me more details on your use case and we can provide you with more guidance. -Kim Product Manager of Task Queues On Thursday, February 2, 2017 at 7:34:00 AM UTC-8, Joshua Fox wrote: > > This Task Queues <https://cloud.google.com/appengine/docs/java/taskqueue/> > page > says "Tip: In some cases Google Cloud Pub/Sub > <https://cloud.google.com/pubsub/overview> is a good alternative to pull > queues." > > Is there a doc that says more about that? > > - In what cases are Pub/Sub or Task Queues preferable? > - Why is PubSub only an alternative for pull task-queues given that both > push- and pull-style subscriptions are supported in Pub/Sub? > - Since Pub/Sub appears more general, why do Task Queues exist? Is it just > the scheduled push feature? Or the specific support for serialization of > Task objects? > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/70d72de0-84b8-44bd-974b-df6bca6f0ead%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
