Consider using a roll forward model with tasks ensuring the transactions complete.
There is an old article by Nick Johnson covering the approach and I have used it to ensure reliable completion of a set of transactions (and that was in the day of 1min task deadline) http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine T On Saturday, October 29, 2016 at 11:43:46 PM UTC+8, Kaan Soral wrote: > > My initial problem with push queues was that they were using their retries > completely internally, so once you get them to do "infinite retries", the > losses became 0% > > The problem with pull queues was throughput, sadly, pull queues are much > better aimed to battle contention, especially when used together with > > *Ps. I assume, for my original problem, manual retrying is the simplest > solution* > > What kind of a method with push queues do you use to solve the contention > issue tho? > I forgot my original method that used only push queues, my current method > uses pull queues for tasks, and push queues for processing, there are > delays, there are losses (it's always an ongoing battle), that's why with > my new project, I just kept it strictly to simple transactions > > On Saturday, October 29, 2016 at 6:19:34 PM UTC+3, PK wrote: >> >> I have found push task queues very reliable and use them all the time. >> >> PK >> http://www.gae123.com >> >> On Oct 29, 2016, at 7:29 AM, Kaan Soral <kaan...@gmail.com> wrote: >> >> Thanks for sharing retrying, indeed it's pretty interesting >> >> My situation is pretty unique, that's why there are usually 3 consecutive >> requests in the worst case and at most 5-6, that's why an architecture >> change is an overkill >> >> I had a lot of experience with both push queue and pull queue based >> solutions in the past, and I can safely say that, "better let them content" >> >> In my opinion, both pull queues and push queues are extremely unreliable, >> so on this project, I made it so that the occasional contention is safe to >> occur >> >> However, I hoped with retries=6 the transactions would untangle them on >> their own, but they don't >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengi...@googlegroups.com. >> To post to this group, send email to google-a...@googlegroups.com. >> Visit this group at https://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/64683751-e390-4f16-ad0c-ec4a94246256%40googlegroups.com >> >> <https://groups.google.com/d/msgid/google-appengine/64683751-e390-4f16-ad0c-ec4a94246256%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> >> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/3d245479-a476-4aba-8fef-d7fc8ebedaf6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
