I do not think you mentioned the fact you are using a wrapper for the task-queue until now. Have you tried saving a very simple entity inside the transaction causing a problem to verify that it is not an issue with the other library?
Even if you are not using a wrapper, I would try to save a simple entity and see if your error goes away. Robert On Fri, Oct 29, 2010 at 03:12, Matija <matija.jerko...@gmail.com> wrote: > Thank you for your suggestions. To be honest I know that I can put > them in queue via batch put, but we are using Vince Bonfanti's > deferred task queue helper (slightly enhanced for our internal use > with transactions). It has no support for batch putting, but probably > we could add support to it and try it. But as you probably already > know, entities batch put doesn't guarantee that every entity will > successfully be saved or none (except in transaction), we can presume > that tasks also would not. So problem is probably somewhere in api > service for task queue. > > But with batch put our task request would need less time to finish and > with less request latency we could get more instances. Hm... with more > instances we could get more transactional task queuing contention > problems. > > I think that it would be nice that somebody from App Engine team > explains how can we avoid it, if it is possible. Maybe this is app > engine design problem and we should not worry about it, like knowing > that for one put statement in let's say every 8000 (I read some > article) you can expect not to finish because of its distributed > nature. Problem is that they are quiet and because of it we suspect > everything. > > On Oct 29, 12:39 am, hawkett <hawk...@gmail.com> wrote: >> Yet another consideration - you need to be aware that when you raise >> multiple tasks in quick succession, it is possible (likely even) that >> they will execute concurrently. If each task is trying to update the >> same entity, they may very well compete with each other, and enter a >> contention/retry deadlock. >> >> On Oct 28, 11:33 pm, hawkett <hawk...@gmail.com> wrote: >> >> >> >> >> >> >> >> > One possible consideration - when you raise multiple tasks, do you do >> > so in a single call to the task queue API (batch put) or multiple >> > calls? If the latter, a great test would be to change to batch put and >> > see if the problem goes away. >> >> > On Oct 28, 11:29 pm, hawkett <hawk...@gmail.com> wrote: >> >> > > Hi Matija, >> >> > > I can see your use case - you want to make sure that either all >> > > tasks get raised, or none. I can also see your worry - that tasks are >> > > actually being *fired* before the transaction is complete. I agree >> > > this would be a completely incorrect situation. I also transactionally >> > > raise multiple tasks, but I always have a datastore operation in >> > > conjunction (and use python) - I haven't encountered this issue of >> > > tasks being fired before the transaction completes, but I wonder if >> > > google has made the assumption that a transaction will always contain >> > > a datastore operation, and perhaps their logic is misbehaving if it >> > > does not? >> >> > > An example of how this might happen is if their logic is waiting for a >> > > counter of outstanding db operations to reach 0 before raising tasks. >> > > This would be flawed logic. Thanks, >> >> > > Colin >> >> > > On Oct 28, 8:21 am, Matija <matija.jerko...@gmail.com> wrote: >> >> > > > I am not saving any entities but I am putting more than one task in >> > > > queue. It depends of the data that current task is processing. Let's >> > > > say two tasks (always less then five). >> >> > > > At the end of 'transaction' I will have two tasks in queue or none. >> >> > > > This usually works, but sometimes I get this >> > > > ConcurrentModificationException exception. Complete application chunk >> > > > works eventually because task with execute again and will finish >> > > > correctly, but it would be nice to understand this exception so that I >> > > > can avoid it. >> >> > > > On Oct 28, 1:58 am, Tim Hoffman <zutes...@gmail.com> wrote: >> >> > > > > If you are not saving any entities why are you using a transaction to >> > > > > put the task into the queue. >> >> > > > > The idea of a queuing a task transactionally is that the task >> > > > > doesn't >> > > > > get fired if the transaction fails (ie you save of entities fails). >> >> > > > > Seems completely redundant >> >> > > > > Rgds >> >> > > > > T >> >> > > > > On Oct 21, 11:18 pm, Matija <matija.jerko...@gmail.com> wrote: >> >> > > > > > Hi, >> > > > > > I am >> > > > > > usinghttp://code.google.com/appengine/docs/java/taskqueue/overview.html#Ta... >> > > > > > way to enqueue two tasks in transaction. But I am not saving any >> > > > > > datastore entities. I am using objectify. >> >> > > > > > I got this message: >> >> > > > > > java.util.ConcurrentModificationException: too much contention on >> > > > > > these datastore entities. please try again. >> > > > > > at >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast >> > > > > > oreApiHelper.java: >> > > > > > 39) >> > > > > > at >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(Datastor >> > > > > > eApiHelper.java: >> > > > > > 75) >> > > > > > at >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction >> > > > > > Impl.java: >> > > > > > 44) >> > > > > > at >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction >> > > > > > Impl.java: >> > > > > > 52) >> > > > > > at >> > > > > > com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.j >> > > > > > ava: >> > > > > > 64) >> >> > > > > > When this happened only one task needed to be added to queue, so >> > > > > > why >> > > > > > this error ? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
