I must say that this exception happens very seldom, but we are still at developing stage with not realistic testing data. I am not saying that it is constant problem and maybe it is by design. It resolves very easy because next same task execution runs successfully as it should by the app engine task design.
Problem is that we don't know at what task adding rate exception could happen. Maybe instead of tree like chain of task we would create single contiguous list with task by task execution when it is acceptable. Especially if tasks are fighting with front web request for same instances and add to save average latency. On Oct 29, 9:03 pm, Robert Kluin <robert.kl...@gmail.com> wrote: > I do not think you mentioned the fact you are using a wrapper for the > task-queue until now. Have you tried saving a very simple entity > inside the transaction causing a problem to verify that it is not an > issue with the other library? > > Even if you are not using a wrapper, I would try to save a simple > entity and see if your error goes away. > > Robert > > > > > > > > On Fri, Oct 29, 2010 at 03:12, Matija <matija.jerko...@gmail.com> wrote: > > Thank you for your suggestions. To be honest I know that I can put > > them in queue via batch put, but we are using Vince Bonfanti's > > deferred task queue helper (slightly enhanced for our internal use > > with transactions). It has no support for batch putting, but probably > > we could add support to it and try it. But as you probably already > > know, entities batch put doesn't guarantee that every entity will > > successfully be saved or none (except in transaction), we can presume > > that tasks also would not. So problem is probably somewhere in api > > service for task queue. > > > But with batch put our task request would need less time to finish and > > with less request latency we could get more instances. Hm... with more > > instances we could get more transactional task queuing contention > > problems. > > > I think that it would be nice that somebody from App Engine team > > explains how can we avoid it, if it is possible. Maybe this is app > > engine design problem and we should not worry about it, like knowing > > that for one put statement in let's say every 8000 (I read some > > article) you can expect not to finish because of its distributed > > nature. Problem is that they are quiet and because of it we suspect > > everything. > > > On Oct 29, 12:39 am, hawkett <hawk...@gmail.com> wrote: > >> Yet another consideration - you need to be aware that when you raise > >> multiple tasks in quick succession, it is possible (likely even) that > >> they will execute concurrently. If each task is trying to update the > >> same entity, they may very well compete with each other, and enter a > >> contention/retry deadlock. > > >> On Oct 28, 11:33 pm, hawkett <hawk...@gmail.com> wrote: > > >> > One possible consideration - when you raise multiple tasks, do you do > >> > so in a single call to the task queue API (batch put) or multiple > >> > calls? If the latter, a great test would be to change to batch put and > >> > see if the problem goes away. > > >> > On Oct 28, 11:29 pm, hawkett <hawk...@gmail.com> wrote: > > >> > > Hi Matija, > > >> > > I can see your use case - you want to make sure that either all > >> > > tasks get raised, or none. I can also see your worry - that tasks are > >> > > actually being *fired* before the transaction is complete. I agree > >> > > this would be a completely incorrect situation. I also transactionally > >> > > raise multiple tasks, but I always have a datastore operation in > >> > > conjunction (and use python) - I haven't encountered this issue of > >> > > tasks being fired before the transaction completes, but I wonder if > >> > > google has made the assumption that a transaction will always contain > >> > > a datastore operation, and perhaps their logic is misbehaving if it > >> > > does not? > > >> > > An example of how this might happen is if their logic is waiting for a > >> > > counter of outstanding db operations to reach 0 before raising tasks. > >> > > This would be flawed logic. Thanks, > > >> > > Colin > > >> > > On Oct 28, 8:21 am, Matija <matija.jerko...@gmail.com> wrote: > > >> > > > I am not saving any entities but I am putting more than one task in > >> > > > queue. It depends of the data that current task is processing. Let's > >> > > > say two tasks (always less then five). > > >> > > > At the end of 'transaction' I will have two tasks in queue or none. > > >> > > > This usually works, but sometimes I get this > >> > > > ConcurrentModificationException exception. Complete application chunk > >> > > > works eventually because task with execute again and will finish > >> > > > correctly, but it would be nice to understand this exception so that > >> > > > I > >> > > > can avoid it. > > >> > > > On Oct 28, 1:58 am, Tim Hoffman <zutes...@gmail.com> wrote: > > >> > > > > If you are not saving any entities why are you using a transaction > >> > > > > to > >> > > > > put the task into the queue. > > >> > > > > The idea of a queuing a task transactionally is that the task > >> > > > > doesn't > >> > > > > get fired if the transaction fails (ie you save of entities fails). > > >> > > > > Seems completely redundant > > >> > > > > Rgds > > >> > > > > T > > >> > > > > On Oct 21, 11:18 pm, Matija <matija.jerko...@gmail.com> wrote: > > >> > > > > > Hi, > >> > > > > > I am > >> > > > > > usinghttp://code.google.com/appengine/docs/java/taskqueue/overview.html#Ta... > >> > > > > > way to enqueue two tasks in transaction. But I am not saving any > >> > > > > > datastore entities. I am using objectify. > > >> > > > > > I got this message: > > >> > > > > > java.util.ConcurrentModificationException: too much contention on > >> > > > > > these datastore entities. please try again. > >> > > > > > at > >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast > >> > > > > > oreApiHelper.java: > >> > > > > > 39) > >> > > > > > at > >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(Datastor > >> > > > > > eApiHelper.java: > >> > > > > > 75) > >> > > > > > at > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction > >> > > > > > Impl.java: > >> > > > > > 44) > >> > > > > > at > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction > >> > > > > > Impl.java: > >> > > > > > 52) > >> > > > > > at > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.j > >> > > > > > ava: > >> > > > > > 64) > > >> > > > > > When this happened only one task needed to be added to queue, so > >> > > > > > why > >> > > > > > this error ? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
