Hi Matja, Looking at the situation again, it doesn't seem to be related to the transaction in which you are raising the tasks. I'm reading that it is when the tasks are fired that you get the problem. This seems to point very much to a problem with multiple tasks each trying to update the same entity group (in a transaction) at the same time. So there isn't a db operation when you raise the tasks, and that's fine - the tasks get raised, and is behaving as expected. The transient nature of the error also points to this - it is only when you get concurrent execution of multiple tasks on the same entity group, which may not happen very often. In this situation, I would say that the task retry behaviour is actually the result you want.
I'm sure it is the case, but I just wanted to check that you are aware that the transaction that raises the task is not the same transaction as when the running task executes a write operation. Apart from leaning on the retry behaviour, you have the option to chain multiple updates to the same entity group so they occur sequentially - i.e. raise task1, execute task1, raise task2, execute task2 - you can embed the information required for task2 inside the payload of task1. Serialising operations on the same entity group like this will guarantee that you don't get contention between task1 and task2. Cheers, Colin On Oct 29, 9:37 pm, Matija <matija.jerko...@gmail.com> wrote: > I must say that this exception happens very seldom, but we are still > at developing stage with not realistic testing data. I am not saying > that it is constant problem and maybe it is by design. It resolves > very easy because next same task execution runs successfully as it > should by the app engine task design. > > Problem is that we don't know at what task adding rate exception could > happen. Maybe instead of tree like chain of task we would create > single contiguous list with task by task execution when it is > acceptable. Especially if tasks are fighting with front web request > for same instances and add to save average latency. > > On Oct 29, 9:03 pm, Robert Kluin <robert.kl...@gmail.com> wrote: > > > > > > > > > I do not think you mentioned the fact you are using a wrapper for the > > task-queue until now. Have you tried saving a very simple entity > > inside the transaction causing a problem to verify that it is not an > > issue with the other library? > > > Even if you are not using a wrapper, I would try to save a simple > > entity and see if your error goes away. > > > Robert > > > On Fri, Oct 29, 2010 at 03:12, Matija <matija.jerko...@gmail.com> wrote: > > > Thank you for your suggestions. To be honest I know that I can put > > > them in queue via batch put, but we are using Vince Bonfanti's > > > deferred task queue helper (slightly enhanced for our internal use > > > with transactions). It has no support for batch putting, but probably > > > we could add support to it and try it. But as you probably already > > > know, entities batch put doesn't guarantee that every entity will > > > successfully be saved or none (except in transaction), we can presume > > > that tasks also would not. So problem is probably somewhere in api > > > service for task queue. > > > > But with batch put our task request would need less time to finish and > > > with less request latency we could get more instances. Hm... with more > > > instances we could get more transactional task queuing contention > > > problems. > > > > I think that it would be nice that somebody from App Engine team > > > explains how can we avoid it, if it is possible. Maybe this is app > > > engine design problem and we should not worry about it, like knowing > > > that for one put statement in let's say every 8000 (I read some > > > article) you can expect not to finish because of its distributed > > > nature. Problem is that they are quiet and because of it we suspect > > > everything. > > > > On Oct 29, 12:39 am, hawkett <hawk...@gmail.com> wrote: > > >> Yet another consideration - you need to be aware that when you raise > > >> multiple tasks in quick succession, it is possible (likely even) that > > >> they will execute concurrently. If each task is trying to update the > > >> same entity, they may very well compete with each other, and enter a > > >> contention/retry deadlock. > > > >> On Oct 28, 11:33 pm, hawkett <hawk...@gmail.com> wrote: > > > >> > One possible consideration - when you raise multiple tasks, do you do > > >> > so in a single call to the task queue API (batch put) or multiple > > >> > calls? If the latter, a great test would be to change to batch put and > > >> > see if the problem goes away. > > > >> > On Oct 28, 11:29 pm, hawkett <hawk...@gmail.com> wrote: > > > >> > > Hi Matija, > > > >> > > I can see your use case - you want to make sure that either all > > >> > > tasks get raised, or none. I can also see your worry - that tasks are > > >> > > actually being *fired* before the transaction is complete. I agree > > >> > > this would be a completely incorrect situation. I also > > >> > > transactionally > > >> > > raise multiple tasks, but I always have a datastore operation in > > >> > > conjunction (and use python) - I haven't encountered this issue of > > >> > > tasks being fired before the transaction completes, but I wonder if > > >> > > google has made the assumption that a transaction will always contain > > >> > > a datastore operation, and perhaps their logic is misbehaving if it > > >> > > does not? > > > >> > > An example of how this might happen is if their logic is waiting for > > >> > > a > > >> > > counter of outstanding db operations to reach 0 before raising tasks. > > >> > > This would be flawed logic. Thanks, > > > >> > > Colin > > > >> > > On Oct 28, 8:21 am, Matija <matija.jerko...@gmail.com> wrote: > > > >> > > > I am not saving any entities but I am putting more than one task in > > >> > > > queue. It depends of the data that current task is processing. > > >> > > > Let's > > >> > > > say two tasks (always less then five). > > > >> > > > At the end of 'transaction' I will have two tasks in queue or none. > > > >> > > > This usually works, but sometimes I get this > > >> > > > ConcurrentModificationException exception. Complete application > > >> > > > chunk > > >> > > > works eventually because task with execute again and will finish > > >> > > > correctly, but it would be nice to understand this exception so > > >> > > > that I > > >> > > > can avoid it. > > > >> > > > On Oct 28, 1:58 am, Tim Hoffman <zutes...@gmail.com> wrote: > > > >> > > > > If you are not saving any entities why are you using a > > >> > > > > transaction to > > >> > > > > put the task into the queue. > > > >> > > > > The idea of a queuing a task transactionally is that the task > > >> > > > > doesn't > > >> > > > > get fired if the transaction fails (ie you save of entities > > >> > > > > fails). > > > >> > > > > Seems completely redundant > > > >> > > > > Rgds > > > >> > > > > T > > > >> > > > > On Oct 21, 11:18 pm, Matija <matija.jerko...@gmail.com> wrote: > > > >> > > > > > Hi, > > >> > > > > > I am > > >> > > > > > usinghttp://code.google.com/appengine/docs/java/taskqueue/overview.html#Ta... > > >> > > > > > way to enqueue two tasks in transaction. But I am not saving > > >> > > > > > any > > >> > > > > > datastore entities. I am using objectify. > > > >> > > > > > I got this message: > > > >> > > > > > java.util.ConcurrentModificationException: too much contention > > >> > > > > > on > > >> > > > > > these datastore entities. please try again. > > >> > > > > > at > > >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast > > >> > > > > > oreApiHelper.java: > > >> > > > > > 39) > > >> > > > > > at > > >> > > > > > com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(Datastor > > >> > > > > > eApiHelper.java: > > >> > > > > > 75) > > >> > > > > > at > > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction > > >> > > > > > Impl.java: > > >> > > > > > 44) > > >> > > > > > at > > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(Transaction > > >> > > > > > Impl.java: > > >> > > > > > 52) > > >> > > > > > at > > >> > > > > > com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.j > > >> > > > > > ava: > > >> > > > > > 64) > > > >> > > > > > When this happened only one task needed to be added to queue, > > >> > > > > > so why > > >> > > > > > this error ? > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appeng...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com. > > > For more options, visit this group > > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
