Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Jeff Schnitzer]

A bit of a blast from the past on this thread, but...

I've always had a trickle of serialization problems in production
using the Deferred servlet.  Finally I had one that was consistent and
repeatable.  I forced it to base64 encode always (not just dev mode)
and now it works consistently.

FYI.

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: any plans for deferred.defer in Java?

[David Chandler]

John,

I think my earlier serialization error problems in production were
because I was unnwittingly using an old version of GAE (1.2.6). Vince
was never able to reproduce the issue in production. I believe
everything is working in dev and prod as of 1.2.8. Give it a whirl and
post your feedback.

/dmc
http://turbomanage.wordpress.com

On Jan 1, 1:42 am, John Howe  wrote:
> I greatly appreciate the work that has been done here. But let me ask my
> question another way. Is it known that this does not work?
>
> On Thu, Dec 31, 2009 at 5:19 PM, John Howe  wrote:
> > I can't seem to find any additional postings on this topic and I'm not
> > sure what the conclusion for being able to use the Deferred capability
> > with the latest Java SDK release. I'm just wondering if Vince's
> > solution works on both server and development environments.
>
> > On Dec 16, 10:42 am, David Chandler  wrote:
> > > Thanks, Vince.
>
> > > Regarding a way to access the Guice injector, I've figured out I can
> > > simply create and access the injector via a static factory, so no
> > > changes are needed to Deferred or Deferrable. This works for AppEngine
> > > since there is only one WAR per JVM. If there were more than web app,
> > > I'm not sure whether Guice ServletModules would work, but that's not
> > > an issue for now.
>
> > > /dmchttp://turbomanage.wordpress.com
>
> > > On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
>
> > > > The behavior you're reporting is exactly the opposite of what I'm
> > seeing.
> > > > For me, deserialization always fails on the dev server and always works
> > in
> > > > production. What I'm seeing on the dev server is that the bytes
> > retrieved
> > > > from the task payload (req.getInputStream) are not the same bytes that
> > are
> > > > sent; the result is the StreamCorruptedException. Again, though, I only
> > see
> > > > this on the dev server and never in production. Can you send me an
> > example
> > > > that fails in production? I'd like to understand what's going on before
> > > > applying the Base64 encoder workaround.
>
> > > > I'm not at all familiar with Guice, but I think I understand what
> > you're
> > > > trying to do. One thought I've had is that we could pass the
> > > > HttpServletRequest object to the doTask() method. This would allow you
> > to
> > > > add attributes (objects) to the HttpServletRequest within your Guice
> > filter
> > > > and then retrieve them within your doTask() method.
>
> > > > Vince
>
> > > > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler  > >wrote:
>
> > > > > I've solved the serialization problems by Base64 encoding the
> > > > > serialized task before calling payload() and decoding it in the
> > > > > deserialize(HttpServletRequest) method. I'm guessing something in the
> > > > > task queue chain (either task queue payload storage or the servlet
> > > > > call when task is run) has problems transmitting the binary data, as
> > > > > the exception I was getting at one point was a
> > > > > java.io.StreamCorruptedException.
>
> > > > > I'll send another patch to Vince. My solution works, but I'm not
> > > > > completely satisfied that I've diagnosed the problem correctly. I
> > used
> > > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc
> > > > > instead? I see it in appengine.repackaged chain, as well, but don't
> > > > > see it as part of an official Google API.
>
> > > > > Somewhat related, I've wired up the Deferred servlet through the
> > > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> > > > > the task objects are not created through Guice, but rather by
> > > > > deserializing, I modified doTask() to accept an additional Injector
> > > > > argument. We probably don't want that in the official version of
> > > > > Deferrable, though.
>
> > > > > Is there a better way for my Deferrable classes to get access to a
> > > > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> > > > > results in a duplicate PersistenceManagerFactory being created since
> > > > > other servlets are also being served through Guice.
>
> > > > > Thank you,
> > > > > /dmc
>
> > > > > On Nov 27, 5:42 pm, David Chandler  wrote:
> > > > > > Jeff,
>
> > > > > > I'm seeing problems with deserialization, too, when deployed on
> > > > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > > > > > just fine, but not so in AppEngine. I get the same error whether
> > the
> > > > > > task payload is the serialized Deferrable task itself or just the
> > Key
> > > > > > with the task in the db, but I haven't yet figured out which is the
> > > > > > cause of the problem.
>
> > > > > > The error I'm getting on AppEngine is invalid type code: 00 in
> > > > > > Deferred.deserialize method. I've tried Base64 encoding after
> > > > > > serialization in case it's related to special binary escape
> > sequences
> > > > > > that happen to be in the serialized object stream but still no
> > > > > > success.
>
> > > > > 

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Issue #2097 still exists on the development server; however, I just
committed a workaround. This workaround has a dependency on Apache Commons
Codec:

   http://commons.apache.org/codec/

I'm not aware of any issues in production. For convenience, here's the link
again to the source:


http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

Let
me know if you have any problems (or if you're using it
successfully--positive feedback is always welcomed!).

Vince

On Fri, Jan 1, 2010 at 1:42 AM, John Howe  wrote:

> I greatly appreciate the work that has been done here. But let me ask my
> question another way. Is it known that this does not work?
>
>
> On Thu, Dec 31, 2009 at 5:19 PM, John Howe  wrote:
>
>> I can't seem to find any additional postings on this topic and I'm not
>> sure what the conclusion for being able to use the Deferred capability
>> with the latest Java SDK release. I'm just wondering if Vince's
>> solution works on both server and development environments.
>>
>> On Dec 16, 10:42 am, David Chandler  wrote:
>> > Thanks, Vince.
>> >
>> > Regarding a way to access the Guice injector, I've figured out I can
>> > simply create and access the injector via a static factory, so no
>> > changes are needed to Deferred or Deferrable. This works for AppEngine
>> > since there is only one WAR per JVM. If there were more than web app,
>> > I'm not sure whether Guice ServletModules would work, but that's not
>> > an issue for now.
>> >
>> > /dmchttp://turbomanage.wordpress.com
>> >
>> > On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
>> >
>> >
>> >
>> > > The behavior you're reporting is exactly the opposite of what I'm
>> seeing.
>> > > For me, deserialization always fails on the dev server and always
>> works in
>> > > production. What I'm seeing on the dev server is that the bytes
>> retrieved
>> > > from the task payload (req.getInputStream) are not the same bytes that
>> are
>> > > sent; the result is the StreamCorruptedException. Again, though, I
>> only see
>> > > this on the dev server and never in production. Can you send me an
>> example
>> > > that fails in production? I'd like to understand what's going on
>> before
>> > > applying the Base64 encoder workaround.
>> >
>> > > I'm not at all familiar with Guice, but I think I understand what
>> you're
>> > > trying to do. One thought I've had is that we could pass the
>> > > HttpServletRequest object to the doTask() method. This would allow you
>> to
>> > > add attributes (objects) to the HttpServletRequest within your Guice
>> filter
>> > > and then retrieve them within your doTask() method.
>> >
>> > > Vince
>> >
>> > > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <
>> turboman...@gmail.com>wrote:
>> >
>> > > > I've solved the serialization problems by Base64 encoding the
>> > > > serialized task before calling payload() and decoding it in the
>> > > > deserialize(HttpServletRequest) method. I'm guessing something in
>> the
>> > > > task queue chain (either task queue payload storage or the servlet
>> > > > call when task is run) has problems transmitting the binary data, as
>> > > > the exception I was getting at one point was a
>> > > > java.io.StreamCorruptedException.
>> >
>> > > > I'll send another patch to Vince. My solution works, but I'm not
>> > > > completely satisfied that I've diagnosed the problem correctly. I
>> used
>> > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc
>> > > > instead? I see it in appengine.repackaged chain, as well, but don't
>> > > > see it as part of an official Google API.
>> >
>> > > > Somewhat related, I've wired up the Deferred servlet through the
>> > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks.
>> Since
>> > > > the task objects are not created through Guice, but rather by
>> > > > deserializing, I modified doTask() to accept an additional Injector
>> > > > argument. We probably don't want that in the official version of
>> > > > Deferrable, though.
>> >
>> > > > Is there a better way for my Deferrable classes to get access to a
>> > > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
>> > > > results in a duplicate PersistenceManagerFactory being created since
>> > > > other servlets are also being served through Guice.
>> >
>> > > > Thank you,
>> > > > /dmc
>> >
>> > > > On Nov 27, 5:42 pm, David Chandler  wrote:
>> > > > > Jeff,
>> >
>> > > > > I'm seeing problems with deserialization, too, when deployed on
>> > > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
>> > > > > just fine, but not so in AppEngine. I get the same error whether
>> the
>> > > > > task payload is the serialized Deferrable task itself or just the
>> Key
>> > > > > with the task in the db, but I haven't yet figured out which is
>> the
>> > > > > cause of the problem.
>> >
>> > > > > The error I'm

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[John Howe]

I greatly appreciate the work that has been done here. But let me ask my
question another way. Is it known that this does not work?

On Thu, Dec 31, 2009 at 5:19 PM, John Howe  wrote:

> I can't seem to find any additional postings on this topic and I'm not
> sure what the conclusion for being able to use the Deferred capability
> with the latest Java SDK release. I'm just wondering if Vince's
> solution works on both server and development environments.
>
> On Dec 16, 10:42 am, David Chandler  wrote:
> > Thanks, Vince.
> >
> > Regarding a way to access the Guice injector, I've figured out I can
> > simply create and access the injector via a static factory, so no
> > changes are needed to Deferred or Deferrable. This works for AppEngine
> > since there is only one WAR per JVM. If there were more than web app,
> > I'm not sure whether Guice ServletModules would work, but that's not
> > an issue for now.
> >
> > /dmchttp://turbomanage.wordpress.com
> >
> > On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
> >
> >
> >
> > > The behavior you're reporting is exactly the opposite of what I'm
> seeing.
> > > For me, deserialization always fails on the dev server and always works
> in
> > > production. What I'm seeing on the dev server is that the bytes
> retrieved
> > > from the task payload (req.getInputStream) are not the same bytes that
> are
> > > sent; the result is the StreamCorruptedException. Again, though, I only
> see
> > > this on the dev server and never in production. Can you send me an
> example
> > > that fails in production? I'd like to understand what's going on before
> > > applying the Base64 encoder workaround.
> >
> > > I'm not at all familiar with Guice, but I think I understand what
> you're
> > > trying to do. One thought I've had is that we could pass the
> > > HttpServletRequest object to the doTask() method. This would allow you
> to
> > > add attributes (objects) to the HttpServletRequest within your Guice
> filter
> > > and then retrieve them within your doTask() method.
> >
> > > Vince
> >
> > > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler  >wrote:
> >
> > > > I've solved the serialization problems by Base64 encoding the
> > > > serialized task before calling payload() and decoding it in the
> > > > deserialize(HttpServletRequest) method. I'm guessing something in the
> > > > task queue chain (either task queue payload storage or the servlet
> > > > call when task is run) has problems transmitting the binary data, as
> > > > the exception I was getting at one point was a
> > > > java.io.StreamCorruptedException.
> >
> > > > I'll send another patch to Vince. My solution works, but I'm not
> > > > completely satisfied that I've diagnosed the problem correctly. I
> used
> > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc
> > > > instead? I see it in appengine.repackaged chain, as well, but don't
> > > > see it as part of an official Google API.
> >
> > > > Somewhat related, I've wired up the Deferred servlet through the
> > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> > > > the task objects are not created through Guice, but rather by
> > > > deserializing, I modified doTask() to accept an additional Injector
> > > > argument. We probably don't want that in the official version of
> > > > Deferrable, though.
> >
> > > > Is there a better way for my Deferrable classes to get access to a
> > > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> > > > results in a duplicate PersistenceManagerFactory being created since
> > > > other servlets are also being served through Guice.
> >
> > > > Thank you,
> > > > /dmc
> >
> > > > On Nov 27, 5:42 pm, David Chandler  wrote:
> > > > > Jeff,
> >
> > > > > I'm seeing problems with deserialization, too, when deployed on
> > > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > > > > just fine, but not so in AppEngine. I get the same error whether
> the
> > > > > task payload is the serialized Deferrable task itself or just the
> Key
> > > > > with the task in the db, but I haven't yet figured out which is the
> > > > > cause of the problem.
> >
> > > > > The error I'm getting on AppEngine is invalid type code: 00 in
> > > > > Deferred.deserialize method. I've tried Base64 encoding after
> > > > > serialization in case it's related to special binary escape
> sequences
> > > > > that happen to be in the serialized object stream but still no
> > > > > success.
> >
> > > > > I'm pretty sure AppEngine has written some custom code to do
> > > > > serialization because I previously got an AppEngine acess control
> > > > > exception when calling enableResolveObject(true) in a subclass of
> > > > > ObjectInputStream, and I'm wondering if that's in play somehow...
> >
> > > > > /dmc
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googleg

[appengine-java] Re: any plans for deferred.defer in Java?

[John Howe]

I can't seem to find any additional postings on this topic and I'm not
sure what the conclusion for being able to use the Deferred capability
with the latest Java SDK release. I'm just wondering if Vince's
solution works on both server and development environments.

On Dec 16, 10:42 am, David Chandler  wrote:
> Thanks, Vince.
>
> Regarding a way to access the Guice injector, I've figured out I can
> simply create and access the injector via a static factory, so no
> changes are needed to Deferred or Deferrable. This works for AppEngine
> since there is only one WAR per JVM. If there were more than web app,
> I'm not sure whether Guice ServletModules would work, but that's not
> an issue for now.
>
> /dmchttp://turbomanage.wordpress.com
>
> On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
>
>
>
> > The behavior you're reporting is exactly the opposite of what I'm seeing.
> > For me, deserialization always fails on the dev server and always works in
> > production. What I'm seeing on the dev server is that the bytes retrieved
> > from the task payload (req.getInputStream) are not the same bytes that are
> > sent; the result is the StreamCorruptedException. Again, though, I only see
> > this on the dev server and never in production. Can you send me an example
> > that fails in production? I'd like to understand what's going on before
> > applying the Base64 encoder workaround.
>
> > I'm not at all familiar with Guice, but I think I understand what you're
> > trying to do. One thought I've had is that we could pass the
> > HttpServletRequest object to the doTask() method. This would allow you to
> > add attributes (objects) to the HttpServletRequest within your Guice filter
> > and then retrieve them within your doTask() method.
>
> > Vince
>
> > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler 
> > wrote:
>
> > > I've solved the serialization problems by Base64 encoding the
> > > serialized task before calling payload() and decoding it in the
> > > deserialize(HttpServletRequest) method. I'm guessing something in the
> > > task queue chain (either task queue payload storage or the servlet
> > > call when task is run) has problems transmitting the binary data, as
> > > the exception I was getting at one point was a
> > > java.io.StreamCorruptedException.
>
> > > I'll send another patch to Vince. My solution works, but I'm not
> > > completely satisfied that I've diagnosed the problem correctly. I used
> > > Apache commons-codec for Base64. Is it safe to use com.sun.misc
> > > instead? I see it in appengine.repackaged chain, as well, but don't
> > > see it as part of an official Google API.
>
> > > Somewhat related, I've wired up the Deferred servlet through the
> > > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> > > the task objects are not created through Guice, but rather by
> > > deserializing, I modified doTask() to accept an additional Injector
> > > argument. We probably don't want that in the official version of
> > > Deferrable, though.
>
> > > Is there a better way for my Deferrable classes to get access to a
> > > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> > > results in a duplicate PersistenceManagerFactory being created since
> > > other servlets are also being served through Guice.
>
> > > Thank you,
> > > /dmc
>
> > > On Nov 27, 5:42 pm, David Chandler  wrote:
> > > > Jeff,
>
> > > > I'm seeing problems with deserialization, too, when deployed on
> > > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > > > just fine, but not so in AppEngine. I get the same error whether the
> > > > task payload is the serialized Deferrable task itself or just the Key
> > > > with the task in the db, but I haven't yet figured out which is the
> > > > cause of the problem.
>
> > > > The error I'm getting on AppEngine is invalid type code: 00 in
> > > > Deferred.deserialize method. I've tried Base64 encoding after
> > > > serialization in case it's related to special binary escape sequences
> > > > that happen to be in the serialized object stream but still no
> > > > success.
>
> > > > I'm pretty sure AppEngine has written some custom code to do
> > > > serialization because I previously got an AppEngine acess control
> > > > exception when calling enableResolveObject(true) in a subclass of
> > > > ObjectInputStream, and I'm wondering if that's in play somehow...
>
> > > > /dmc

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: any plans for deferred.defer in Java?

[David Chandler]

Thanks, Vince.

Regarding a way to access the Guice injector, I've figured out I can
simply create and access the injector via a static factory, so no
changes are needed to Deferred or Deferrable. This works for AppEngine
since there is only one WAR per JVM. If there were more than web app,
I'm not sure whether Guice ServletModules would work, but that's not
an issue for now.

/dmc
http://turbomanage.wordpress.com

On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
> The behavior you're reporting is exactly the opposite of what I'm seeing.
> For me, deserialization always fails on the dev server and always works in
> production. What I'm seeing on the dev server is that the bytes retrieved
> from the task payload (req.getInputStream) are not the same bytes that are
> sent; the result is the StreamCorruptedException. Again, though, I only see
> this on the dev server and never in production. Can you send me an example
> that fails in production? I'd like to understand what's going on before
> applying the Base64 encoder workaround.
>
> I'm not at all familiar with Guice, but I think I understand what you're
> trying to do. One thought I've had is that we could pass the
> HttpServletRequest object to the doTask() method. This would allow you to
> add attributes (objects) to the HttpServletRequest within your Guice filter
> and then retrieve them within your doTask() method.
>
> Vince
>
> On Fri, Nov 27, 2009 at 7:53 PM, David Chandler wrote:
>
> > I've solved the serialization problems by Base64 encoding the
> > serialized task before calling payload() and decoding it in the
> > deserialize(HttpServletRequest) method. I'm guessing something in the
> > task queue chain (either task queue payload storage or the servlet
> > call when task is run) has problems transmitting the binary data, as
> > the exception I was getting at one point was a
> > java.io.StreamCorruptedException.
>
> > I'll send another patch to Vince. My solution works, but I'm not
> > completely satisfied that I've diagnosed the problem correctly. I used
> > Apache commons-codec for Base64. Is it safe to use com.sun.misc
> > instead? I see it in appengine.repackaged chain, as well, but don't
> > see it as part of an official Google API.
>
> > Somewhat related, I've wired up the Deferred servlet through the
> > GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> > the task objects are not created through Guice, but rather by
> > deserializing, I modified doTask() to accept an additional Injector
> > argument. We probably don't want that in the official version of
> > Deferrable, though.
>
> > Is there a better way for my Deferrable classes to get access to a
> > Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> > results in a duplicate PersistenceManagerFactory being created since
> > other servlets are also being served through Guice.
>
> > Thank you,
> > /dmc
>
> > On Nov 27, 5:42 pm, David Chandler  wrote:
> > > Jeff,
>
> > > I'm seeing problems with deserialization, too, when deployed on
> > > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > > just fine, but not so in AppEngine. I get the same error whether the
> > > task payload is the serialized Deferrable task itself or just the Key
> > > with the task in the db, but I haven't yet figured out which is the
> > > cause of the problem.
>
> > > The error I'm getting on AppEngine is invalid type code: 00 in
> > > Deferred.deserialize method. I've tried Base64 encoding after
> > > serialization in case it's related to special binary escape sequences
> > > that happen to be in the serialized object stream but still no
> > > success.
>
> > > I'm pretty sure AppEngine has written some custom code to do
> > > serialization because I previously got an AppEngine acess control
> > > exception when calling enableResolveObject(true) in a subclass of
> > > ObjectInputStream, and I'm wondering if that's in play somehow...
>
> > > /dmc

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

The behavior you're reporting is exactly the opposite of what I'm seeing.
For me, deserialization always fails on the dev server and always works in
production. What I'm seeing on the dev server is that the bytes retrieved
from the task payload (req.getInputStream) are not the same bytes that are
sent; the result is the StreamCorruptedException. Again, though, I only see
this on the dev server and never in production. Can you send me an example
that fails in production? I'd like to understand what's going on before
applying the Base64 encoder workaround.

I'm not at all familiar with Guice, but I think I understand what you're
trying to do. One thought I've had is that we could pass the
HttpServletRequest object to the doTask() method. This would allow you to
add attributes (objects) to the HttpServletRequest within your Guice filter
and then retrieve them within your doTask() method.

Vince

On Fri, Nov 27, 2009 at 7:53 PM, David Chandler wrote:

> I've solved the serialization problems by Base64 encoding the
> serialized task before calling payload() and decoding it in the
> deserialize(HttpServletRequest) method. I'm guessing something in the
> task queue chain (either task queue payload storage or the servlet
> call when task is run) has problems transmitting the binary data, as
> the exception I was getting at one point was a
> java.io.StreamCorruptedException.
>
> I'll send another patch to Vince. My solution works, but I'm not
> completely satisfied that I've diagnosed the problem correctly. I used
> Apache commons-codec for Base64. Is it safe to use com.sun.misc
> instead? I see it in appengine.repackaged chain, as well, but don't
> see it as part of an official Google API.
>
> Somewhat related, I've wired up the Deferred servlet through the
> GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> the task objects are not created through Guice, but rather by
> deserializing, I modified doTask() to accept an additional Injector
> argument. We probably don't want that in the official version of
> Deferrable, though.
>
> Is there a better way for my Deferrable classes to get access to a
> Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> results in a duplicate PersistenceManagerFactory being created since
> other servlets are also being served through Guice.
>
> Thank you,
> /dmc
>
> On Nov 27, 5:42 pm, David Chandler  wrote:
> > Jeff,
> >
> > I'm seeing problems with deserialization, too, when deployed on
> > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > just fine, but not so in AppEngine. I get the same error whether the
> > task payload is the serialized Deferrable task itself or just the Key
> > with the task in the db, but I haven't yet figured out which is the
> > cause of the problem.
> >
> > The error I'm getting on AppEngine is invalid type code: 00 in
> > Deferred.deserialize method. I've tried Base64 encoding after
> > serialization in case it's related to special binary escape sequences
> > that happen to be in the serialized object stream but still no
> > success.
> >
> > I'm pretty sure AppEngine has written some custom code to do
> > serialization because I previously got an AppEngine acess control
> > exception when calling enableResolveObject(true) in a subclass of
> > ObjectInputStream, and I'm wondering if that's in play somehow...
> >
> > /dmc
> >
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I just committed the following changes:

  - Added additional overloads to the defer() method to allow optional
TaskOptions to be specified. This lets you set the following task options:
url, countdownMIllis, etaMillis, taskName.

  - Added support for optional "queueName" and "taskUrl" init parameters.

  - The defer() method and its overloads no longer throw IOException;
instead, an unchecked QueueFailureException is thrown if there's a failure
to serialize the task object (Deferrable instance).

  - The defer() method and its overloads now return a TaskHandle.

  - A stacktrace is now logged upon failure to deserialize the task.

  - Updated javadoc comments.

For convenience, here's the direct link to the source:

http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

As always, looking forward to comments and feedback.

Vince

On Thu, Nov 26, 2009 at 5:39 AM, Nick Johnson (Google) <
nick.john...@google.com> wrote:

> Hi Vince,
>
> I haven't had a chance to integrate your code yet, so feel free to continue
> making modifications. In particular, I think it would be useful to provide
> the same set of options the Python deferred API accepts, which include 'eta'
> and 'countdown' parameters, as well as specifying the queue and URL.
>
> -Nick
>
> On Wed, Nov 25, 2009 at 7:03 PM, Vince Bonfanti wrote:
>
>> Hi Jeff,
>>
>> Thanks for the suggestions and the code. David Chandler sent me a patch to
>> support user-specified queue names (almost exactly the same as your
>> changes), and I've committed that patch to SVN. Regarding your other
>> changes:
>>
>>   - I've probably make the  init parameter optional and have
>> it default to "/_ah/" rather than throw an exception from the
>> init method if it's missing.
>>
>>   - While we're at it, the default queue name could be optionally
>> specified by an init parameter also.
>>
>>   - I'm not sure a stacktrace from the doPost method is really going to
>> give you much more information, but would be open to that  change.
>>
>> It looks like the Deferred class is going to be added to the official
>> GAE/J SDK, though I'm not sure when. Rather than make any more changes to
>> the code right now, I'd rather wait to see what shows up the SDK and then
>> work through normal Google channels to get any further modifications made.
>> In the mean time, I'm glad it's working for you.
>>
>> Vince
>>
>> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer wrote:
>>
>>> Attached is a modified version of that class that lets you define any
>>> path you want for the servlet and lets you specify which queue to use
>>> like this: Deferred.defer(task, "queueName");
>>>
>>> (I needed this for my own purposes)
>>>
>>> Do with it as you wish :-)
>>>
>>> The only other major change I would make is to stop masking all the
>>> exceptions during the task processing.  Or at least, if you're going
>>> to log the exception and stop propagation, log the whole thing so we
>>> get a stacktrace in the logs.
>>>
>>> Jeff
>>>
>>>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: any plans for deferred.defer in Java?

[David Chandler]

I've solved the serialization problems by Base64 encoding the
serialized task before calling payload() and decoding it in the
deserialize(HttpServletRequest) method. I'm guessing something in the
task queue chain (either task queue payload storage or the servlet
call when task is run) has problems transmitting the binary data, as
the exception I was getting at one point was a
java.io.StreamCorruptedException.

I'll send another patch to Vince. My solution works, but I'm not
completely satisfied that I've diagnosed the problem correctly. I used
Apache commons-codec for Base64. Is it safe to use com.sun.misc
instead? I see it in appengine.repackaged chain, as well, but don't
see it as part of an official Google API.

Somewhat related, I've wired up the Deferred servlet through the
GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
the task objects are not created through Guice, but rather by
deserializing, I modified doTask() to accept an additional Injector
argument. We probably don't want that in the official version of
Deferrable, though.

Is there a better way for my Deferrable classes to get access to a
Guice "context"? I'm trying to avoid Guice.createInjector(), as it
results in a duplicate PersistenceManagerFactory being created since
other servlets are also being served through Guice.

Thank you,
/dmc

On Nov 27, 5:42 pm, David Chandler  wrote:
> Jeff,
>
> I'm seeing problems with deserialization, too, when deployed on
> AppEngine. In dev, I can deserialize(serialize(task)) and it works
> just fine, but not so in AppEngine. I get the same error whether the
> task payload is the serialized Deferrable task itself or just the Key
> with the task in the db, but I haven't yet figured out which is the
> cause of the problem.
>
> The error I'm getting on AppEngine is invalid type code: 00 in
> Deferred.deserialize method. I've tried Base64 encoding after
> serialization in case it's related to special binary escape sequences
> that happen to be in the serialized object stream but still no
> success.
>
> I'm pretty sure AppEngine has written some custom code to do
> serialization because I previously got an AppEngine acess control
> exception when calling enableResolveObject(true) in a subclass of
> ObjectInputStream, and I'm wondering if that's in play somehow...
>
> /dmc
>
> On Nov 26, 5:31 am, Jeff Schnitzer  wrote:
>
> > I should have said this before, but thanks for providing this code!
>
> > For reasons that are still not clear to me, my task was not
> > deserializing properly (very odd complaints about HashMap).  Not
> > having a stack trace made it even more confusing than usual.
> > Ultimately I fixed the problem by making my class Externalizable and
> > serializing my object graph as JSON.
>
> > I would definitely want to see deserialization exceptions in the logs.
> >  Even if the container is behaving perfectly rationally, developers
> > can easily hork up the process with Externalizable.  What is to be
> > gained by hiding the stack trace?
>
> > Thanks,
> > JeffOn Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti  
> > wrote:
> > > Hi Jeff,
> > > Thanks for the suggestions and the code. David Chandler sent me a patch to
> > > support user-specified queue names (almost exactly the same as your
> > > changes), and I've committed that patch to SVN. Regarding your other
> > > changes:
> > >   - I've probably make the  init parameter optional and have 
> > > it
> > > default to "/_ah/" rather than throw an exception from the 
> > > init
> > > method if it's missing.
> > >   - While we're at it, the default queue name could be optionally 
> > > specified
> > > by an init parameter also.
> > >   - I'm not sure a stacktrace from the doPost method is really going to 
> > > give
> > > you much more information, but would be open to that  change.
> > > It looks like the Deferred class is going to be added to the official 
> > > GAE/J
> > > SDK, though I'm not sure when. Rather than make any more changes to the 
> > > code
> > > right now, I'd rather wait to see what shows up the SDK and then work
> > > through normal Google channels to get any further modifications made. In 
> > > the
> > > mean time, I'm glad it's working for you.
> > > Vince
> > > On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer  
> > > wrote:
>
> > >> Attached is a modified version of that class that lets you define any
> > >> path you want for the servlet and lets you specify which queue to use
> > >> like this: Deferred.defer(task, "queueName");
>
> > >> (I needed this for my own purposes)
>
> > >> Do with it as you wish :-)
>
> > >> The only other major change I would make is to stop masking all the
> > >> exceptions during the task processing.  Or at least, if you're going
> > >> to log the exception and stop propagation, log the whole thing so we
> > >> get a stacktrace in the logs.
>
> > >> Jeff
>
> > >> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
> > >> wrote:
> > >> > Vince, this is great! I hadn't been watchi

[appengine-java] Re: any plans for deferred.defer in Java?

[David Chandler]

Jeff,

I'm seeing problems with deserialization, too, when deployed on
AppEngine. In dev, I can deserialize(serialize(task)) and it works
just fine, but not so in AppEngine. I get the same error whether the
task payload is the serialized Deferrable task itself or just the Key
with the task in the db, but I haven't yet figured out which is the
cause of the problem.

The error I'm getting on AppEngine is invalid type code: 00 in
Deferred.deserialize method. I've tried Base64 encoding after
serialization in case it's related to special binary escape sequences
that happen to be in the serialized object stream but still no
success.

I'm pretty sure AppEngine has written some custom code to do
serialization because I previously got an AppEngine acess control
exception when calling enableResolveObject(true) in a subclass of
ObjectInputStream, and I'm wondering if that's in play somehow...

/dmc

On Nov 26, 5:31 am, Jeff Schnitzer  wrote:
> I should have said this before, but thanks for providing this code!
>
> For reasons that are still not clear to me, my task was not
> deserializing properly (very odd complaints about HashMap).  Not
> having a stack trace made it even more confusing than usual.
> Ultimately I fixed the problem by making my class Externalizable and
> serializing my object graph as JSON.
>
> I would definitely want to see deserialization exceptions in the logs.
>  Even if the container is behaving perfectly rationally, developers
> can easily hork up the process with Externalizable.  What is to be
> gained by hiding the stack trace?
>
> Thanks,
> JeffOn Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti  
> wrote:
> > Hi Jeff,
> > Thanks for the suggestions and the code. David Chandler sent me a patch to
> > support user-specified queue names (almost exactly the same as your
> > changes), and I've committed that patch to SVN. Regarding your other
> > changes:
> >   - I've probably make the  init parameter optional and have it
> > default to "/_ah/" rather than throw an exception from the init
> > method if it's missing.
> >   - While we're at it, the default queue name could be optionally specified
> > by an init parameter also.
> >   - I'm not sure a stacktrace from the doPost method is really going to give
> > you much more information, but would be open to that  change.
> > It looks like the Deferred class is going to be added to the official GAE/J
> > SDK, though I'm not sure when. Rather than make any more changes to the code
> > right now, I'd rather wait to see what shows up the SDK and then work
> > through normal Google channels to get any further modifications made. In the
> > mean time, I'm glad it's working for you.
> > Vince
> > On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer  wrote:
>
> >> Attached is a modified version of that class that lets you define any
> >> path you want for the servlet and lets you specify which queue to use
> >> like this: Deferred.defer(task, "queueName");
>
> >> (I needed this for my own purposes)
>
> >> Do with it as you wish :-)
>
> >> The only other major change I would make is to stop masking all the
> >> exceptions during the task processing.  Or at least, if you're going
> >> to log the exception and stop propagation, log the whole thing so we
> >> get a stacktrace in the logs.
>
> >> Jeff
>
> >> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
> >> wrote:
> >> > Vince, this is great! I hadn't been watching my own issue, so didn't
> >> > see this until now. Thanks so much!
>
> >> > Only enhancement I would suggest is to enable multiple deferred
> >> > queues. For example. I would like to defer tasks in an email throttle
> >> > queue separately from a general background queue. Perhaps the
> >> > Deferrable interface could have a getQueueName() method, or
> >> > Deferred.defer could have an additional signature defer(Deferrable
> >> > task, String queueName).
>
> >> > Thanks again,
> >> > /dmc
>
> >> --
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Nick Johnson (Google)]

Hi Vince,

I haven't had a chance to integrate your code yet, so feel free to continue
making modifications. In particular, I think it would be useful to provide
the same set of options the Python deferred API accepts, which include 'eta'
and 'countdown' parameters, as well as specifying the queue and URL.

-Nick

On Wed, Nov 25, 2009 at 7:03 PM, Vince Bonfanti  wrote:

> Hi Jeff,
>
> Thanks for the suggestions and the code. David Chandler sent me a patch to
> support user-specified queue names (almost exactly the same as your
> changes), and I've committed that patch to SVN. Regarding your other
> changes:
>
>   - I've probably make the  init parameter optional and have
> it default to "/_ah/" rather than throw an exception from the
> init method if it's missing.
>
>   - While we're at it, the default queue name could be optionally specified
> by an init parameter also.
>
>   - I'm not sure a stacktrace from the doPost method is really going to
> give you much more information, but would be open to that  change.
>
> It looks like the Deferred class is going to be added to the official GAE/J
> SDK, though I'm not sure when. Rather than make any more changes to the code
> right now, I'd rather wait to see what shows up the SDK and then work
> through normal Google channels to get any further modifications made. In the
> mean time, I'm glad it's working for you.
>
> Vince
>
> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer wrote:
>
>> Attached is a modified version of that class that lets you define any
>> path you want for the servlet and lets you specify which queue to use
>> like this: Deferred.defer(task, "queueName");
>>
>> (I needed this for my own purposes)
>>
>> Do with it as you wish :-)
>>
>> The only other major change I would make is to stop masking all the
>> exceptions during the task processing.  Or at least, if you're going
>> to log the exception and stop propagation, log the whole thing so we
>> get a stacktrace in the logs.
>>
>> Jeff
>>
>> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
>> wrote:
>> > Vince, this is great! I hadn't been watching my own issue, so didn't
>> > see this until now. Thanks so much!
>> >
>> > Only enhancement I would suggest is to enable multiple deferred
>> > queues. For example. I would like to defer tasks in an email throttle
>> > queue separately from a general background queue. Perhaps the
>> > Deferrable interface could have a getQueueName() method, or
>> > Deferred.defer could have an additional signature defer(Deferrable
>> > task, String queueName).
>> >
>> > Thanks again,
>> > /dmc
>>
>> --
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>



-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Jeff Schnitzer]

I should have said this before, but thanks for providing this code!

For reasons that are still not clear to me, my task was not
deserializing properly (very odd complaints about HashMap).  Not
having a stack trace made it even more confusing than usual.
Ultimately I fixed the problem by making my class Externalizable and
serializing my object graph as JSON.

I would definitely want to see deserialization exceptions in the logs.
 Even if the container is behaving perfectly rationally, developers
can easily hork up the process with Externalizable.  What is to be
gained by hiding the stack trace?

Thanks,
Jeff

On Wed, Nov 25, 2009 at 11:03 AM, Vince Bonfanti  wrote:
> Hi Jeff,
> Thanks for the suggestions and the code. David Chandler sent me a patch to
> support user-specified queue names (almost exactly the same as your
> changes), and I've committed that patch to SVN. Regarding your other
> changes:
>   - I've probably make the  init parameter optional and have it
> default to "/_ah/" rather than throw an exception from the init
> method if it's missing.
>   - While we're at it, the default queue name could be optionally specified
> by an init parameter also.
>   - I'm not sure a stacktrace from the doPost method is really going to give
> you much more information, but would be open to that  change.
> It looks like the Deferred class is going to be added to the official GAE/J
> SDK, though I'm not sure when. Rather than make any more changes to the code
> right now, I'd rather wait to see what shows up the SDK and then work
> through normal Google channels to get any further modifications made. In the
> mean time, I'm glad it's working for you.
> Vince
> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer  wrote:
>>
>> Attached is a modified version of that class that lets you define any
>> path you want for the servlet and lets you specify which queue to use
>> like this: Deferred.defer(task, "queueName");
>>
>> (I needed this for my own purposes)
>>
>> Do with it as you wish :-)
>>
>> The only other major change I would make is to stop masking all the
>> exceptions during the task processing.  Or at least, if you're going
>> to log the exception and stop propagation, log the whole thing so we
>> get a stacktrace in the logs.
>>
>> Jeff
>>
>> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
>> wrote:
>> > Vince, this is great! I hadn't been watching my own issue, so didn't
>> > see this until now. Thanks so much!
>> >
>> > Only enhancement I would suggest is to enable multiple deferred
>> > queues. For example. I would like to defer tasks in an email throttle
>> > queue separately from a general background queue. Perhaps the
>> > Deferrable interface could have a getQueueName() method, or
>> > Deferred.defer could have an additional signature defer(Deferrable
>> > task, String queueName).
>> >
>> > Thanks again,
>> > /dmc
>>
>> --
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Jeff,

Thanks for the suggestions and the code. David Chandler sent me a patch to
support user-specified queue names (almost exactly the same as your
changes), and I've committed that patch to SVN. Regarding your other
changes:

  - I've probably make the  init parameter optional and have it
default to "/_ah/" rather than throw an exception from the init
method if it's missing.

  - While we're at it, the default queue name could be optionally specified
by an init parameter also.

  - I'm not sure a stacktrace from the doPost method is really going to give
you much more information, but would be open to that  change.

It looks like the Deferred class is going to be added to the official GAE/J
SDK, though I'm not sure when. Rather than make any more changes to the code
right now, I'd rather wait to see what shows up the SDK and then work
through normal Google channels to get any further modifications made. In the
mean time, I'm glad it's working for you.

Vince

On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer  wrote:

> Attached is a modified version of that class that lets you define any
> path you want for the servlet and lets you specify which queue to use
> like this: Deferred.defer(task, "queueName");
>
> (I needed this for my own purposes)
>
> Do with it as you wish :-)
>
> The only other major change I would make is to stop masking all the
> exceptions during the task processing.  Or at least, if you're going
> to log the exception and stop propagation, log the whole thing so we
> get a stacktrace in the logs.
>
> Jeff
>
> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
> wrote:
> > Vince, this is great! I hadn't been watching my own issue, so didn't
> > see this until now. Thanks so much!
> >
> > Only enhancement I would suggest is to enable multiple deferred
> > queues. For example. I would like to defer tasks in an email throttle
> > queue separately from a general background queue. Perhaps the
> > Deferrable interface could have a getQueueName() method, or
> > Deferred.defer could have an additional signature defer(Deferrable
> > task, String queueName).
> >
> > Thanks again,
> > /dmc
>
> --
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Jeff Schnitzer]

Attached is a modified version of that class that lets you define any
path you want for the servlet and lets you specify which queue to use
like this: Deferred.defer(task, "queueName");

(I needed this for my own purposes)

Do with it as you wish :-)

The only other major change I would make is to stop masking all the
exceptions during the task processing.  Or at least, if you're going
to log the exception and stop propagation, log the whole thing so we
get a stacktrace in the logs.

Jeff

On Fri, Nov 20, 2009 at 1:03 PM, David Chandler  wrote:
> Vince, this is great! I hadn't been watching my own issue, so didn't
> see this until now. Thanks so much!
>
> Only enhancement I would suggest is to enable multiple deferred
> queues. For example. I would like to defer tasks in an email throttle
> queue separately from a general background queue. Perhaps the
> Deferrable interface could have a getQueueName() method, or
> Deferred.defer could have an additional signature defer(Deferrable
> task, String queueName).
>
> Thanks again,
> /dmc

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.




Deferred.java
Description: Binary data

[appengine-java] Re: any plans for deferred.defer in Java?

[David Chandler]

Vince, this is great! I hadn't been watching my own issue, so didn't
see this until now. Thanks so much!

Only enhancement I would suggest is to enable multiple deferred
queues. For example. I would like to defer tasks in an email throttle
queue separately from a general background queue. Perhaps the
Deferrable interface could have a getQueueName() method, or
Deferred.defer could have an additional signature defer(Deferrable
task, String queueName).

Thanks again,
/dmc

On Nov 10, 4:37 pm, Vince Bonfanti  wrote:
> I've faxed over the CLA and created issue #2381 for this:
>
>    http://code.google.com/p/googleappengine/issues/detail?id=2381
>
> Here are the changes I've made since the original implementation:
>
>  - The Deferrable.doTask method is now declared to throw Exception
> (instead of PermanentTaskFailure). This allows implementations to
> throw any subclass of Exception in order to have the task retried.
> (PermanentTaskFailure remains a subclass of Exception).
>
>  - The Deferrable.doTask method no longer takes any arguments.
>
>  - Removed the TaskHolder nested class since it's no longer needed.
>
>  - Allow an extra 240 bytes for task overhead; in testing the actual
> overhead was 101 bytes, so this will leave some head-room. In
> addition, the IllegalArgumentException that's thrown if the task size
> is exceeded is caught (and logged), and the task it then written to
> the datastore.
>
>  - An exception is logged if payload is not an instance of Deferrable.
>
>  - If the task is written to the datastore and there's a subsequent
> failure to queue the task, the datastore entity is deleted.
>
>  - Try twice for datatstore puts and deletes in case of
> DatastoreTimeoutException.
>
>  - Try twice when queuing a task in case of TransientFailureException.
>
>  - Added comments.
>
> Let me know if you have additional feedback.
>
> Vince
>
> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>
>  wrote:
> > Hi Vince,
>
> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti  wrote:
>
> >> Hi Nick,
>
> >> Thanks for your feedback. Yes, I'm very interested in seeing this
> >> included in the official SDK. What are the steps from here?
>
> > See this doc:
> >http://groups.google.com/group/google-appengine/web/how-to-submit-a-p...
>
> >> See responses to specific questions inserted below.
>
> >> Vince
>
> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> >> > eliminate the need to declare it on methods that throw it.
>
> >> That's fine. I generally prefer checked exceptions, but if the
> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> change.
>
> > I wouldn't say we prefer them in general - but personally, I think common
> > uses of PermanentTaskFailure fit into the same sort of category as out of
> > memory errors, divisions by zero, and so forth - they're not predictable at
> > design-time.
>
> >> > - If you do the above, you could make the Deferrable interface extend
> >> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> >> > since the implications of Runnable are different.
>
> >> I think Runnable is not appropriate. I also looked at Callable, but in
> >> the end thought a new interface--Deferrable--was the best option.
>
> > Fair enough.
>
> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> > Deferrable, since users will already have to declare a deferrable class
> >> > and
> >> > instantiate it. I'm open to convincing, though.
>
> >> I put the arguments there to mimic the Python API and then realized
> >> later they're not really needed. Removing the arguments also
> >> simplifies the code by eliminating the need for the TaskHolder
> >> class--the Deferrable class could be serialized directly. I'm happy to
> >> remove the arguments if that's what the AppEngine team decides is
> >> best.
>
> > Since the Java version is inevitably different due to the need for a custom
> > class instead of a function object, I'd vote to drop the arguments.
>
> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> >> > Python, the actual limit seems to include some overhead that varies from
> >> > task to task.
>
> >> No, I haven't tested this yet.
>
> >> > - If payload isn't an instance of TaskHolder, the task will silently
> >> > return
> >> > - it should log an error.
>
> >> I think the only way payload won't be an instance of TaskHolder is if
> >> the deserialize() method returns null, in which case an error has
> >> already been logged. But I'll double-check this.
>
> > The other possibility is that something else serialized an object and sent
> > it to your task handler - or possibly, that a future version of your own
> > code did so. I think logging an error (but returning a 200 status code)
> > would be the best policy, so these sorts of situations are debuggable.
>
> > -Nick Johnson
>
> >> > Are you interested in having this included in the official SDK?
> >> > -Nick
> >> > On Sat, Oc

[appengine-java] Re: any plans for deferred.defer in Java?

[Nick Johnson (Google)]

Hi Vince,

On Wed, Nov 11, 2009 at 2:07 PM, Vince Bonfanti  wrote:

>
> Hi Nick,
>
> I'm taking this off-list for now.
>
> I've visited codereview.appspot.com, but am confused. I tried creating
> a new issue, but:
>
>  - what do I specify for "SVN base"?
>

Leave it blank if it'll let you - otherwise, use *http*://
googleappengine.googlecode.com/svn/trunk/java/


>  - it will only let me upload a patch file (not a java source file),
> but I'm not sure how to create a patch for a brand-new file?
>

Try "diff /dev/null /path/to/your/file". :)


>
> Can you offer some help? thanks.
>
> Vince
>
> On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google)
>  wrote:
> > Hi Vince,
> > Thanks for doing this! Could you upload your code to
> codereview.appspot.com
> > and link to it from the issue? We can continue the discussion there.
> > -Nick
> >
> > On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti 
> wrote:
> >>
> >> I've faxed over the CLA and created issue #2381 for this:
> >>
> >>   http://code.google.com/p/googleappengine/issues/detail?id=2381
> >>
> >> Here are the changes I've made since the original implementation:
> >>
> >>  - The Deferrable.doTask method is now declared to throw Exception
> >> (instead of PermanentTaskFailure). This allows implementations to
> >> throw any subclass of Exception in order to have the task retried.
> >> (PermanentTaskFailure remains a subclass of Exception).
> >>
> >>  - The Deferrable.doTask method no longer takes any arguments.
> >>
> >>  - Removed the TaskHolder nested class since it's no longer needed.
> >>
> >>  - Allow an extra 240 bytes for task overhead; in testing the actual
> >> overhead was 101 bytes, so this will leave some head-room. In
> >> addition, the IllegalArgumentException that's thrown if the task size
> >> is exceeded is caught (and logged), and the task it then written to
> >> the datastore.
> >>
> >>  - An exception is logged if payload is not an instance of Deferrable.
> >>
> >>  - If the task is written to the datastore and there's a subsequent
> >> failure to queue the task, the datastore entity is deleted.
> >>
> >>  - Try twice for datatstore puts and deletes in case of
> >> DatastoreTimeoutException.
> >>
> >>  - Try twice when queuing a task in case of TransientFailureException.
> >>
> >>  - Added comments.
> >>
> >> Let me know if you have additional feedback.
> >>
> >> Vince
> >>
> >> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
> >>  wrote:
> >> > Hi Vince,
> >> >
> >> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti 
> >> > wrote:
> >> >>
> >> >> Hi Nick,
> >> >>
> >> >> Thanks for your feedback. Yes, I'm very interested in seeing this
> >> >> included in the official SDK. What are the steps from here?
> >> >
> >> > See this doc:
> >> >
> >> >
> http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
> >> >
> >> >>
> >> >> See responses to specific questions inserted below.
> >> >>
> >> >> Vince
> >> >>
> >> >> > - Perhaps make PermanentTaskFailure a runtime exception, which
> would
> >> >> > eliminate the need to declare it on methods that throw it.
> >> >>
> >> >> That's fine. I generally prefer checked exceptions, but if the
> >> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> >> change.
> >> >
> >> > I wouldn't say we prefer them in general - but personally, I think
> >> > common
> >> > uses of PermanentTaskFailure fit into the same sort of category as out
> >> > of
> >> > memory errors, divisions by zero, and so forth - they're not
> predictable
> >> > at
> >> > design-time.
> >> >>
> >> >> > - If you do the above, you could make the Deferrable interface
> extend
> >> >> > Runnable, as suggested in the meeting. I'm not sure that's
> >> >> > appropriate,
> >> >> > since the implications of Runnable are different.
> >> >>
> >> >> I think Runnable is not appropriate. I also looked at Callable, but
> in
> >> >> the end thought a new interface--Deferrable--was the best option.
> >> >
> >> > Fair enough.
> >> >
> >> >>
> >> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> >> > Deferrable, since users will already have to declare a deferrable
> >> >> > class
> >> >> > and
> >> >> > instantiate it. I'm open to convincing, though.
> >> >>
> >> >> I put the arguments there to mimic the Python API and then realized
> >> >> later they're not really needed. Removing the arguments also
> >> >> simplifies the code by eliminating the need for the TaskHolder
> >> >> class--the Deferrable class could be serialized directly. I'm happy
> to
> >> >> remove the arguments if that's what the AppEngine team decides is
> >> >> best.
> >> >
> >> > Since the Java version is inevitably different due to the need for a
> >> > custom
> >> > class instead of a function object, I'd vote to drop the arguments.
> >> >
> >> >>
> >> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least
> in
> >> >> > Python, the actual limit seems to include some overhead that vari

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Nick,

I'm taking this off-list for now.

I've visited codereview.appspot.com, but am confused. I tried creating
a new issue, but:

 - what do I specify for "SVN base"?
 - it will only let me upload a patch file (not a java source file),
but I'm not sure how to create a patch for a brand-new file?

Can you offer some help? thanks.

Vince

On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google)
 wrote:
> Hi Vince,
> Thanks for doing this! Could you upload your code to codereview.appspot.com
> and link to it from the issue? We can continue the discussion there.
> -Nick
>
> On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti  wrote:
>>
>> I've faxed over the CLA and created issue #2381 for this:
>>
>>   http://code.google.com/p/googleappengine/issues/detail?id=2381
>>
>> Here are the changes I've made since the original implementation:
>>
>>  - The Deferrable.doTask method is now declared to throw Exception
>> (instead of PermanentTaskFailure). This allows implementations to
>> throw any subclass of Exception in order to have the task retried.
>> (PermanentTaskFailure remains a subclass of Exception).
>>
>>  - The Deferrable.doTask method no longer takes any arguments.
>>
>>  - Removed the TaskHolder nested class since it's no longer needed.
>>
>>  - Allow an extra 240 bytes for task overhead; in testing the actual
>> overhead was 101 bytes, so this will leave some head-room. In
>> addition, the IllegalArgumentException that's thrown if the task size
>> is exceeded is caught (and logged), and the task it then written to
>> the datastore.
>>
>>  - An exception is logged if payload is not an instance of Deferrable.
>>
>>  - If the task is written to the datastore and there's a subsequent
>> failure to queue the task, the datastore entity is deleted.
>>
>>  - Try twice for datatstore puts and deletes in case of
>> DatastoreTimeoutException.
>>
>>  - Try twice when queuing a task in case of TransientFailureException.
>>
>>  - Added comments.
>>
>> Let me know if you have additional feedback.
>>
>> Vince
>>
>> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>>  wrote:
>> > Hi Vince,
>> >
>> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti 
>> > wrote:
>> >>
>> >> Hi Nick,
>> >>
>> >> Thanks for your feedback. Yes, I'm very interested in seeing this
>> >> included in the official SDK. What are the steps from here?
>> >
>> > See this doc:
>> >
>> > http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
>> >
>> >>
>> >> See responses to specific questions inserted below.
>> >>
>> >> Vince
>> >>
>> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> >> > eliminate the need to declare it on methods that throw it.
>> >>
>> >> That's fine. I generally prefer checked exceptions, but if the
>> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> >> change.
>> >
>> > I wouldn't say we prefer them in general - but personally, I think
>> > common
>> > uses of PermanentTaskFailure fit into the same sort of category as out
>> > of
>> > memory errors, divisions by zero, and so forth - they're not predictable
>> > at
>> > design-time.
>> >>
>> >> > - If you do the above, you could make the Deferrable interface extend
>> >> > Runnable, as suggested in the meeting. I'm not sure that's
>> >> > appropriate,
>> >> > since the implications of Runnable are different.
>> >>
>> >> I think Runnable is not appropriate. I also looked at Callable, but in
>> >> the end thought a new interface--Deferrable--was the best option.
>> >
>> > Fair enough.
>> >
>> >>
>> >> > - I'm not sure there's any need to be able to pass arguments to the
>> >> > Deferrable, since users will already have to declare a deferrable
>> >> > class
>> >> > and
>> >> > instantiate it. I'm open to convincing, though.
>> >>
>> >> I put the arguments there to mimic the Python API and then realized
>> >> later they're not really needed. Removing the arguments also
>> >> simplifies the code by eliminating the need for the TaskHolder
>> >> class--the Deferrable class could be serialized directly. I'm happy to
>> >> remove the arguments if that's what the AppEngine team decides is
>> >> best.
>> >
>> > Since the Java version is inevitably different due to the need for a
>> > custom
>> > class instead of a function object, I'd vote to drop the arguments.
>> >
>> >>
>> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
>> >> > Python, the actual limit seems to include some overhead that varies
>> >> > from
>> >> > task to task.
>> >>
>> >> No, I haven't tested this yet.
>> >>
>> >> > - If payload isn't an instance of TaskHolder, the task will silently
>> >> > return
>> >> > - it should log an error.
>> >>
>> >> I think the only way payload won't be an instance of TaskHolder is if
>> >> the deserialize() method returns null, in which case an error has
>> >> already been logged. But I'll double-check this.
>> >
>> > The other possibility is that something else serialized an object

[appengine-java] Re: any plans for deferred.defer in Java?

[Nick Johnson (Google)]

Hi Vince,

Thanks for doing this! Could you upload your code to
codereview.appspot.comand link to it from the issue? We can continue
the discussion there.

-Nick

On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti  wrote:

>
> I've faxed over the CLA and created issue #2381 for this:
>
>   http://code.google.com/p/googleappengine/issues/detail?id=2381
>
> Here are the changes I've made since the original implementation:
>
>  - The Deferrable.doTask method is now declared to throw Exception
> (instead of PermanentTaskFailure). This allows implementations to
> throw any subclass of Exception in order to have the task retried.
> (PermanentTaskFailure remains a subclass of Exception).
>
>  - The Deferrable.doTask method no longer takes any arguments.
>
>  - Removed the TaskHolder nested class since it's no longer needed.
>
>  - Allow an extra 240 bytes for task overhead; in testing the actual
> overhead was 101 bytes, so this will leave some head-room. In
> addition, the IllegalArgumentException that's thrown if the task size
> is exceeded is caught (and logged), and the task it then written to
> the datastore.
>
>  - An exception is logged if payload is not an instance of Deferrable.
>
>  - If the task is written to the datastore and there's a subsequent
> failure to queue the task, the datastore entity is deleted.
>
>  - Try twice for datatstore puts and deletes in case of
> DatastoreTimeoutException.
>
>  - Try twice when queuing a task in case of TransientFailureException.
>
>  - Added comments.
>
> Let me know if you have additional feedback.
>
> Vince
>
> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>  wrote:
> > Hi Vince,
> >
> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti 
> wrote:
> >>
> >> Hi Nick,
> >>
> >> Thanks for your feedback. Yes, I'm very interested in seeing this
> >> included in the official SDK. What are the steps from here?
> >
> > See this doc:
> >
> http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
> >
> >>
> >> See responses to specific questions inserted below.
> >>
> >> Vince
> >>
> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> >> > eliminate the need to declare it on methods that throw it.
> >>
> >> That's fine. I generally prefer checked exceptions, but if the
> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
> >> change.
> >
> > I wouldn't say we prefer them in general - but personally, I think common
> > uses of PermanentTaskFailure fit into the same sort of category as out of
> > memory errors, divisions by zero, and so forth - they're not predictable
> at
> > design-time.
> >>
> >> > - If you do the above, you could make the Deferrable interface extend
> >> > Runnable, as suggested in the meeting. I'm not sure that's
> appropriate,
> >> > since the implications of Runnable are different.
> >>
> >> I think Runnable is not appropriate. I also looked at Callable, but in
> >> the end thought a new interface--Deferrable--was the best option.
> >
> > Fair enough.
> >
> >>
> >> > - I'm not sure there's any need to be able to pass arguments to the
> >> > Deferrable, since users will already have to declare a deferrable
> class
> >> > and
> >> > instantiate it. I'm open to convincing, though.
> >>
> >> I put the arguments there to mimic the Python API and then realized
> >> later they're not really needed. Removing the arguments also
> >> simplifies the code by eliminating the need for the TaskHolder
> >> class--the Deferrable class could be serialized directly. I'm happy to
> >> remove the arguments if that's what the AppEngine team decides is
> >> best.
> >
> > Since the Java version is inevitably different due to the need for a
> custom
> > class instead of a function object, I'd vote to drop the arguments.
> >
> >>
> >> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> >> > Python, the actual limit seems to include some overhead that varies
> from
> >> > task to task.
> >>
> >> No, I haven't tested this yet.
> >>
> >> > - If payload isn't an instance of TaskHolder, the task will silently
> >> > return
> >> > - it should log an error.
> >>
> >> I think the only way payload won't be an instance of TaskHolder is if
> >> the deserialize() method returns null, in which case an error has
> >> already been logged. But I'll double-check this.
> >
> > The other possibility is that something else serialized an object and
> sent
> > it to your task handler - or possibly, that a future version of your own
> > code did so. I think logging an error (but returning a 200 status code)
> > would be the best policy, so these sorts of situations are debuggable.
> >
> > -Nick Johnson
> >>
> >> > Are you interested in having this included in the official SDK?
> >> > -Nick
> >> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti 
> >> > wrote:
> >> >>
> >> >> This looked like an interesting problem, and I already had most of
> the
> >> >> pieces in place, so here's my first attempt, whic

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I've faxed over the CLA and created issue #2381 for this:

   http://code.google.com/p/googleappengine/issues/detail?id=2381

Here are the changes I've made since the original implementation:

 - The Deferrable.doTask method is now declared to throw Exception
(instead of PermanentTaskFailure). This allows implementations to
throw any subclass of Exception in order to have the task retried.
(PermanentTaskFailure remains a subclass of Exception).

 - The Deferrable.doTask method no longer takes any arguments.

 - Removed the TaskHolder nested class since it's no longer needed.

 - Allow an extra 240 bytes for task overhead; in testing the actual
overhead was 101 bytes, so this will leave some head-room. In
addition, the IllegalArgumentException that's thrown if the task size
is exceeded is caught (and logged), and the task it then written to
the datastore.

 - An exception is logged if payload is not an instance of Deferrable.

 - If the task is written to the datastore and there's a subsequent
failure to queue the task, the datastore entity is deleted.

 - Try twice for datatstore puts and deletes in case of
DatastoreTimeoutException.

 - Try twice when queuing a task in case of TransientFailureException.

 - Added comments.

Let me know if you have additional feedback.

Vince

On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
 wrote:
> Hi Vince,
>
> On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti  wrote:
>>
>> Hi Nick,
>>
>> Thanks for your feedback. Yes, I'm very interested in seeing this
>> included in the official SDK. What are the steps from here?
>
> See this doc:
> http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
>
>>
>> See responses to specific questions inserted below.
>>
>> Vince
>>
>> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> > eliminate the need to declare it on methods that throw it.
>>
>> That's fine. I generally prefer checked exceptions, but if the
>> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> change.
>
> I wouldn't say we prefer them in general - but personally, I think common
> uses of PermanentTaskFailure fit into the same sort of category as out of
> memory errors, divisions by zero, and so forth - they're not predictable at
> design-time.
>>
>> > - If you do the above, you could make the Deferrable interface extend
>> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
>> > since the implications of Runnable are different.
>>
>> I think Runnable is not appropriate. I also looked at Callable, but in
>> the end thought a new interface--Deferrable--was the best option.
>
> Fair enough.
>
>>
>> > - I'm not sure there's any need to be able to pass arguments to the
>> > Deferrable, since users will already have to declare a deferrable class
>> > and
>> > instantiate it. I'm open to convincing, though.
>>
>> I put the arguments there to mimic the Python API and then realized
>> later they're not really needed. Removing the arguments also
>> simplifies the code by eliminating the need for the TaskHolder
>> class--the Deferrable class could be serialized directly. I'm happy to
>> remove the arguments if that's what the AppEngine team decides is
>> best.
>
> Since the Java version is inevitably different due to the need for a custom
> class instead of a function object, I'd vote to drop the arguments.
>
>>
>> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
>> > Python, the actual limit seems to include some overhead that varies from
>> > task to task.
>>
>> No, I haven't tested this yet.
>>
>> > - If payload isn't an instance of TaskHolder, the task will silently
>> > return
>> > - it should log an error.
>>
>> I think the only way payload won't be an instance of TaskHolder is if
>> the deserialize() method returns null, in which case an error has
>> already been logged. But I'll double-check this.
>
> The other possibility is that something else serialized an object and sent
> it to your task handler - or possibly, that a future version of your own
> code did so. I think logging an error (but returning a 200 status code)
> would be the best policy, so these sorts of situations are debuggable.
>
> -Nick Johnson
>>
>> > Are you interested in having this included in the official SDK?
>> > -Nick
>> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti 
>> > wrote:
>> >>
>> >> This looked like an interesting problem, and I already had most of the
>> >> pieces in place, so here's my first attempt, which is implemented in a
>> >> single class (also attached, along with some test files):
>> >>
>> >>
>> >>
>> >>  http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
>> >>
>> >> I'm more than happy to contribute this for inclusion in the SDK.
>> >>
>> >> Some caveats:
>> >>
>> >>   1) Because of issue #2097
>> >> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
>> >> doesn't work on the develo

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I'm not sure about adding a TemporaryTaskFailure
(TransientTaskFailure?) exception, but regardless of whether we do or
not, it seems that the Deferrable.doTask method should be declared to
throw Exception. Otherwise, implementations will be restricted to
throwing only RuntimeException (or subclasses thereof). Think of cases
where you're using third-party libraries and you simply want
exceptions to propagate up.

My current implementation is based on the Python implementation, which
says this about task failures:

"'Failure' in the case of a deferred task is defined as your task
throwing any uncaught exception. Normally, automatic retries are what
you want - for example, if the task does datastore operations, and the
datastore threw a Timeout exception. If you want to deliberately cause
your task to fail so it will be retried, you can simply throw an
exception."

That seems reasonable: any uncaught exception causes a task retry;
normal termination or PermanentTaskFailure exception do not retry.

Vince

On Tue, Nov 10, 2009 at 1:35 AM, Roy  wrote:
>
> I've started looking at using Vince's Defer helper class.
>
> As a suggestion, could we add a TemporaryTaskFailure exception so it
> is explicit what behaviour is expected. I realise that a task retry is
> the default behaviour by propagating an exception, I just think it's
> better documentation if it's explicit.
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Roy]

I've started looking at using Vince's Defer helper class.

As a suggestion, could we add a TemporaryTaskFailure exception so it
is explicit what behaviour is expected. I realise that a task retry is
the default behaviour by propagating an exception, I just think it's
better documentation if it's explicit.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Nick Johnson (Google)]

Hi Vince,

On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti  wrote:

>
> Hi Nick,
>
> Thanks for your feedback. Yes, I'm very interested in seeing this
> included in the official SDK. What are the steps from here?
>

See this doc:
http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk


>
> See responses to specific questions inserted below.
>
> Vince
>
> > - Perhaps make PermanentTaskFailure a runtime exception, which would
> > eliminate the need to declare it on methods that throw it.
>
> That's fine. I generally prefer checked exceptions, but if the
> AppEngine team prefers unchecked exceptions, I'm happy to make this
> change.
>

I wouldn't say we prefer them in general - but personally, I think common
uses of PermanentTaskFailure fit into the same sort of category as out of
memory errors, divisions by zero, and so forth - they're not predictable at
design-time.


> > - If you do the above, you could make the Deferrable interface extend
> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> > since the implications of Runnable are different.
>
> I think Runnable is not appropriate. I also looked at Callable, but in
> the end thought a new interface--Deferrable--was the best option.
>

Fair enough.


>
> > - I'm not sure there's any need to be able to pass arguments to the
> > Deferrable, since users will already have to declare a deferrable class
> and
> > instantiate it. I'm open to convincing, though.
>
> I put the arguments there to mimic the Python API and then realized
> later they're not really needed. Removing the arguments also
> simplifies the code by eliminating the need for the TaskHolder
> class--the Deferrable class could be serialized directly. I'm happy to
> remove the arguments if that's what the AppEngine team decides is
> best.
>

Since the Java version is inevitably different due to the need for a custom
class instead of a function object, I'd vote to drop the arguments.


>
> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> > Python, the actual limit seems to include some overhead that varies from
> > task to task.
>
> No, I haven't tested this yet.
>
> > - If payload isn't an instance of TaskHolder, the task will silently
> return
> > - it should log an error.
>
> I think the only way payload won't be an instance of TaskHolder is if
> the deserialize() method returns null, in which case an error has
> already been logged. But I'll double-check this.
>

The other possibility is that something else serialized an object and sent
it to your task handler - or possibly, that a future version of your own
code did so. I think logging an error (but returning a 200 status code)
would be the best policy, so these sorts of situations are debuggable.

-Nick Johnson


> > Are you interested in having this included in the official SDK?
> > -Nick
> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti 
> wrote:
> >>
> >> This looked like an interesting problem, and I already had most of the
> >> pieces in place, so here's my first attempt, which is implemented in a
> >> single class (also attached, along with some test files):
> >>
> >>
> >>
> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
> >>
> >> I'm more than happy to contribute this for inclusion in the SDK.
> >>
> >> Some caveats:
> >>
> >>   1) Because of issue #2097
> >> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
> >> doesn't work on the development server (it does work in production).
> >> So go star that issue!
> >>
> >>   2) I've only done minimal testing on the production server (see the
> >> attached test files).
> >>
> >>   3) This post is the only documentation that's currently available.
> >>
> >> First, add the following your your web.xml:
> >>
> >>
> >>Deferred
> >>
> >>
>  com.newatlanta.appengine.taskqueue.Deferred
> >>
> >>
> >>Deferred
> >>/_ah/queue/deferred
> >>
> >>
> >> Second, define the "deferred" queue within queue.xml (use whatever
> >> rate you want):
> >>
> >>
> >>deferred
> >>10/s
> >>
> >>
> >> Next, create a class that implements the
> >> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> >> "doTask()" method of this class is where you implement your task
> >> logic.
> >>
> >> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> >> method to queue up your task:
> >>
> >>TestDeferred testDeferred = new TestDeferred(); // implements
> >> Deferrable
> >>Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );
> >>
> >> (Note that it would be possible to pass arguments as attributes to
> >> your Deferrable instance, rather than using varargs; I'm not sure it
> >> makes much difference which you choose).
> >>
> >> Just as for the Python implementation, if the arguments size exceeds
> >> 10KB, the arguments are stored in a datastore entity, which i

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Nick,

Thanks for your feedback. Yes, I'm very interested in seeing this
included in the official SDK. What are the steps from here?

See responses to specific questions inserted below.

Vince

> - Perhaps make PermanentTaskFailure a runtime exception, which would
> eliminate the need to declare it on methods that throw it.

That's fine. I generally prefer checked exceptions, but if the
AppEngine team prefers unchecked exceptions, I'm happy to make this
change.

> - If you do the above, you could make the Deferrable interface extend
> Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> since the implications of Runnable are different.

I think Runnable is not appropriate. I also looked at Callable, but in
the end thought a new interface--Deferrable--was the best option.

> - I'm not sure there's any need to be able to pass arguments to the
> Deferrable, since users will already have to declare a deferrable class and
> instantiate it. I'm open to convincing, though.

I put the arguments there to mimic the Python API and then realized
later they're not really needed. Removing the arguments also
simplifies the code by eliminating the need for the TaskHolder
class--the Deferrable class could be serialized directly. I'm happy to
remove the arguments if that's what the AppEngine team decides is
best.

> - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> Python, the actual limit seems to include some overhead that varies from
> task to task.

No, I haven't tested this yet.

> - If payload isn't an instance of TaskHolder, the task will silently return
> - it should log an error.

I think the only way payload won't be an instance of TaskHolder is if
the deserialize() method returns null, in which case an error has
already been logged. But I'll double-check this.

> Are you interested in having this included in the official SDK?
> -Nick
> On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti  wrote:
>>
>> This looked like an interesting problem, and I already had most of the
>> pieces in place, so here's my first attempt, which is implemented in a
>> single class (also attached, along with some test files):
>>
>>
>>  http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
>>
>> I'm more than happy to contribute this for inclusion in the SDK.
>>
>> Some caveats:
>>
>>   1) Because of issue #2097
>> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
>> doesn't work on the development server (it does work in production).
>> So go star that issue!
>>
>>   2) I've only done minimal testing on the production server (see the
>> attached test files).
>>
>>   3) This post is the only documentation that's currently available.
>>
>> First, add the following your your web.xml:
>>
>>    
>>        Deferred
>>
>>  com.newatlanta.appengine.taskqueue.Deferred
>>    
>>    
>>        Deferred
>>        /_ah/queue/deferred
>>    
>>
>> Second, define the "deferred" queue within queue.xml (use whatever
>> rate you want):
>>
>>    
>>        deferred
>>        10/s
>>    
>>
>> Next, create a class that implements the
>> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
>> "doTask()" method of this class is where you implement your task
>> logic.
>>
>> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
>> method to queue up your task:
>>
>>    TestDeferred testDeferred = new TestDeferred(); // implements
>> Deferrable
>>    Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );
>>
>> (Note that it would be possible to pass arguments as attributes to
>> your Deferrable instance, rather than using varargs; I'm not sure it
>> makes much difference which you choose).
>>
>> Just as for the Python implementation, if the arguments size exceeds
>> 10KB, the arguments are stored in a datastore entity, which is then
>> deleted when the task is executed. Also, your doTask() method can
>> throw a PermanentTaskFailure exception to halt retries; any other
>> exceptions cause the task to be retried.
>>
>> Let me know if you find this useful, have any questions or encounter
>> any problems.
>>
>> Vince
>>
>> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) 
>> wrote:
>> > Hi David. This may be coming to Java eventually, but it hasn't been
>> > started
>> > yet. If you or anyone else is interested in contributing, let me know.
>> >
>> > - Jason
>> >
>> > On Wed, Oct 28, 2009 at 7:52 AM, David Chandler 
>> > wrote:
>> >>
>> >> Re: http://code.google.com/appengine/articles/deferred.html
>> >>
>> >> Will this be coming to AppEngine for Java?
>> >>
>> >> David Chandler
>> >> http://turbomanage.wordpress.com
>> >>
>> >>
>> >
>> >
>> > >
>> >
>>
>>
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are sub

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I just committed an update to this to remove the static
DatastoreService instance.

Vince

On Sat, Oct 31, 2009 at 2:08 PM, Vince Bonfanti  wrote:
> This looked like an interesting problem, and I already had most of the
> pieces in place, so here's my first attempt, which is implemented in a
> single class (also attached, along with some test files):
>
>    
> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
>
> I'm more than happy to contribute this for inclusion in the SDK.
>
> Some caveats:
>
>   1) Because of issue #2097
> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
> doesn't work on the development server (it does work in production).
> So go star that issue!
>
>   2) I've only done minimal testing on the production server (see the
> attached test files).
>
>   3) This post is the only documentation that's currently available.
>
> First, add the following your your web.xml:
>
>    
>        Deferred
>        
> com.newatlanta.appengine.taskqueue.Deferred
>    
>    
>        Deferred
>        /_ah/queue/deferred
>    
>
> Second, define the "deferred" queue within queue.xml (use whatever
> rate you want):
>
>    
>        deferred
>        10/s
>    
>
> Next, create a class that implements the
> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> "doTask()" method of this class is where you implement your task
> logic.
>
> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> method to queue up your task:
>
>    TestDeferred testDeferred = new TestDeferred(); // implements Deferrable
>    Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );
>
> (Note that it would be possible to pass arguments as attributes to
> your Deferrable instance, rather than using varargs; I'm not sure it
> makes much difference which you choose).
>
> Just as for the Python implementation, if the arguments size exceeds
> 10KB, the arguments are stored in a datastore entity, which is then
> deleted when the task is executed. Also, your doTask() method can
> throw a PermanentTaskFailure exception to halt retries; any other
> exceptions cause the task to be retried.
>
> Let me know if you find this useful, have any questions or encounter
> any problems.
>
> Vince
>
> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)  wrote:
>> Hi David. This may be coming to Java eventually, but it hasn't been started
>> yet. If you or anyone else is interested in contributing, let me know.
>>
>> - Jason
>>
>> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler 
>> wrote:
>>>
>>> Re: http://code.google.com/appengine/articles/deferred.html
>>>
>>> Will this be coming to AppEngine for Java?
>>>
>>> David Chandler
>>> http://turbomanage.wordpress.com
>>>
>>>
>>
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

This looked like an interesting problem, and I already had most of the
pieces in place, so here's my first attempt, which is implemented in a
single class (also attached, along with some test files):


http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

I'm more than happy to contribute this for inclusion in the SDK.

Some caveats:

   1) Because of issue #2097
(http://code.google.com/p/googleappengine/issues/detail?id=2097), this
doesn't work on the development server (it does work in production).
So go star that issue!

   2) I've only done minimal testing on the production server (see the
attached test files).

   3) This post is the only documentation that's currently available.

First, add the following your your web.xml:


Deferred

com.newatlanta.appengine.taskqueue.Deferred


Deferred
/_ah/queue/deferred


Second, define the "deferred" queue within queue.xml (use whatever
rate you want):


deferred
10/s


Next, create a class that implements the
com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
"doTask()" method of this class is where you implement your task
logic.

Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
method to queue up your task:

TestDeferred testDeferred = new TestDeferred(); // implements Deferrable
Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );

(Note that it would be possible to pass arguments as attributes to
your Deferrable instance, rather than using varargs; I'm not sure it
makes much difference which you choose).

Just as for the Python implementation, if the arguments size exceeds
10KB, the arguments are stored in a datastore entity, which is then
deleted when the task is executed. Also, your doTask() method can
throw a PermanentTaskFailure exception to halt retries; any other
exceptions cause the task to be retried.

Let me know if you find this useful, have any questions or encounter
any problems.

Vince

On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)  wrote:
> Hi David. This may be coming to Java eventually, but it hasn't been started
> yet. If you or anyone else is interested in contributing, let me know.
>
> - Jason
>
> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler 
> wrote:
>>
>> Re: http://code.google.com/appengine/articles/deferred.html
>>
>> Will this be coming to AppEngine for Java?
>>
>> David Chandler
>> http://turbomanage.wordpress.com
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---



TestDeferred.java
Description: Binary data


testDeferred.jsp
Description: Binary data


Deferred.java
Description: Binary data

[appengine-java] Re: any plans for deferred.defer in Java?

[Jason (Google)]

Hi David. This may be coming to Java eventually, but it hasn't been started
yet. If you or anyone else is interested in contributing, let me know.

- Jason

On Wed, Oct 28, 2009 at 7:52 AM, David Chandler wrote:

>
> Re: http://code.google.com/appengine/articles/deferred.html
>
> Will this be coming to AppEngine for Java?
>
> David Chandler
> http://turbomanage.wordpress.com
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---