[appengine-java] java.util.concurrent.locks.Lock and ReadWriteLock implementations for GAE/J

[Vince Bonfanti]

I needed a set of distributed locks for
GaeVFS that
work across multiple JVM instances, and chose to implement them to the
Lockand
ReadWriteLockinterfaces
of the
java.util.concurrent.lockspackage
in the hope that they might be generally useful for other
applications. The source
codefor
the initial implementation of these locks is available from the GaeVFS
project web site in the
com.newatlanta.appengine.lockspackage
(junit tests are in the
com.newatlanta.appengine.junit.lockspackage),
which includes the following classes:

ExclusiveLock
Implements a mutual exclusion lock with the following features:


   - re-entrant locking by the thread that owns the lock (matching unlocks
  are required to fully release the lock)
  - only the thread that owns the lock can unlock it
  - locks automatically expire after 30 seconds (the GAE request timeout
  limit)
  - locks can be acquired with a single call to memcache

ReadWriteLock
Implements a "many readers, one writer" scheme with the following features:


   - the write lock is mutually exclusive and can only be acquired if no
  other threads are holding the read lock
  - the read lock is shared and can only be acquired if no other threads
  are holding the write lock
  - the write lock is re-entrant by the thread that owns the lock
  (matching unlocks are required to fully release the lock)
  - the thread that owns the write lock may also acquire the read lock
  (it's not necessary to unlock the read lock--releasing the write
lock also
  releases the read lock)
  - write locks automatically expire after 30 seconds (the GAE request
  timeout limit)

Using the ReadWriteLock can be fairly expensive; it takes a minimum of two
memcache calls to acquire the write lock, and a minimum of three memcache
calls to acquire a read lock. Therefore, it might be better to use an
ExclusiveLock--which can be acquired with a minimum of one memcache
call--depending on our application. For example, if your application's read
operations are short and fast, then an ExclusiveLock might perform better
than a ReadWriteLock.

SharedLock
Implements a counter-based shared lock that isn't particularly useful by
itself, but is used as a base class for the ReadWriteLock implementation.

AbstractLock
The base class for ExclusiveLock and SharedLock.


The implementations of these locks are based on the atomic features of the
MemcacheService,
and are therefore unreliable because memcache can evict them at any time.
This shouldn't be too much of a concern for the ExclusiveLock or the "write
lock" of the ReadWriteLock because the maximum lifetime of these locks is
only 30 seconds (the maximum lifetime of a request). However, this is a
concern for the counter-based SharedLock and the "read lock" of the
ReadWriteLock. I plan to address this by persisting the counters to the GAE
datastore via a write-behind mechanism as soon as Task Queues are available
for GAE/J.

I haven't done any performance testing of these locks (maybe when profiling
is available on GAE/J?). As with any synchronization mechanism, use these
sparingly, since misuse can badly harm the performance of your application.

Any comments or feedback is welcomed.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Profiling in java

[Vince Bonfanti]

Beautiful! I really needed this--thanks.

But, "Delegate" is an interface, so I think the ProfilingDelegate
needs to look something like this:

import com.google.apphosting.api.ApiProxy.Delegate;
import com.google.apphosting.api.ApiProxy.Environment;
import com.google.apphosting.api.ApiProxy.LogRecord;

class ProfilingDelegate implements Delegate {
Delegate parent;
public ProfilingDelegate(Delegate parent) {
  this.parent = parent;
}
public byte[] makeSyncCall(Environment env, String pkg, String
method, byte[] request) {
  long start = System.nanoTime();
  byte[] result = parent.makeSyncCall(env, pkg, method, request);
  log.info(pkg + "." + method + ": " + System.nanoTime() - start);
  return result;
}
public void log(Environment env, LogRecord logRec) {
parent.log(env, logRec);
}
}

On Thu, Aug 20, 2009 at 12:02 PM, Toby Reyelts wrote:
> You'll need to profile two separate things:
> 1) Compute-bound execution
> 2) Time spent in rpcs to our services
> 1) Compute-bound performance should be in the same rough ballpark on the
> dev_appserver as our production servers. You can use any standard Java
> profiler on your dev_appserer to measure this.
> 2) Time spent in services is going to vary wildly between production and the
> dev_appserver. For example, the datastore for the dev_appserver is just an
> in-memory store with periodic writebacks. The datastore for production is a
> distributed database. You can measure these times in production using a
> profiling ApiProxy.Delegate:
> class ProfilingDelegate extends Delegate {
> Delegate parent; public ProfilingDelegate(Delegate parent) { this.parent =
> parent; } public byte[] makeSyncCall(Environment env, String pkg, String
> method, byte[] request) { long start = System.nanoTime(); byte[] result =
> parent.makeSyncCall(env, pkg, method, request); log.log(INFO, pkg + “.” +
> method + “: “ + System.nanoTime() - start); return result; } }
> ApiProxy.setDelegate(new ProfilingDelegate(ApiProxy.getDelegate()));
> If you're feeling particularly aggressive, you can use the App Engine status
> site in conjunction with an ApiProxy.Delegate to come up with an estimation
> of how much time you'll spend in RPCs on the production and even insert
> corresponding sleeps into your dev_appserver.
> On Thu, Aug 20, 2009 at 5:17 AM, mar_novice  wrote:
>>
>> is there a way to do profiling with google app engine java?
>>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] DatastoreService.put(Iterable) success/failure indicators

[Vince Bonfanti]

I think I've seen this question somewhere else, but I've searched and
can't seem to find it, so I apologize in advance is this is a
duplicate.

I'm using the datastore low-level API to do a batch put via
DatastoreService.put(Iterable). Some questions related to this
method:

  1. If this method succeeds (does not throw an exception) does that
guarantee that all entities were written to the datastore
successfully?

  2. If this method fails (throws a DatastoreFailureException) does
that mean that none of the entities were written, or that some were
and some weren't? If the latter, is there any way to know which
entities were successfully written and which weren't?

  3. If I invoke DatastoreService.put(Transaction, Iterable)
and it throws a DatastoreFailureException, does invoking
Transaction.rollback() guarantee that none of the entities are written
to the datastore? (I'm pretty sure the answer to this one must be
"yes").

When doing a batch put, I need to be able to guarantee that all
entities are either written, or all not written. I think the best way
is to use a transaction, but thought I'd ask to make sure my
understanding of how this works is correct. Thanks.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Memcache Service Enhancement

[Vince Bonfanti]

I'd be very interested in a facility that would allow you to batch
arbitrary memcache operations...if it meant performance savings. If
the performance was the same as doing the operations individually then
it wouldn't be as interesting.

Vince

On Thu, Aug 13, 2009 at 10:02 AM, Martyn wrote:
>
> I have developed a simple wrapper to aid batch processing.  This is
> just an interface to a managed Map that is later used for a putAll.
>
> However, one of the operations I would like to batch is an increment,
> not a put. The current putAll does not support this.
>
> It seems pretty clear that a generic putAll could take a set of
> Operation objects:
>
> GetOperation(Key);
> PutOperation(Key, Value);
> IncrementOperation(Key, Value);
>
> This would enable far more flexibility, with each Operation providing
> an isComplete and value method, for example.
>
> The only guarantee need be the operation sequence... at least that is
> all I need to support my use of increment as a kind of shared memory
> semaphore ;-)
>
> Is anyone else interested in something similar?
>
> - Martyn
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Index control in the low-level datastore API

[Vince Bonfanti]

Look at the Entity.setUnindexedProperty() method.

On Fri, Aug 21, 2009 at 7:51 PM, David Given wrote:
>
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> I don't seem to see any APIs in DatastoreService or Entity that allow me
> to do anything with indices. In particular, I want specify that certain
> fields in my Entity shouldn't be indexed to prevent ballooning database
> size.
>
> This must be possible, because otherwise the JPA and JDO APIs wouldn't
> be able to do it --- where should I be looking?
>
> - --
> ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─
> │
> │ "People who think they know everything really annoy those of us who
> │ know we don't." --- Bjarne Stroustrup
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.9 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iD8DBQFKjzLkf9E0noFvlzgRAufQAKCOGknj3KKYCAT+pHbzr56gRYU0VACfQRmy
> BYZOg4Mu1C+nAFt+Lj4Swu4=
> =Is9m
> -END PGP SIGNATURE-
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: How to query multivalued properties in low-level Datastore API?

[Vince Bonfanti]

Yes, adding multiple filters (with any FilterOperator) does an AND operation.

Vince

On Tue, Aug 25, 2009 at 10:51 AM, ted stockwell wrote:
>
>
> Thanks much.
> One more question...
> If I want to select recipes that have both hamburger and olives should
> I just add two filters???
>   Query query = new Query("recipe");
>   query.addFilter("ingredients", Query.FilterOperator.EQUAL,
> "hamburger");
>   query.addFilter("ingredients", Query.FilterOperator.EQUAL,
> "olives");
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Service logic conflicts with transaction policy for entity groups

[Vince Bonfanti]

I thought it did, but I may not have fully understood the question.
Using distributed locks would allow him to implement atomicity
(synchronization) of transactions in his application code without
modifying his data model, something like this:

exclusiveLock.lock();
try {
// modify entity A
// modify entity B
} finally {
exclusiveLock.unlock();
}

Assuming that entities "A" and "B" are in different entity groups, and
therefore he can't use the built-in transaction support provided by
the datastore.

Vince

On Wed, Aug 26, 2009 at 12:44 PM, objectuser wrote:
>
> Hey, Vince,
>
> That's really cool.  Does that address the need of the OP though?  I
> may just not be understanding it fully ...
>
> Thanks!
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Service logic conflicts with transaction policy for entity groups

[Vince Bonfanti]

I've implemented a set of distributed locks for GAE that I posted a
message about previously (especially note the current limitations of
reliability of shared/read locks):

  
http://groups.google.com/group/google-appengine-java/browse_thread/thread/6abc2092fc3870be/d495d9d82abbf912?lnk=gst&q=lock#d495d9d82abbf912

Here's a direct link to the source code:

  
http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/locks/

Since posting that message, I've done some timings on memcache calls
and found that they usually take about 15ms; I've seen low times
around 6-7ms and high times as much as 60ms.

The ExclusiveLock is the most efficient, since it takes only one
memcache call to acquire and one to release. If you use the
ReadWriteLock class, it takes two memcache calls to acquire a write
lock, and three memcache calls to acquire a read lock (it only takes
one memcache call to release either a write or read lock).

So, while using ReadWriteLock allows the greatest concurrency in your
application, unless the read operations take longer than 45ms on
average, you might be better off using ExclusiveLocks.

Let me know if you find this useful.

Vince

On Wed, Aug 26, 2009 at 10:31 AM, randal wrote:
>
> Hello.
>
> I'm trying to create a service method that encapsulates a particular
> business logic. I want to make this feature transactional such that
> its job is accomplished atomically(?). The problem is the service
> logic involves accessing different entities that do not belong to the
> same entity group which is not allowed in GAE.
>
> At the moment, I've temporarily disabled transaction management to the
> service method. However, I feel I'd need to manage transactions
> eventually. I'm thinking of revising the model design but from how I
> see it, the model classes are good as they are--unrelated by
> ownership.
>
> Btw, within the service logic is some methods that are transactional.
> I'm using Spring framework to annotate transaction management.
>
> Can anyone help me on how I can go about this?
>
> Thanks.
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Service logic conflicts with transaction policy for entity groups

[Vince Bonfanti]

Yes, that's what I meant (I should have given a fuller answer the
first time). Any error checking and "rollback" to undo "modify entity
A" in case of failure of "modify Entity B" would have to be coded
manually within the lock/unlock block.

Vince

On Wed, Aug 26, 2009 at 1:09 PM, objectuser wrote:
>
> I see what you're saying ... you're addressing the integrity issues
> around concurrent modifications.  As is, "what if someone makes an
> inconsistent modification to A while I'm trying to update B" ...
>
> I was thinking of it in terms of, what if "modify entity A" works and
> "modify entity B" fails.
>
> Good stuff!
>
> On Aug 26, 11:51 am, Vince Bonfanti  wrote:
>> I thought it did, but I may not have fully understood the question.
>> Using distributed locks would allow him to implement atomicity
>> (synchronization) of transactions in his application code without
>> modifying his data model, something like this:
>>
>>     exclusiveLock.lock();
>>     try {
>>         // modify entity A
>>         // modify entity B
>>     } finally {
>>         exclusiveLock.unlock();
>>     }
>>
>> Assuming that entities "A" and "B" are in different entity groups, and
>> therefore he can't use the built-in transaction support provided by
>> the datastore.
>>
>> Vince
>>
>> On Wed, Aug 26, 2009 at 12:44 PM, objectuser wrote:
>>
>> > Hey, Vince,
>>
>> > That's really cool.  Does that address the need of the OP though?  I
>> > may just not be understanding it fully ...
>>
>> > Thanks!
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Java memcache "increment with initial value"

[Vince Bonfanti]

I notice that the Python memcache API supports an "increment with
initial value" operation:


http://code.google.com/appengine/docs/python/memcache/clientclass.html#Client_incr

Are there any plans to support this on Java? In the Python
implementation, is this simply done as three memcache calls, something
like this:

Long counter = memcache.increment( key, 1 );
if ( counter == null ) {
memcache.put( key, (long)0, null, SetPolicy.ADD_ONLY_IF_NOT_PRESENT );
counter = memcache.increment( key, 1 );
}

If that's all Python is doing, then I don't see any value of adding it
to Java; but, if the Python implementation is somehow more efficient
than the three memcache calls in the Java example above, then I'd like
to see it added to Java.

Thanks.

Vince

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] new stack traces after upgrading to SDK 1.2.5

[Vince Bonfanti]

After upgrading to SDK 1.2.5, I've noticed a set of stack traces that
show up in both the development environment and on the production
server. These only appear if the logging level is set to INFO, and
they show up as INFO logs on the production server. I'm pretty sure
these are new in SDK 1.2.5 since I've been running at the INFO logging
level for quite some time. They only happen for the first request
after the application has been restarted. From my production server:

com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue$SystemLoader
loadFinalizer: Not allowed to access system class loader.

com.google.appengine.repackaged.com.google.common.base.internal.Finalizer
getInheritableThreadLocalsField: Couldn't access
Thread.inheritableThreadLocals. Reference finalizer threads will
inherit thread local values.

com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue
: Failed to start reference finalizer thread. Reference cleanup
will only occur when new references are created.
java.lang.reflect.InvocationTargetException
at 
com.google.appengine.runtime.Request.process-ad303eae7c610e90(Request.java)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Method.java:40)
at 
com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue.(FinalizableReferenceQueue.java:124)
at 
com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools$WeakInterningPool.(InterningPools.java:104)
at 
com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools.newWeakInterningPool(InterningPools.java:48)
at 
com.google.appengine.repackaged.com.google.io.protocol.ProtocolSupport.(ProtocolSupport.java:55)
at 
com.google.apphosting.api.DatastorePb$Query.(DatastorePb.java:1072)
at 
com.google.apphosting.api.DatastorePb$Query$1.(DatastorePb.java:2355)
at 
com.google.apphosting.api.DatastorePb$Query.(DatastorePb.java:2355)
at 
com.google.appengine.api.datastore.QueryTranslator.convertToPb(QueryTranslator.java:27)
at 
com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.convertToPb(DatastoreServiceImpl.java:357)
at 
com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.runQuery(DatastoreServiceImpl.java:339)
at 
com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.access$100(DatastoreServiceImpl.java:269)
at 
com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl$1.iterator(DatastoreServiceImpl.java:303)
at 
org.apache.jsp.test.listEntities_jsp._jspService(listEntities_jsp.java:49)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at 
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
...truncated

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: new stack traces after upgrading to SDK 1.2.5

[Vince Bonfanti]

That's fine. :)

I just wanted to make sure someone was aware of this, since it
appeared to be something new introduced with the latest SDK.

On Fri, Sep 4, 2009 at 4:14 PM, Toby Reyelts wrote:
> It's a message that you can safely ignore. There is a reason it's logged at
> INFO. :)
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] QueueFactory.getQueue( String )

[Vince Bonfanti]

My first question on task queues...

I'd like to design my application to use a named (configured) queue if
it exists, but to drop back to using the default queue if the named
queue isn't configured. I'd like to do something like this:

Queue q = QueueFactory.getQueue( "myQueue" );
if ( q == null ) {
q = QueueFactory.getDefaultQueue();
}

But, the documentation for QueueFactory.getQueue( String ) implies
that this isn't how it works:

"Attempting to use a non-existing queue name may result in errors
at the point of use of the Queue object and not when calling
getQueue(String)."

Is there a way to simply and reliably determine whether a given queue
name has been configured?

Vince

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Bulk writes to datastore

[Vince Bonfanti]

Your two "quick notes" seem to be contradictory. In order to use
transactions, don't all of the entities have to be in the same entity
group?

Vince

On Fri, Sep 4, 2009 at 8:24 PM, Jason (Google) wrote:
> Batch puts are supported, yes, and as of yesterday's release, calling
> makePersistentAll (JDO) and the equivalent JPA call will take advantage of
> this support (previously, you had to use the low-level API).
>
> Two quick notes:
>
> 1) All of the entities that you're persisting should be in separate entity
> groups since two entities in the same entity group can't be written to
> consecutively, and you will see datastore timeout exceptions if many
> simultaneous write requests come in for the same entity or entity group.
> 2) Batch puts do not operate in a transaction. This means that some writes
> may succeed but others may not, so if you need the ability to rollback,
> you'll need transactions.
>
> - Jason
>
> Let me know if you have any more questions on this.
>
> - Jason
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: QueueFactory.getQueue( String )

[Vince Bonfanti]

I found the answer to this: the Queue.add() method throws
IllegalArgumentException if the specified queue isn't configured (BTW,
the message in the IllegalArgumentException is "The specified queue is
unknown :" but doesn't actually include the queue name in the message
string). This isn't as nice as what I'd like, but I can see that the
queue.xml file isn't processed until Queue.add() is invoked.

I guess my workaround will be to try to add a dummy task to the queue
when I create it:

private static Queue queue;

static {
queue = QueueFactory.getQueue( "myQueue" );
try {
queue.add();
} catch ( IllegalArgumentException e ) {
queue = QueueFactory.getDefaultQueue();
}
}

I'm trying to simplify configuration for users of GaeVFS
(http://code.google.com/p/gaevfs/). Rather than requiring them to
configure task queues for GaeVFS, I'd like to have it use the default
queue by default. However, if they want to change the behavior for
specific queues, then they can configure them explicitly.

Should I open an issue for a simpler way to detect if a queue is
configured? or is this something that's unlikely to change?

Vince

On Fri, Sep 4, 2009 at 6:19 PM, Vince Bonfanti wrote:
> My first question on task queues...
>
> I'd like to design my application to use a named (configured) queue if
> it exists, but to drop back to using the default queue if the named
> queue isn't configured. I'd like to do something like this:
>
>    Queue q = QueueFactory.getQueue( "myQueue" );
>    if ( q == null ) {
>        q = QueueFactory.getDefaultQueue();
>    }
>
> But, the documentation for QueueFactory.getQueue( String ) implies
> that this isn't how it works:
>
>    "Attempting to use a non-existing queue name may result in errors
> at the point of use of the Queue object and not when calling
> getQueue(String)."
>
> Is there a way to simply and reliably determine whether a given queue
> name has been configured?
>
> Vince
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] support for task queues in development environment?

[Vince Bonfanti]

Task queues are working properly when I deploy my app to Google's
servers, but never get executed in my development environment (Eclipse
Galileo). There are no errors, no logs, no indication of what's going
wrong. Queue.add() returns successfully, but my tasks never get
executed. I'm running the exact same code in development and in
production. Any clues, or any way for me to figure out what's going
wrong? Are task queues supported in the development environment?
Thanks.

Vince

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: support for task queues in development environment?

[Vince Bonfanti]

Perfect! At least for initial debugging, that's actually better than
having the tasks execute automatically--it makes it easier to see
what's going on with multiple tasks being queued.

Thanks.

On Sun, Sep 6, 2009 at 7:05 PM, Max Ross wrote:
> Hi Vince,
>
> In order to get the task queue into the 1.2.5 release we had to take a
> shortcut with the local development environment.  Tasks execute
> automatically in production but not locally.  However, in the local
> environment you can kick off your tasks manually by visiting the task queue
> admin console page, accessible via a link off of
> http://localhost:8080/_ah/admin
>
> We thought we had this properly documented but it appears we do not.  I've
> filed a bug and we'll get this taken care of quickly.
>
> Sorry for the trouble.
>
> Thanks,
> Max
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: QueueFactory.getQueue( String )

[Vince Bonfanti]

Done (#2089).

BTW, I've opened two issues on the Task Queue API that I consider
fairly serious (at least for what I'm trying to accomplish):

  #2088. A datastore Key that has a parent does not survive a
round-trip through keyToString and stringToKey and still be usable as
a memcache key. The bug report contains more details and a simple
example that doesn't require task queues (so technically, this isn't a
task queue issue).

  #2090. TaskOptions doesn't support multiple parameters with the same
name. Again, the bug report has more details about why this is
important.

I'd be interested in getting some feedback on these.

Vince

On Tue, Sep 8, 2009 at 4:11 PM, Jason (Google) wrote:
> Hi Vince. I think this sounds reasonable. Please open a new issue.
>
> - Jason
>
> On Sat, Sep 5, 2009 at 11:48 AM, Vince Bonfanti  wrote:
>>
>> I found the answer to this: the Queue.add() method throws
>> IllegalArgumentException if the specified queue isn't configured (BTW,
>> the message in the IllegalArgumentException is "The specified queue is
>> unknown :" but doesn't actually include the queue name in the message
>> string). This isn't as nice as what I'd like, but I can see that the
>> queue.xml file isn't processed until Queue.add() is invoked.
>>
>> I guess my workaround will be to try to add a dummy task to the queue
>> when I create it:
>>
>>    private static Queue queue;
>>
>>    static {
>>        queue = QueueFactory.getQueue( "myQueue" );
>>        try {
>>            queue.add();
>>        } catch ( IllegalArgumentException e ) {
>>            queue = QueueFactory.getDefaultQueue();
>>        }
>>    }
>>
>> I'm trying to simplify configuration for users of GaeVFS
>> (http://code.google.com/p/gaevfs/). Rather than requiring them to
>> configure task queues for GaeVFS, I'd like to have it use the default
>> queue by default. However, if they want to change the behavior for
>> specific queues, then they can configure them explicitly.
>>
>> Should I open an issue for a simpler way to detect if a queue is
>> configured? or is this something that's unlikely to change?
>>
>> Vince
>>
>> On Fri, Sep 4, 2009 at 6:19 PM, Vince Bonfanti wrote:
>> > My first question on task queues...
>> >
>> > I'd like to design my application to use a named (configured) queue if
>> > it exists, but to drop back to using the default queue if the named
>> > queue isn't configured. I'd like to do something like this:
>> >
>> >    Queue q = QueueFactory.getQueue( "myQueue" );
>> >    if ( q == null ) {
>> >        q = QueueFactory.getDefaultQueue();
>> >    }
>> >
>> > But, the documentation for QueueFactory.getQueue( String ) implies
>> > that this isn't how it works:
>> >
>> >    "Attempting to use a non-existing queue name may result in errors
>> > at the point of use of the Queue object and not when calling
>> > getQueue(String)."
>> >
>> > Is there a way to simply and reliably determine whether a given queue
>> > name has been configured?
>> >
>> > Vince
>> >
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: support for task queues in development environment?

[Vince Bonfanti]

Being able to examine the task queue, and release tasks manually--and
in different order than they were queued--is proving quite valuable.
It would be nice to keep this as an option when you add automatic
execution.

Vince

On Tue, Sep 8, 2009 at 2:42 PM, Max Ross wrote:
> Interesting.  We were planning on getting rid of manual task execution so
> that dev would completely match prod, but if your experience is shared by
> others it sounds like we should consider leaving the door open for manual
> task execution as well.
>
> Thanks,
> Max
>
> On Mon, Sep 7, 2009 at 8:52 AM, Vince Bonfanti  wrote:
>>
>> Perfect! At least for initial debugging, that's actually better than
>> having the tasks execute automatically--it makes it easier to see
>> what's going on with multiple tasks being queued.
>>
>> Thanks.
>>
>> On Sun, Sep 6, 2009 at 7:05 PM, Max Ross wrote:
>> > Hi Vince,
>> >
>> > In order to get the task queue into the 1.2.5 release we had to take a
>> > shortcut with the local development environment.  Tasks execute
>> > automatically in production but not locally.  However, in the local
>> > environment you can kick off your tasks manually by visiting the task
>> > queue
>> > admin console page, accessible via a link off of
>> > http://localhost:8080/_ah/admin
>> >
>> > We thought we had this properly documented but it appears we do not.
>> > I've
>> > filed a bug and we'll get this taken care of quickly.
>> >
>> > Sorry for the trouble.
>> >
>> > Thanks,
>> > Max
>> >

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] sending binary data (serialized object) via TaskOptions.payload()

[Vince Bonfanti]

I'm trying to send binary data--a serialized Java object--via the
TaskOptions.payload() method, but can't figure out how to send and/or
read the data properly. First, I serialized an object to a byte[],
which I know is working properly. Then I set the byte[] as the task
payload via:

TaskOptions.payload( byte[], contentType );

I've tried different values for contentType;
"application/octet-stream" and "application/x-java-serialized-object"
seemed like the two most reasonable, but this seems to have no effect.

In my task handler (a servlet), I read the data into a byte[] prior to
object deserialization via the following:

   byte[] bytesIn = new byte[ req.getContentLength() ];
   req.getInputStream().readLine( bytesIn, 0, bytesIn.length );

However, the byte[] I read in is not the same as the byte[] I wrote
out. Here are the first few bytes of the array as I wrote it to
TaskOptions.payload (this is cut-and-paste from the Eclipse debugger):

   -84, -19, 0, 5, 115, 114, 0, 38, 99, 111, 109, 46, 103, 111,

and here are the first few bytes I read in, which is tantalizingly
close to what was written out:

63,  63, 5, 115, 114, 38, 99, 111, 109, 46, 103, 111

So then I tried a different tack. I converted the byte[] to a string
in the outward direction:

   TaskOptions.payload( new String( byte[] ) );

and then used req.getReader() to read it in my task handler:

   byte[] bytesIn = req.getReader().readLine().getBytes();

This time I got even closer to my original byte[], but it's still
missing all of the "0" bytes:

   -84, -19,  5, 115, 114,  38, 99, 111, 109, 46, 103, 111, 111

Note that in both cases, the content length reported in the request
header is smaller than the byte[] length when I invoked
TaskOptions.payload().

How do I send a byte[] as the task payload? Any clues would be appreciated.

Vince

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: sending binary data (serialized object) via TaskOptions.payload()

[Vince Bonfanti]

After two days of banging my head against this, it turns out to be a
bug in the development environment, but works fine on the production
servers. Silly me for thinking you should make things work in
development first before deploying to production...  ;-)

Issue #2097 in case anyone's interested in the gory details.

Vince

On Wed, Sep 9, 2009 at 8:20 AM, Vince Bonfanti  wrote:
> I'm trying to send binary data--a serialized Java object--via the
> TaskOptions.payload() method, but can't figure out how to send and/or
> read the data properly. First, I serialized an object to a byte[],
> which I know is working properly. Then I set the byte[] as the task
> payload via:
>
>    TaskOptions.payload( byte[], contentType );
>
> I've tried different values for contentType;
> "application/octet-stream" and "application/x-java-serialized-object"
> seemed like the two most reasonable, but this seems to have no effect.
>
> In my task handler (a servlet), I read the data into a byte[] prior to
> object deserialization via the following:
>
>   byte[] bytesIn = new byte[ req.getContentLength() ];
>   req.getInputStream().readLine( bytesIn, 0, bytesIn.length );
>
> However, the byte[] I read in is not the same as the byte[] I wrote
> out. Here are the first few bytes of the array as I wrote it to
> TaskOptions.payload (this is cut-and-paste from the Eclipse debugger):
>
>   -84, -19, 0, 5, 115, 114, 0, 38, 99, 111, 109, 46, 103, 111,
>
> and here are the first few bytes I read in, which is tantalizingly
> close to what was written out:
>
>    63,  63, 5, 115, 114, 38, 99, 111, 109, 46, 103, 111
>
> So then I tried a different tack. I converted the byte[] to a string
> in the outward direction:
>
>   TaskOptions.payload( new String( byte[] ) );
>
> and then used req.getReader() to read it in my task handler:
>
>   byte[] bytesIn = req.getReader().readLine().getBytes();
>
> This time I got even closer to my original byte[], but it's still
> missing all of the "0" bytes:
>
>   -84, -19,  5, 115, 114,  38, 99, 111, 109, 46, 103, 111, 111
>
> Note that in both cases, the content length reported in the request
> header is smaller than the byte[] length when I invoked
> TaskOptions.payload().
>
> How do I send a byte[] as the task payload? Any clues would be appreciated.
>
> Vince
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Bulk writes to datastore

[Vince Bonfanti]

This is already being done:

http://code.google.com/p/gaevfs/

I recently adding a CachingDatastoreService class that implements a
write-behind cache using task queues (instead of cron), which will be
included in the next release. The code is available now in SVN:

   
http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/datastore/CachingDatastoreService.java

GaeVFS supports two "filesystem"-like APIs, one based on Apache
Commons VFS (http://commons.apache.org/vfs/) and another based on the
JDK7 NIO2 project (http://openjdk.java.net/projects/nio/). The latter
API will also be in the next GaeVFS release, and the code is now in
SVN:

   
http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/nio/

Vince

On Sat, Sep 12, 2009 at 2:53 PM, Larry Cable  wrote:
>
> I am wondering about writing a Servlet that would form/multi-part
> upload large files and cache them in memcache then use the
> cron API to "trickle" persist them into the DS over time ...
>
> could maybe even get adventurous and put a "filesystem"-like API
> over the cache ...
>
> lemme know if anyone would be interested in this ... if there is
> enough interest I'll put something together and open source it.
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: What's the Java equivalent of appcfg.py?

[Vince Bonfanti]

The GaeVFS (Google App Engine Virtual File System) project implements
a distributed, writable file system for Google App Engine:

   http://code.google.com/p/gaevfs/

The current release (0.3) supports an API based on Apache Commons VFS
(http://commons.apache.org/vfs/). The latest code in SVN--which has
not yet been packaged as a release--adds support for an API based on
the JDK7 NIO2 project (http://openjdk.java.net/projects/nio/).

GaeVFS also includes servlet for uploading and download files, and for
doing directory listings:

   
http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/servlet/GaeVfsServlet.java

Finally, GaeVFS also implements a pluggable file system for the H2
relational database to provide SQL database support for Google App
Engine; this is considered experimental for now:

http://code.google.com/p/gaevfs/wiki/H2GAE

Vince

On Mon, Sep 14, 2009 at 2:52 PM, Sam Sach  wrote:
>
> Hi:
>
> I'd like my app to download and upload from the datastore -- that is,
> to work as a file system.  The only API, however, that I've found that
> does this is written in Python.  I'd like to know what if there's a
> Java equivalent.
>
> --Sam
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: testing applications in GAE - jUnit4

[Vince Bonfanti]

Hi Nicolas,

I'm interested in seeing what you've done. Thanks.

Vince

On Tue, Sep 15, 2009 at 9:33 AM, Nicolas Melendez
 wrote:
> Hi EveryBody:
>
> The how-to for testing in GAE is for Junit3
> , there should be also for junit4 which has
> -Annotations
> -Assert of expeted Exceptions
> -Test classes don't extends from TestCase, so we are more free to design
> class hirarchy.
> Also the migration from Junit3 to Junit4 isn't so easy(but not imposible)
> i have made my own implementation for GAE, i can share if people are
> interested.
>
> Thanks, bye!
> NM - Java Developer
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: WARNING: multiple versions of ant detected in path for junit

[Vince Bonfanti]

Yes, I see it and have been ignoring it.

Vince

On Wed, Sep 16, 2009 at 9:58 AM, Sam  wrote:
>
> I have an ant script to run my junit tests which includes appengine-
> java-sdk-1.2.5/lib/impl/appengine-local-runtime.jar.  When I run a
> test I get a warning from junit:
>
>    [junit] WARNING: multiple versions of ant detected in path for
> junit
>    [junit]          jar:file:/D:/bin/eclipse-galileo/plugins/
> org.apache.ant_1.7.1.v20090120-1145/lib/ant.jar!/org/apache/tools/ant/
> Project.class
>    [junit]      and jar:file:/D:/downloads/appengine-java-sdk-1.2.5/
> lib/impl/appengine-local-runtime.jar!/org/apache/tools/ant/
> Project.class
>
> everything works fine, but it's annoying that ant is packaged in
> appengine-local-runtime.jar.  Is there a reason for this?  Anyone else
> encountered this?
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Wrong default urls in TaskQueue. Bug?

[Vince Bonfanti]

I've noticed the same problem. I think it's a bug.

Vince

On Wed, Sep 16, 2009 at 1:53 PM, oizo  wrote:
>
> if i fill queue without TaskOption.url:
> queue = QueueFactory.getDefaultQueue()
> queue.add() or
>
> queue = QueueFactory.getQueue("myqueue")
> queue.add()
>
> i get URLs in local and prod: "/_ah/queue"
> instead of "/_ah/queue/default" or "/_ah/queue/myqueue"
>
> Is it a bug or am i doing something wrong? If i use Python SDK,
> default urls are correct...
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Creating static files from a servlet

[Vince Bonfanti]

Are there any details available for this feature other than this
one-liner? Thanks.

On Wed, Sep 16, 2009 at 1:50 PM, Don Schwarz  wrote:
>
> The "Service for storing and serving large files" item on the Roadmap will
> give you what you want when it launches:
> http://code.google.com/appengine/docs/roadmap.html
> In the mean time your current approach sounds reasonable.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Writing to a local file system using gaevfs

[Vince Bonfanti]

Yes, GaeVFS (http://code.google.com/p/gaevfs/) provides access to the
local file system so that you can use a single API for accessing both
local and "virtual" files. However, local files are read-only; and,
you cannot create a virtual file with the same path and name as a
local file.

Vince

On Thu, Sep 24, 2009 at 5:10 PM, Sam Sach  wrote:
>
> Never mind.  I think I answered my own question.
>
> --Sam
>
> On Sep 24, 4:20 pm, Sam Sach  wrote:
>> Sorry, I meant to write: "if gaevfs is READ only to local file
>> systems".
>>
>> --Sam
>>
>> On Sep 24, 4:15 pm, Sam Sach  wrote:
>>
>> > I'd just like to know if this is possible -- or if GAE is write only
>> > (to local file systems)?
>>
>> > --Sam
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Need GAE logs explanation

[Vince Bonfanti]

RE: "One way to tell is by registering a load-on-startup servlet and
logging from its init method."

Is there any sort of unique identifier for the application instance
that we can include in the log message? I'd like to be able to monitor
how many instances of my application are running. I don't care what
the identifier is and wouldn't try to use it for any other purpose.
The only thing I can find that might work is the "user.dir" system
property, which seems to vary based on application instance.


On Fri, Sep 25, 2009 at 11:26 AM, Toby Reyelts  wrote:
> Yes, it's very possible for two requests (serial or concurrent) to land on
> two different applications instances, one being newly initialized, the other
> already warm. (Does not technically have to be physically separate
> machines).
>
> On Fri, Sep 25, 2009 at 11:01 AM, mably  wrote:
>>
>> Thanx for your answer, the response times seems to be back to normal.
>> I'll keep an eye on it.
>>
>> May be the two pages where served from different machines ?
>>
>> On 25 sep, 16:48, Toby Reyelts  wrote:
>> > Is your application being initialized when you hit that URL? (One way to
>> > tell is by registering a load-on-startup servlet and logging from its
>> > init
>> > method). If so, what's likely happening is that you're not including
>> > your
>> > app initialization when you only time the service method of your
>> > servlet.
>> >
>> >
>> >
>> > On Thu, Sep 24, 2009 at 11:41 AM, mably  wrote:
>> >
>> > > Hello everybody,
>> >
>> > > I dont really understand the logs GAE shows me in the admin console :
>> >
>> > > I have a first URI showing a simple search form that gives me :
>> >
>> > > 09-24 08:24AM 30.341 /default/ftsearch.do?_contenttype_=content 200
>> > > 117ms 103cpu_ms 73api_cpu_ms
>> >
>> > > But my servlet service method lasts only :
>> >
>> > > INFO : com.mably.cms.web.DispatcherServlet service: 6ms
>> >
>> > > And I have a second URI listing data from the datastore that gives
>> > > me :
>> >
>> > > 09-24 08:24AM 28.575 /default/notifications.do 200 38ms 27cpu_ms
>> > > 13api_cpu_ms
>> >
>> > > INFO : com.mably.cms.web.DispatcherServlet service: 10ms
>> >
>> > > What's annoying me is that the fastest (6ms versus 10ms) servlet seems
>> > > to last longer for GAE (117ms vs 38ms) and to take more CPU.
>> >
>> > > Can somebody explain these numbers to me ?
>> >
>> > > Thanx in advance.
>> >
>> > > Francois
>> > > Bordeaux, FRANCE
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Photo and Video services

[Vince Bonfanti]

Hi Diana,

I've created GaeVFS to solve this problem:

http://code.google.com/p/gaevfs/

You can view a demonstration here:

http://gaevfs.appspot.com/

Note that the current released version (0.3) will only upload about
2.0MB before timing out; the latest code in SVN will support the full
10.0MB allowed by Google App Engine.

There's a feature on the roadmap called "Service for storing and
serving large files," but there's apparently no additional information
available on this feature:

http://code.google.com/appengine/docs/roadmap.html

Vince

On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise  wrote:
>
> What options do I have in GAE to allow Users to upload, store, and
> view media (photos, video, audio, etc) from my within my application?
>
> Is there a special data type in datastore that would be used to store
> media?  If I store a media item in datastore then how do I display it
> to the User?  In a regular app server, I would do this as a static
> file reference where I first transfer the data from the database to a
> file accessible under webroot (if it is not already present).  But in
> GAE the application can NOT create files in the static area due to
> permissions...so is there another technique to accomplish this?
>
> Would videos best be served indirectly via youtube?  If so, how do I
> know they will NOT be removed prematurely by youtube?
>
> Thanks.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Photo and Video services

[Vince Bonfanti]

You were unable to upload a small photo on my demo site, or in your
development environment? I just successfully uploaded an image to the
demo site, which you can download from here:

http://gaevfs.appspot.com/gaevfs/images/

Yes, the images get stored in "blocks" of 1MB or less. GaeVFS
implements a virtual file system, so you refer to the file using URL
links, just as you would for a static file. For example, here's an
image I just uploaded:

http://gaevfs.appspot.com/gaevfs/images/img4.jpg

The GaeVfsServlet handles both upload and download, and demonstrates
how this is done:


http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/servlet/GaeVfsServlet.java

The GaeVfsServlet also handles setting the content type for the response.

Vince

On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise  wrote:
>
> Vince, I was unable to upload small photos...getting error code 500!
>
> What is the basic principle behind this solution?  It appears you are
> storing the file in datastore in increments of 1M or less...along the
> lines of what Raphael was getting at.  How do you display a list of
> images from the datastore on a webpage?
>
> On Oct 22, 1:33 pm, Vince Bonfanti  wrote:
>> Hi Diana,
>>
>> I've created GaeVFS to solve this problem:
>>
>>    http://code.google.com/p/gaevfs/
>>
>> You can view a demonstration here:
>>
>>    http://gaevfs.appspot.com/
>>
>> Note that the current released version (0.3) will only upload about
>> 2.0MB before timing out; the latest code in SVN will support the full
>> 10.0MB allowed by Google App Engine.
>>
>> There's a feature on the roadmap called "Service for storing and
>> serving large files," but there's apparently no additional information
>> available on this feature:
>>
>>    http://code.google.com/appengine/docs/roadmap.html
>>
>> Vince
>>
>>
>>
>> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise  
>> wrote:
>>
>> > What options do I have in GAE to allow Users to upload, store, and
>> > view media (photos, video, audio, etc) from my within my application?
>>
>> > Is there a special data type in datastore that would be used to store
>> > media?  If I store a media item in datastore then how do I display it
>> > to the User?  In a regular app server, I would do this as a static
>> > file reference where I first transfer the data from the database to a
>> > file accessible under webroot (if it is not already present).  But in
>> > GAE the application can NOT create files in the static area due to
>> > permissions...so is there another technique to accomplish this?
>>
>> > Would videos best be served indirectly via youtube?  If so, how do I
>> > know they will NOT be removed prematurely by youtube?
>>
>> > Thanks.- Hide quoted text -
>>
>> - Show quoted text -
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Photo and Video services

[Vince Bonfanti]

Can you try a different browser? From the logs it looks like you're
using IE7. I just tried with IE8, Chrome, and Firefox 3 (all on
Windows) and they all worked fine. Something about the path being sent
by the browser is causing the error--if we can narrow it to IE7 then I
can investigate further.

The only "degradation" I've ever seen is that larger files take longer
to download, which you'd expect. However, I've never done any real
load or stress testing.

Vince

On Thu, Oct 22, 2009 at 6:07 PM, Diana Cruise  wrote:
>
> Actually, I tried simple files without spaces also and they failed
> too.  When I hit your photo I noticed execellent response time...have
> you noticed any particular degradation when displaying lists of
> photos, for example?
>
> On Oct 22, 3:58 pm, Diana Cruise  wrote:
>> Thanks Baz.
>>
>> Vince, I hit your demo site and just entered a path like /gaevfs/mypic
>> or /gaevfs/images/mypic, then selected a local small photo.  On
>> submitting the form, received the 500 error...perhaps the photo had
>> spaces in the name?
>>
>> On Oct 22, 3:10 pm, Vince Bonfanti  wrote:
>>
>>
>>
>> > You were unable to upload a small photo on my demo site, or in your
>> > development environment? I just successfully uploaded an image to the
>> > demo site, which you can download from here:
>>
>> >    http://gaevfs.appspot.com/gaevfs/images/
>>
>> > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS
>> > implements a virtual file system, so you refer to the file using URL
>> > links, just as you would for a static file. For example, here's an
>> > image I just uploaded:
>>
>> >    http://gaevfs.appspot.com/gaevfs/images/img4.jpg
>>
>> > The GaeVfsServlet handles both upload and download, and demonstrates
>> > how this is done:
>>
>> >    http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant...
>>
>> > The GaeVfsServlet also handles setting the content type for the response.
>>
>> > Vince
>>
>> > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise  
>> > wrote:
>>
>> > > Vince, I was unable to upload small photos...getting error code 500!
>>
>> > > What is the basic principle behind this solution?  It appears you are
>> > > storing the file in datastore in increments of 1M or less...along the
>> > > lines of what Raphael was getting at.  How do you display a list of
>> > > images from the datastore on a webpage?
>>
>> > > On Oct 22, 1:33 pm, Vince Bonfanti  wrote:
>> > >> Hi Diana,
>>
>> > >> I've created GaeVFS to solve this problem:
>>
>> > >>    http://code.google.com/p/gaevfs/
>>
>> > >> You can view a demonstration here:
>>
>> > >>    http://gaevfs.appspot.com/
>>
>> > >> Note that the current released version (0.3) will only upload about
>> > >> 2.0MB before timing out; the latest code in SVN will support the full
>> > >> 10.0MB allowed by Google App Engine.
>>
>> > >> There's a feature on the roadmap called "Service for storing and
>> > >> serving large files," but there's apparently no additional information
>> > >> available on this feature:
>>
>> > >>    http://code.google.com/appengine/docs/roadmap.html
>>
>> > >> Vince
>>
>> > >> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise 
>> > >>  wrote:
>>
>> > >> > What options do I have in GAE to allow Users to upload, store, and
>> > >> > view media (photos, video, audio, etc) from my within my application?
>>
>> > >> > Is there a special data type in datastore that would be used to store
>> > >> > media?  If I store a media item in datastore then how do I display it
>> > >> > to the User?  In a regular app server, I would do this as a static
>> > >> > file reference where I first transfer the data from the database to a
>> > >> > file accessible under webroot (if it is not already present).  But in
>> > >> > GAE the application can NOT create files in the static area due to
>> > >> > permissions...so is there another technique to accomplish this?
>>
>> > >> > Would videos best be served indirectly via youtube?  If so, how do I
>> > >> > know they will NOT be removed prematurely by youtube?
>>
>> > >> > Thanks.- Hide quoted text -
>>
>> > >> - Show quoted text -- Hide quoted text -
>>
>> > - Show quoted text -- Hide quoted text -
>>
>> - Show quoted text -
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Photo and Video services

[Vince Bonfanti]

That's the most GAE will let you upload in one request (for now).
Again, there's a "Service for storing and serving large files" feature
on the roadmap, but the details of this feature have not be revealed.

Vince

On Thu, Oct 22, 2009 at 7:40 PM, Diana Cruise  wrote:
>
> What is the nature of the 10Mb limit again?
>
> On Oct 22, 6:38 pm, Diana Cruise  wrote:
>> Yes, that worked ok from another system so it must be an IE7
>> problem...good catch!
>>
>> On Oct 22, 5:55 pm, Vince Bonfanti  wrote:
>>
>>
>>
>> > Can you try a different browser? From the logs it looks like you're
>> > using IE7. I just tried with IE8, Chrome, and Firefox 3 (all on
>> > Windows) and they all worked fine. Something about the path being sent
>> > by the browser is causing the error--if we can narrow it to IE7 then I
>> > can investigate further.
>>
>> > The only "degradation" I've ever seen is that larger files take longer
>> > to download, which you'd expect. However, I've never done any real
>> > load or stress testing.
>>
>> > Vince
>>
>> > On Thu, Oct 22, 2009 at 6:07 PM, Diana Cruise  
>> > wrote:
>>
>> > > Actually, I tried simple files without spaces also and they failed
>> > > too.  When I hit your photo I noticed execellent response time...have
>> > > you noticed any particular degradation when displaying lists of
>> > > photos, for example?
>>
>> > > On Oct 22, 3:58 pm, Diana Cruise  wrote:
>> > >> Thanks Baz.
>>
>> > >> Vince, I hit your demo site and just entered a path like /gaevfs/mypic
>> > >> or /gaevfs/images/mypic, then selected a local small photo.  On
>> > >> submitting the form, received the 500 error...perhaps the photo had
>> > >> spaces in the name?
>>
>> > >> On Oct 22, 3:10 pm, Vince Bonfanti  wrote:
>>
>> > >> > You were unable to upload a small photo on my demo site, or in your
>> > >> > development environment? I just successfully uploaded an image to the
>> > >> > demo site, which you can download from here:
>>
>> > >> >    http://gaevfs.appspot.com/gaevfs/images/
>>
>> > >> > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS
>> > >> > implements a virtual file system, so you refer to the file using URL
>> > >> > links, just as you would for a static file. For example, here's an
>> > >> > image I just uploaded:
>>
>> > >> >    http://gaevfs.appspot.com/gaevfs/images/img4.jpg
>>
>> > >> > The GaeVfsServlet handles both upload and download, and demonstrates
>> > >> > how this is done:
>>
>> > >> >    
>> > >> > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant...
>>
>> > >> > The GaeVfsServlet also handles setting the content type for the 
>> > >> > response.
>>
>> > >> > Vince
>>
>> > >> > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise 
>> > >> >  wrote:
>>
>> > >> > > Vince, I was unable to upload small photos...getting error code 500!
>>
>> > >> > > What is the basic principle behind this solution?  It appears you 
>> > >> > > are
>> > >> > > storing the file in datastore in increments of 1M or less...along 
>> > >> > > the
>> > >> > > lines of what Raphael was getting at.  How do you display a list of
>> > >> > > images from the datastore on a webpage?
>>
>> > >> > > On Oct 22, 1:33 pm, Vince Bonfanti  wrote:
>> > >> > >> Hi Diana,
>>
>> > >> > >> I've created GaeVFS to solve this problem:
>>
>> > >> > >>    http://code.google.com/p/gaevfs/
>>
>> > >> > >> You can view a demonstration here:
>>
>> > >> > >>    http://gaevfs.appspot.com/
>>
>> > >> > >> Note that the current released version (0.3) will only upload about
>> > >> > >> 2.0MB before timing out; the latest code in SVN will support the 
>> > >> > >> full
>> > >> > >> 10.0MB allowed by Google App Engine.
>>
>> > >> > >> There's a feature on the roadmap called "Se

[appengine-java] Re: Photo and Video services

[Vince Bonfanti]

No, not yet. It's on the TODO list (see line 171 of the GaeVfsServlet
source code). If you--or anyone else--wants to add this I'll be happy
to accept a patch.

Vince

On Fri, Oct 23, 2009 at 11:42 AM, Diana Cruise  wrote:
>
> In terms of performance does gaevfs accmmodate http resource timestamp
> checks to avoid re-downloading repeated requests for the same images?
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: extracting zip files on GAE

[Vince Bonfanti]

I assume you're using GaeVFS (http://code.google.com/p/gaevfs/)?
Basically, you need to create a java.util.ZipInputStream instance to
read the zip file.

GaeVFS implements a file system API based on Apache Commons VFS
(http://commons.apache.org/vfs/). To open an InputStream for a file
using GaeVFS do the following:

FileSystemManager fsManager = GaeVFS.getManager();
FileObject zipFileObject = fsManager.resolveFile(
"/path/to/myZipFile.zip" );
InputStream in = zipFileObject.getContent().getInputStream();

Now you can create a ZipInputStream to read the file:

ZipInputStream zipIn = new ZipInputStream( in );

If you're so inclined, you can reduce this all to:

ZipInputStream zipIn = new ZipInputStream( GaeVFS.resolveFile(
"/path/to/myZipFile.zip" ).getContent().getInputStream() );

Don't forget to close the ZipInputStream when you're finished with it.

See the following for more information:

http://code.google.com/p/gaevfs/wiki/UsingGaeVFS
http://code.google.com/p/gaevfs/wiki/CombinedLocalOption
http://code.google.com/p/gaevfs/wiki/ApplicationPortability

Vince

On Fri, Oct 23, 2009 at 5:09 AM, deft  wrote:
>
> once I have uploaded a zip file onto the GAE Virtual File System, how
> do I access the zip entries. am really stuck here. could someone help
> me out?!
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] trouble with SDK 1.2.6

[Vince Bonfanti]

I'm having no luck with SDK 1.2.6 within Eclipse 3.5.1 (Windows). Yes,
I've added the -javaagent VM argument to my debug configuration.
However, if I try to do almost anything at all within my
ServletContextListener.contextInitialized() method I get this:

2009-10-23 17:48:59.192::INFO:  Logging to STDERR via org.mortbay.log.StdErrLog
2009-10-23 17:48:59.526::INFO:  jetty-6.1.x
2009-10-23 17:49:00.487::WARN:  failed
com.google.apphosting.utils.jetty.devappenginewebappcont...@429c19{/,C:\Users\vinceb\workspace\gaevfs\war}
java.lang.NoClassDefFoundError:
com/google/appengine/tools/development/agent/AppEngineDevAgent
at 
com.google.appengine.tools.development.agent.runtime.Runtime.(Runtime.java:32)
at 
com.newatlanta.appengine.servlet.GaeVfsServletEventListener.contextInitialized(GaeVfsServletEventListener.java:48)
at 
org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
at 
org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
at 
org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
at 
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
com.google.appengine.tools.development.JettyContainerService.startContainer(JettyContainerService.java:181)
at 
com.google.appengine.tools.development.AbstractContainerService.startup(AbstractContainerService.java:116)
at 
com.google.appengine.tools.development.DevAppServerImpl.start(DevAppServerImpl.java:217)
at 
com.google.appengine.tools.development.DevAppServerMain$StartAction.apply(DevAppServerMain.java:162)
at 
com.google.appengine.tools.util.Parser$ParseResult.applyArgs(Parser.java:48)
at 
com.google.appengine.tools.development.DevAppServerMain.(DevAppServerMain.java:113)
at 
com.google.appengine.tools.development.DevAppServerMain.main(DevAppServerMain.java:89)
Caused by: java.lang.ClassNotFoundException:
com.google.appengine.tools.development.agent.AppEngineDevAgent
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at 
com.google.appengine.tools.development.IsolatedAppClassLoader.loadClass(IsolatedAppClassLoader.java:151)
at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
... 20 more
2009-10-23 17:49:00.489::WARN:  failed
jettycontainerservice$apiproxyhand...@116318b
java.lang.NoClassDefFoundError:
com/google/appengine/tools/development/agent/AppEngineDevAgent
at 
com.google.appengine.tools.development.agent.runtime.Runtime.(Runtime.java:32)
at 
com.newatlanta.appengine.servlet.GaeVfsServletEventListener.contextInitialized(GaeVfsServletEventListener.java:48)
at 
org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
at 
org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
at 
org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
at 
org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
at org.mortbay.jetty.Server.doStart(Server.java:217)
at 
org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
at 
com.google.appengine.tools.development.JettyContainerService.startContainer(JettyContainerService.java:181)
at 
com.google.appengine.tools.development.AbstractContainerService.startup(AbstractContainerService.java:116)
at 
com.google.appengine.tools.development.DevAppServerImpl.start(DevAppServerImpl.java:217)
at 
com.google.appengine.tools.development.DevAppServerMain$StartAction.apply(DevAppServerMain.java:162)
at 
com.google.appengine.tools.util.Parser$ParseResult.applyArgs(Parser.java:48)
 

[appengine-java] SDK does not upload files that start with "."

[Vince Bonfanti]

The SDK (1.2.5) does not upload file with names that start with "."
such as ".h2.server.properties". I assume this is because such files
are considered hidden on Linux/UNIX; but, I'm running on Windows, so
this must be something in the SDK itself and not caused by the file
system.

Is this intentional, or should I open a bug report?

Vince

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: trouble with SDK 1.2.6

[Vince Bonfanti]

I just figured out the problem. I had added appengine-api-stubs.jar
and appengine-local-runtime.jar to my project build path to support
junit testing. Upgrading to the 1.2.6 versions only changed the error
message, but removing them from the build path solved the problem. Now
I just have to figure out how to configure things properly to support
junit testing...

Thanks.

Vince

2009/10/23 Miguel Méndez :
> I'm going to check and see if I can reproduce this using your project.
>
> On Fri, Oct 23, 2009 at 2:06 PM, Vince Bonfanti  wrote:
>>
>> I'm having no luck with SDK 1.2.6 within Eclipse 3.5.1 (Windows). Yes,
>> I've added the -javaagent VM argument to my debug configuration.
>> However, if I try to do almost anything at all within my
>> ServletContextListener.contextInitialized() method I get this:
>>
>> 2009-10-23 17:48:59.192::INFO:  Logging to STDERR via
>> org.mortbay.log.StdErrLog
>> 2009-10-23 17:48:59.526::INFO:  jetty-6.1.x
>> 2009-10-23 17:49:00.487::WARN:  failed
>>
>> com.google.apphosting.utils.jetty.devappenginewebappcont...@429c19{/,C:\Users\vinceb\workspace\gaevfs\war}
>> java.lang.NoClassDefFoundError:
>> com/google/appengine/tools/development/agent/AppEngineDevAgent
>>        at
>> com.google.appengine.tools.development.agent.runtime.Runtime.(Runtime.java:32)
>>        at
>> com.newatlanta.appengine.servlet.GaeVfsServletEventListener.contextInitialized(GaeVfsServletEventListener.java:48)
>>        at
>> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
>>        at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
>>        at
>> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1218)
>>        at
>> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:500)
>>        at
>> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:448)
>>        at
>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
>>        at
>> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
>>        at
>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
>>        at
>> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:117)
>>        at org.mortbay.jetty.Server.doStart(Server.java:217)
>>        at
>> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:40)
>>        at
>> com.google.appengine.tools.development.JettyContainerService.startContainer(JettyContainerService.java:181)
>>        at
>> com.google.appengine.tools.development.AbstractContainerService.startup(AbstractContainerService.java:116)
>>        at
>> com.google.appengine.tools.development.DevAppServerImpl.start(DevAppServerImpl.java:217)
>>        at
>> com.google.appengine.tools.development.DevAppServerMain$StartAction.apply(DevAppServerMain.java:162)
>>        at
>> com.google.appengine.tools.util.Parser$ParseResult.applyArgs(Parser.java:48)
>>        at
>> com.google.appengine.tools.development.DevAppServerMain.(DevAppServerMain.java:113)
>>        at
>> com.google.appengine.tools.development.DevAppServerMain.main(DevAppServerMain.java:89)
>> Caused by: java.lang.ClassNotFoundException:
>> com.google.appengine.tools.development.agent.AppEngineDevAgent
>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>        at
>> com.google.appengine.tools.development.IsolatedAppClassLoader.loadClass(IsolatedAppClassLoader.java:151)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>>        ... 20 more
>> 2009-10-23 17:49:00.489::WARN:  failed
>> jettycontainerservice$apiproxyhand...@116318b
>> java.lang.NoClassDefFoundError:
>> com/google/appengine/tools/development/agent/AppEngineDevAgent
>>        at
>> com.google.appengine.tools.development.agent.runtime.Runtime.(Runtime.java:32)
>>        at
>> com.newatlanta.appengine.servlet.GaeVfsServletEventListener.contextInitialized(GaeVfsServletEventListener.java:48)
>>        at
>> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:530)
>>        at org.mortbay.jetty.servlet.Context.startContext(Context.java:135)
>>        at
>> org.mortbay.jetty.webapp.WebAppContext.startContext(We

[appengine-java] Re: trouble with SDK 1.2.6

[Vince Bonfanti]

It looks like the Eclipse debug configuration automatically picks up
the complete build path for the project as its classpath. Manually
editing the debug configuration to remove appengine-api-stubs.jar and
appengine-local-runtime.jar from the classpath fixes the problem.

Vince

On Fri, Oct 23, 2009 at 3:01 PM, Vince Bonfanti  wrote:
>
> I just figured out the problem. I had added appengine-api-stubs.jar
> and appengine-local-runtime.jar to my project build path to support
> junit testing. Upgrading to the 1.2.6 versions only changed the error
> message, but removing them from the build path solved the problem. Now
> I just have to figure out how to configure things properly to support
> junit testing...
>
> Thanks.
>
> Vince
>
> 2009/10/23 Miguel Méndez :
>> I'm going to check and see if I can reproduce this using your project.
>>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Eclipse Plugin Update for App Engine SDK 1.2.7?

[Vince Bonfanti]

I think the 1.2.7 update is only for Python. The latest Java SDK is 1.2.6.

On Fri, Oct 23, 2009 at 12:02 PM, hildenl  wrote:
>
> Has anyone been able to update to 1.2.7 from Eclipse?  I've got the
> http://dl.google.com/eclipse/plugin/3.5 configured as a software site
> and its enabled, but when I "Check for Updates", 1.2.7 never shows up
> as an option and yet its been out since the 18th,
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: trouble with SDK 1.2.6

[Vince Bonfanti]

No, that didn't help. Again, just to make sure I'm being clear:

 - The only reason appengine-api-stubs.jar and
appengine-local-runtime.jar are in my project classpath is to be able
to compile my junit tests. They're not needed for debugging my
project. However...

 - When creating a debug configuration, Eclipse automatically uses the
project classpath by default, which causes the exceptions I showed
earlier when trying to debug. Adding appengine-agent.jar to the top of
the project and debug classpaths did not solve the problem.

 - By manually editing my debug configuration to remove
appengine-api-stubs.jar and appengine-local-runtime.jar from the debug
classpath, everything works fine.

At this point I'm happy.

Vince

On Fri, Oct 23, 2009 at 3:55 PM, Toby Reyelts  wrote:
> Vince,
>
> You should be able to workaround the problem you're having with your unit
> tests by adding $SDK/lib/agent/appengine-agent.jar to the very top of your
> system classpath. Please let me know if that does not work for you.
>
> On Fri, Oct 23, 2009 at 3:23 PM, Vince Bonfanti  wrote:
>>
>> It looks like the Eclipse debug configuration automatically picks up
>> the complete build path for the project as its classpath. Manually
>> editing the debug configuration to remove appengine-api-stubs.jar and
>> appengine-local-runtime.jar from the classpath fixes the problem.
>>
>> Vince
>>
>> On Fri, Oct 23, 2009 at 3:01 PM, Vince Bonfanti 
>> wrote:
>> >
>> > I just figured out the problem. I had added appengine-api-stubs.jar
>> > and appengine-local-runtime.jar to my project build path to support
>> > junit testing. Upgrading to the 1.2.6 versions only changed the error
>> > message, but removing them from the build path solved the problem. Now
>> > I just have to figure out how to configure things properly to support
>> > junit testing...
>> >
>> > Thanks.
>> >
>> > Vince
>> >
>> > 2009/10/23 Miguel Méndez :
>> >> I'm going to check and see if I can reproduce this using your project.
>> >>
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: Dealing with unsupported APIs (not listed on JRE Class White List)

[Vince Bonfanti]

RE: "Since Jena is an open-source (third party) library, I don't have
control over it."

Because Jena is open-source, you *do* have control over it. See what
it would take to modify Jena to run on GAE, then submit your patches
back to the project owners.

On Thu, Oct 29, 2009 at 11:20 AM, Pion  wrote:
>
> I am using Jena which is an open-source Java framework for building
> Semantic Web applications. It's been working fine on my local Eclipse
> development. I have just deployed it and it fails because Jena uses
> java.rmi.server.UID -- please see the error logs below.
>
> java.rmi.server.UID is not listed on
> http://code.google.com/intl/de/appengine/docs/java/jrewhitelist.html.
>
> Since Jena is an open-source (third party) library, I don't have
> control over it. One potential solution is to host Jena outside GAE
> such as Amazon EC2. Obviously, it is not ideal.
> It solves the limitation of the JRE Class White List but it will
> introduce other problems (scalability, another indirection,
> maintenance, etc).
>
> I am sure this is a common issue as many of us are porting existing
> apps/libraries/SDKs to Google App Engine.
>
> I'd appreciate your advice on this issue.
> Thanks in advance for your help.
>
> My environments: GAE Java 1.2.6, GWT 1.7.1, Eclipse(Galileo), Windows
> Vista (32-bit).
>
> ERROR LOGS
> 10-29 07:38AM 59.438 /col/input 500 6020ms 5638cpu_ms 0kb Mozilla/5.0
> (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/
> 3.5.3 (.NET CLR 3.5.30729),gzip(gfe)
> 98.232.111.83 - - [29/Oct/2009:07:39:05 -0700] "POST /col/input HTTP/
> 1.1" 500 113 "http://foafan2000.appspot.com/col/
> 69AAFD5DE4B9915A24B29439E261B4A5.cache.html" "Mozilla/5.0 (Windows; U;
> Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET
> CLR 3.5.30729),gzip(gfe)" "foafan2000.appspot.com"
> E 10-29 07:39AM 05.433
> javax.servlet.ServletContext log: Exception while dispatching incoming
> RPC call
> com.google.gwt.user.server.rpc.UnexpectedException: Service method
> 'public abstract java.lang.String com.col.client.CInputService.getFoaf
> (java.lang.String)' threw an unexpected exception:
> java.lang.NoClassDefFoundError: java.rmi.server.UID is a restricted
> class. Please see the Google App Engine developer's guide for more
> details.
>        at com.google.gwt.user.server.rpc.RPC.encodeResponseForFailure
> (RPC.java:360)
>        at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse
> (RPC.java:546)
>        at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall
> (RemoteServiceServlet.java:166)
>        at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost
> (RemoteServiceServlet.java:86)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
>        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
> 487)
>        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
> (ServletHandler.java:1093)
>        at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter
> (SaveSessionFilter.java:35)
>        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
> (ServletHandler.java:1084)
>        at
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter
> (TransactionCleanupFilter.java:43)
>        at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter
> (ServletHandler.java:1084)
>        at org.mortbay.jetty.servlet.ServletHandler.handle
> (ServletHandler.java:360)
>        at org.mortbay.jetty.security.SecurityHandler.handle
> (SecurityHandler.java:216)
>        at org.mortbay.jetty.servlet.SessionHandler.handle
> (SessionHandler.java:181)
>        at org.mortbay.jetty.handler.ContextHandler.handle
> (ContextHandler.java:712)
>        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
> 405)
>        at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle
> (AppVersionHandlerMap.java:238)
>        at org.mortbay.jetty.handler.HandlerWrapper.handle
> (HandlerWrapper.java:139)
>        at org.mortbay.jetty.Server.handle(Server.java:313)
>        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
> 506)
>        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete
> (HttpConnection.java:830)
>        at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable
> (RpcRequestParser.java:76)
>        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
>        at
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest
> (JettyServletEngineAdapter.java:139)
>        at com.google.apphosting.runtime.JavaRuntime.handleRequest
> (JavaRuntime.java:239)
>        at com.google.apphosting.base.RuntimePb$EvaluationRuntime
> $6.handleBlockingRequest(RuntimePb.java:5135)
>        at com.google.apphosting.base.RuntimePb$EvaluationRuntime
> $6.handleBlockingRequest(RuntimePb.java:5133)
>        at com.googl

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

This looked like an interesting problem, and I already had most of the
pieces in place, so here's my first attempt, which is implemented in a
single class (also attached, along with some test files):


http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

I'm more than happy to contribute this for inclusion in the SDK.

Some caveats:

   1) Because of issue #2097
(http://code.google.com/p/googleappengine/issues/detail?id=2097), this
doesn't work on the development server (it does work in production).
So go star that issue!

   2) I've only done minimal testing on the production server (see the
attached test files).

   3) This post is the only documentation that's currently available.

First, add the following your your web.xml:


Deferred

com.newatlanta.appengine.taskqueue.Deferred


Deferred
/_ah/queue/deferred


Second, define the "deferred" queue within queue.xml (use whatever
rate you want):


deferred
10/s


Next, create a class that implements the
com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
"doTask()" method of this class is where you implement your task
logic.

Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
method to queue up your task:

TestDeferred testDeferred = new TestDeferred(); // implements Deferrable
Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );

(Note that it would be possible to pass arguments as attributes to
your Deferrable instance, rather than using varargs; I'm not sure it
makes much difference which you choose).

Just as for the Python implementation, if the arguments size exceeds
10KB, the arguments are stored in a datastore entity, which is then
deleted when the task is executed. Also, your doTask() method can
throw a PermanentTaskFailure exception to halt retries; any other
exceptions cause the task to be retried.

Let me know if you find this useful, have any questions or encounter
any problems.

Vince

On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)  wrote:
> Hi David. This may be coming to Java eventually, but it hasn't been started
> yet. If you or anyone else is interested in contributing, let me know.
>
> - Jason
>
> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler 
> wrote:
>>
>> Re: http://code.google.com/appengine/articles/deferred.html
>>
>> Will this be coming to AppEngine for Java?
>>
>> David Chandler
>> http://turbomanage.wordpress.com
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---



TestDeferred.java
Description: Binary data


testDeferred.jsp
Description: Binary data


Deferred.java
Description: Binary data

[appengine-java] Re: SimpleDS: an alternative for Datastore persistence

[Vince Bonfanti]

You might be interested in my CachingDatastoreService class:

  
http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/datastore/CachingDatastoreService.java

It has the following features:

 - Implements the com.google.appengine.api.datastore.DatastoreService,
so it's a plug-in replacement for the standard implementation.
 - Automatically caches entities in memcache for datastore reads and writes.
 - Has a write-behind option (the default) that queues all datastore
writes as background tasks (except for transactions, which are always
write-through directly to the datastore).
 - A watchdog task makes sure the write-behind task is always available.
 - If the write-behind task isn't available, defaults to write-through
to insure no loss of data.
 - Supports configurable expiration of memcache entities (the default
is no expiration).

In order to use the CachingDatastoreService, first configure the
write-behind task in web.xml:


CachingDatastoreService

com.newatlanta.appengine.datastore.CachingDatastoreService


CachingDatastoreService
/_ah/queue/write-behind-task


Then configure the write-behind-task queue in queue.xml (use whatever
rate you want):


write-behind-task
5/s


Then replace the following code:

DatastoreService ds = DatastoreServiceFactory().getDatastoreService();

with this code, and then use the DatastoreService methods as you normally would:

DatastoreService ds = new CachingDatastoreService();

The default CachingDatastoreService constructor enables the
CacheOptions.WRITE_BEHIND option and sets the expiration to "null" (no
expiration). There are additional constructors that allow you to
specify CacheOptions.WRITE_THROUGH and/or specify a memcache
expiration value.

Vince

On Mon, Nov 2, 2009 at 2:21 PM, Stakka  wrote:
>
> One feature I see useful for a layer atop the Low-Level API is
> asynchronization. Since you'll never know how much time a Low-Level
> API call will take it should automatically create Tasks for them,
> writes atleast.
>
>
> On Oct 22, 10:37 am, Nacho Coloma  wrote:
>> Hi all,
>>
>> We have been developing a persistence framework for the AppEngine
>> Datastore based on the raw DatastoreService API. For our (simple)
>> persistence case, both JDO and JPA were a bit overkill as we were
>> spending a significant amount of time jumping through hoops to make
>> our application roll, but at the same time the Datastore API was a too
>> low-level solution to be usable.
>>
>> So we reinvented our wheel. In two days.
>>
>> SimpleDS is a light wrapper around the DatastoreService APIs that
>> provide a simple interface for java persistent classes. It does not
>> include fancy stuff or any super features, it's just the
>> DatastoreService ported to a world where Java entities can be
>> persisted directly (using a subset of JPA annotations).  This is _not_
>> a JPA/JDO replacement, and will never be. But we have been using it
>> for some weeks and are quite happy with it.
>>
>> Any kind of feedback from the AppEngine  community would be welcome.
>> Before calling the typical "but we already have JPA/JDO!" argument,
>> please notice the following:
>>
>> * There are lots of considerations in a relational database that do
>> not apply to AppEngine. This allows a specific solution to be
>> simplified big time. Just see the depth of your typical stack trace to
>> understand what I am talking about.
>> * Solutions can be designed for specific cases that are common
>> practice in AppEngine but do not apply to a relational database. See,
>> for example, saving entities with a parent instance.
>> * Transactions also behave a bit differently, where a "one size fits
>> all" approach would probably not be the best solution.
>>
>> To better ilustrate with an example, these are some typical tasks
>> performed with SimpleDS:
>>
>> Retrieve instance:
>> FooBar bar = entityManager.get(key);
>>
>> Transform from Google Datastore Entity to java and viceversa:
>> Entity entity = entityManager.datastoreToJava(bar);
>>
>> Save generating a  primary key, with a parent instance:
>> FooBar bar = new FooBar();
>> entityManager.put(parentKey, bar);
>>
>> More can be seen here:http://code.google.com/p/simpleds/wiki/SimpleTasks
>>
>> Any discussion about the current API state is welcome. This entire
>> thing was rolled in two days and tested in a couple of weeks so there
>> should be some bugs in between.
>>
>> It is important to keep in mind the current list of limitations:
>>
>> * Only the Key class is a supported primary key.
>> * IN and != are not supported (yet). I have big concerns about
>> supporting this, performance-wise.
>> * Relationships are not supported. You can use Keys and collections of
>> Keys for that purpose.
>> * Transactions are not yet included. We are not yet sure about how to
>> proceed here.
>>
>> As I said, this is not conceived to become a feature-complete JPA
>> replacement, so p

[appengine-java] Re: SimpleDS: an alternative for Datastore persistence

[Vince Bonfanti]

P.S. Because of issue 2097
(http://code.google.com/p/googleappengine/issues/detail?id=2097), the
write-behind task doesn't work on the development server, so it always
defaults to write-through (which is a nice test of the watchdog
mechanism). The write-behind task works fine on the production
servers.

On Mon, Nov 2, 2009 at 3:12 PM, Vince Bonfanti  wrote:
> You might be interested in my CachingDatastoreService class:
>
>  http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/datastore/CachingDatastoreService.java
>
> It has the following features:
>
>  - Implements the com.google.appengine.api.datastore.DatastoreService,
> so it's a plug-in replacement for the standard implementation.
>  - Automatically caches entities in memcache for datastore reads and writes.
>  - Has a write-behind option (the default) that queues all datastore
> writes as background tasks (except for transactions, which are always
> write-through directly to the datastore).
>  - A watchdog task makes sure the write-behind task is always available.
>  - If the write-behind task isn't available, defaults to write-through
> to insure no loss of data.
>  - Supports configurable expiration of memcache entities (the default
> is no expiration).
>
> In order to use the CachingDatastoreService, first configure the
> write-behind task in web.xml:
>
>    
>        CachingDatastoreService
>        
> com.newatlanta.appengine.datastore.CachingDatastoreService
>    
>    
>        CachingDatastoreService
>        /_ah/queue/write-behind-task
>    
>
> Then configure the write-behind-task queue in queue.xml (use whatever
> rate you want):
>
>    
>        write-behind-task
>        5/s
>    
>
> Then replace the following code:
>
>    DatastoreService ds = DatastoreServiceFactory().getDatastoreService();
>
> with this code, and then use the DatastoreService methods as you normally 
> would:
>
>    DatastoreService ds = new CachingDatastoreService();
>
> The default CachingDatastoreService constructor enables the
> CacheOptions.WRITE_BEHIND option and sets the expiration to "null" (no
> expiration). There are additional constructors that allow you to
> specify CacheOptions.WRITE_THROUGH and/or specify a memcache
> expiration value.
>
> Vince
>
> On Mon, Nov 2, 2009 at 2:21 PM, Stakka  wrote:
>>
>> One feature I see useful for a layer atop the Low-Level API is
>> asynchronization. Since you'll never know how much time a Low-Level
>> API call will take it should automatically create Tasks for them,
>> writes atleast.
>>
>>
>> On Oct 22, 10:37 am, Nacho Coloma  wrote:
>>> Hi all,
>>>
>>> We have been developing a persistence framework for the AppEngine
>>> Datastore based on the raw DatastoreService API. For our (simple)
>>> persistence case, both JDO and JPA were a bit overkill as we were
>>> spending a significant amount of time jumping through hoops to make
>>> our application roll, but at the same time the Datastore API was a too
>>> low-level solution to be usable.
>>>
>>> So we reinvented our wheel. In two days.
>>>
>>> SimpleDS is a light wrapper around the DatastoreService APIs that
>>> provide a simple interface for java persistent classes. It does not
>>> include fancy stuff or any super features, it's just the
>>> DatastoreService ported to a world where Java entities can be
>>> persisted directly (using a subset of JPA annotations).  This is _not_
>>> a JPA/JDO replacement, and will never be. But we have been using it
>>> for some weeks and are quite happy with it.
>>>
>>> Any kind of feedback from the AppEngine  community would be welcome.
>>> Before calling the typical "but we already have JPA/JDO!" argument,
>>> please notice the following:
>>>
>>> * There are lots of considerations in a relational database that do
>>> not apply to AppEngine. This allows a specific solution to be
>>> simplified big time. Just see the depth of your typical stack trace to
>>> understand what I am talking about.
>>> * Solutions can be designed for specific cases that are common
>>> practice in AppEngine but do not apply to a relational database. See,
>>> for example, saving entities with a parent instance.
>>> * Transactions also behave a bit differently, where a "one size fits
>>> all" approach would probably not be the best solution.
>>>
>>> To better ilustrate with an example, these are some typical tasks
>>> performed with SimpleDS:
>

[appengine-java] Re: SimpleDS: an alternative for Datastore persistence

[Vince Bonfanti]

Great! Since someone else is now using this besides me, I've added
more comments. Also, I've added some testcases and fixed an issue when
invoking put() with entities that contain partial keys. If your
entities don't have complete keys when invoking put(), you should go
get the latest code.

Let me know if you run into any problems, or have suggestions for improvements.

Vince

On Mon, Nov 2, 2009 at 10:47 PM, Roy Smith  wrote:
> Hi Vince
> Thanks for sharing.
> I've modified my persistence framework to incorporate your class and it
> works very well.
> kind regards
> Roy
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: task queue limit >30 sec?

[Vince Bonfanti]

09 at 3:15 PM, Vince Bonfanti  wrote:
> Here's one I got a few days ago. It looks like it timed-out while just
> queuing up a task:
>
> Uncaught exception from servlet
> com.google.apphosting.api.DeadlineExceededException: This request
> (8b0dab1ffe61eb3f) started at 2009/10/31 16:30:47.041 UTC and was
> still executing at 2009/10/31 16:31:16.438 UTC.
>        at sun.misc.Unsafe.park(Native Method)
>        at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
>        at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(Unknown
> Source)
>        at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(Unknown
> Source)
>        at java.util.concurrent.CountDownLatch.await(Unknown Source)
>        at 
> com.google.net.rpc.util.RpcWaiter.waitForRpcsToFinish(RpcWaiter.java:96)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:108)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:37)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:75)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:71)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:71)
>        at 
> com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:37)
>        at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:79)
>        at 
> com.google.appengine.api.labs.taskqueue.QueueApiHelper.makeSyncCall(QueueApiHelper.java:24)
>        at 
> com.google.appengine.api.labs.taskqueue.QueueImpl.add(QueueImpl.java:220)
>        at 
> com.google.appengine.api.labs.taskqueue.QueueImpl.add(QueueImpl.java:210)
>        at 
> com.newatlanta.appengine.datastore.CachingDatastoreService.queueWatchDogTask(CachingDatastoreService.java:388)
>        at 
> com.newatlanta.appengine.datastore.CachingDatastoreService.doGet(CachingDatastoreService.java:374)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>        at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
>        at 
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
>        at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
>        at 
> com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
>        at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
>        at 
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
>        at 
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
>        at 
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
>        at 
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>        at 
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>        at 
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
>        at 
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
>        at 
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238)
>        at 
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
>        at org.mortbay.jetty.Server.handle(Server.java:313)
>        at 
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
>        at 
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
>        at 
> com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
>        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
>        at 
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:139)
>        at 
> com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:239)
>        at 
> com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:5135)
>        at 
> com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:5133)
>        at 
> com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)
>        at 
> com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:363)
>        at com.google.net.rpc.impl.Server$2.run(Server.java:814)
>        at 
> com.google.tracing.LocalTraceSpanRunnable.run(Lo

[appengine-java] Re: task queue limit >30 sec?

[Vince Bonfanti]

Here's one I got a few days ago. It looks like it timed-out while just
queuing up a task:

Uncaught exception from servlet
com.google.apphosting.api.DeadlineExceededException: This request
(8b0dab1ffe61eb3f) started at 2009/10/31 16:30:47.041 UTC and was
still executing at 2009/10/31 16:31:16.438 UTC.
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(Unknown Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(Unknown
Source)
at 
java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(Unknown
Source)
at java.util.concurrent.CountDownLatch.await(Unknown Source)
at 
com.google.net.rpc.util.RpcWaiter.waitForRpcsToFinish(RpcWaiter.java:96)
at 
com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:108)
at 
com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:37)
at 
com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:75)
at 
com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:71)
at java.security.AccessController.doPrivileged(Native Method)
at 
com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:71)
at 
com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:37)
at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:79)
at 
com.google.appengine.api.labs.taskqueue.QueueApiHelper.makeSyncCall(QueueApiHelper.java:24)
at 
com.google.appengine.api.labs.taskqueue.QueueImpl.add(QueueImpl.java:220)
at 
com.google.appengine.api.labs.taskqueue.QueueImpl.add(QueueImpl.java:210)
at 
com.newatlanta.appengine.datastore.CachingDatastoreService.queueWatchDogTask(CachingDatastoreService.java:388)
at 
com.newatlanta.appengine.datastore.CachingDatastoreService.doGet(CachingDatastoreService.java:374)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at 
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093)
at 
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at 
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at 
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084)
at 
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360)
at 
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at 
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at 
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)
at 
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238)
at 
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at 
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506)
at 
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830)
at 
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at 
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:139)
at 
com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:239)
at 
com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:5135)
at 
com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:5133)
at 
com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24)
at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:363)
at com.google.net.rpc.impl.Server$2.run(Server.java:814)
at 
com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56)
at 
com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:516)
at com.google.net.rpc.impl.Server.startRpc(Server.java:769)
at com.google.net.rpc.impl.Server.processRequest(Server.java:351)
at 
com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:437)
at 
com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319)
at 
com.google.n

[appengine-java] Re: Recommended scope for DatastoreService

[Vince Bonfanti]

Yep, I'm going though and making those changes now. Note that the
Queue documentation says, "Implementations of this interface must be
threadsafe." So that one's OK.

Vince

On Thu, Nov 5, 2009 at 12:57 PM, Roy Smith  wrote:
> Hi Vince
> I removed all of the static qualifiers from CachingDatastoreService and I
> instantiate it per request using Guice.
> best
> Roy
>
> On Thu, Nov 5, 2009 at 5:34 PM, Vince Bonfanti  wrote:
>>
>> OK, glad I asked. I assume then the same is true for MemcacheService,
>> since it's not explicitly marked as thread-safe?
>>
>> Vince
>>
>> On Thu, Nov 5, 2009 at 12:26 PM, Max Ross (Google)
>>  wrote:
>> > Since DatastoreService is not explicitly documented as thread-safe, and
>> > since static members are accessible across threads, you should not
>> > create a
>> > static DatastoreService instance and use it across threads.
>> >
>> > On Thu, Nov 5, 2009 at 5:34 AM, Vince Bonfanti 
>> > wrote:
>> >>
>> >> Hi Max,
>> >>
>> >> To your point to "assume that all classes in the api are _not_
>> >> threadsafe unless explicitly documented otherwise." Is there any
>> >> reason not to create a static DatastoreService instance within a
>> >> class:
>> >>
>> >>   private static final DatastoreService datastore =
>> >> DatastoreServiceFactory.getDatastoreService();
>> >>
>> >> Is this static DatastoreService going to be threadsafe?
>> >>
>> >> Vince
>> >>
>> >> On Wed, Nov 4, 2009 at 2:02 PM, Max Ross (Google)
>> >>  wrote:
>> >> > A DatastoreService instance is extremely lightweight so feel free to
>> >> > create
>> >> > them as-needed.  In addition, transactions are not tied to a specific
>> >> > DatastoreService instance but rather to the thread that started the
>> >> > transaction.  Finally, assume that all classes in the api are _not_
>> >> > threadsafe unless explicitly documented otherwise.
>> >> >
>> >> > Hope this helps,
>> >> > Max
>> >> >
>> >> > On Wed, Nov 4, 2009 at 4:35 AM, Roy 
>> >> > wrote:
>> >> >>
>> >> >> Should I be creating a DatastoreService for each "session", or are
>> >> >> they inexpensive enough that I can have one for each Kind? I realise
>> >> >> that for transactions, the latter doesn't work.
>> >> >>
>> >> >> A related question is how threadsafe is an instance of
>> >> >> DatastoreService?
>> >> >>
>> >> >>
>> >> >>
>> >> >
>> >> >
>> >> > >
>> >> >
>> >>
>> >>
>> >
>> >
>> > >
>> >
>>
>>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I just committed an update to this to remove the static
DatastoreService instance.

Vince

On Sat, Oct 31, 2009 at 2:08 PM, Vince Bonfanti  wrote:
> This looked like an interesting problem, and I already had most of the
> pieces in place, so here's my first attempt, which is implemented in a
> single class (also attached, along with some test files):
>
>    
> http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
>
> I'm more than happy to contribute this for inclusion in the SDK.
>
> Some caveats:
>
>   1) Because of issue #2097
> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
> doesn't work on the development server (it does work in production).
> So go star that issue!
>
>   2) I've only done minimal testing on the production server (see the
> attached test files).
>
>   3) This post is the only documentation that's currently available.
>
> First, add the following your your web.xml:
>
>    
>        Deferred
>        
> com.newatlanta.appengine.taskqueue.Deferred
>    
>    
>        Deferred
>        /_ah/queue/deferred
>    
>
> Second, define the "deferred" queue within queue.xml (use whatever
> rate you want):
>
>    
>        deferred
>        10/s
>    
>
> Next, create a class that implements the
> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
> "doTask()" method of this class is where you implement your task
> logic.
>
> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
> method to queue up your task:
>
>    TestDeferred testDeferred = new TestDeferred(); // implements Deferrable
>    Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );
>
> (Note that it would be possible to pass arguments as attributes to
> your Deferrable instance, rather than using varargs; I'm not sure it
> makes much difference which you choose).
>
> Just as for the Python implementation, if the arguments size exceeds
> 10KB, the arguments are stored in a datastore entity, which is then
> deleted when the task is executed. Also, your doTask() method can
> throw a PermanentTaskFailure exception to halt retries; any other
> exceptions cause the task to be retried.
>
> Let me know if you find this useful, have any questions or encounter
> any problems.
>
> Vince
>
> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google)  wrote:
>> Hi David. This may be coming to Java eventually, but it hasn't been started
>> yet. If you or anyone else is interested in contributing, let me know.
>>
>> - Jason
>>
>> On Wed, Oct 28, 2009 at 7:52 AM, David Chandler 
>> wrote:
>>>
>>> Re: http://code.google.com/appengine/articles/deferred.html
>>>
>>> Will this be coming to AppEngine for Java?
>>>
>>> David Chandler
>>> http://turbomanage.wordpress.com
>>>
>>>
>>
>>
>> >>
>>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Nick,

Thanks for your feedback. Yes, I'm very interested in seeing this
included in the official SDK. What are the steps from here?

See responses to specific questions inserted below.

Vince

> - Perhaps make PermanentTaskFailure a runtime exception, which would
> eliminate the need to declare it on methods that throw it.

That's fine. I generally prefer checked exceptions, but if the
AppEngine team prefers unchecked exceptions, I'm happy to make this
change.

> - If you do the above, you could make the Deferrable interface extend
> Runnable, as suggested in the meeting. I'm not sure that's appropriate,
> since the implications of Runnable are different.

I think Runnable is not appropriate. I also looked at Callable, but in
the end thought a new interface--Deferrable--was the best option.

> - I'm not sure there's any need to be able to pass arguments to the
> Deferrable, since users will already have to declare a deferrable class and
> instantiate it. I'm open to convincing, though.

I put the arguments there to mimic the Python API and then realized
later they're not really needed. Removing the arguments also
simplifies the code by eliminating the need for the TaskHolder
class--the Deferrable class could be serialized directly. I'm happy to
remove the arguments if that's what the AppEngine team decides is
best.

> - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
> Python, the actual limit seems to include some overhead that varies from
> task to task.

No, I haven't tested this yet.

> - If payload isn't an instance of TaskHolder, the task will silently return
> - it should log an error.

I think the only way payload won't be an instance of TaskHolder is if
the deserialize() method returns null, in which case an error has
already been logged. But I'll double-check this.

> Are you interested in having this included in the official SDK?
> -Nick
> On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti  wrote:
>>
>> This looked like an interesting problem, and I already had most of the
>> pieces in place, so here's my first attempt, which is implemented in a
>> single class (also attached, along with some test files):
>>
>>
>>  http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java
>>
>> I'm more than happy to contribute this for inclusion in the SDK.
>>
>> Some caveats:
>>
>>   1) Because of issue #2097
>> (http://code.google.com/p/googleappengine/issues/detail?id=2097), this
>> doesn't work on the development server (it does work in production).
>> So go star that issue!
>>
>>   2) I've only done minimal testing on the production server (see the
>> attached test files).
>>
>>   3) This post is the only documentation that's currently available.
>>
>> First, add the following your your web.xml:
>>
>>    
>>        Deferred
>>
>>  com.newatlanta.appengine.taskqueue.Deferred
>>    
>>    
>>        Deferred
>>        /_ah/queue/deferred
>>    
>>
>> Second, define the "deferred" queue within queue.xml (use whatever
>> rate you want):
>>
>>    
>>        deferred
>>        10/s
>>    
>>
>> Next, create a class that implements the
>> com.newatlanta.appengine.taskqueue.Deferred.Deferrable interface; the
>> "doTask()" method of this class is where you implement your task
>> logic.
>>
>> Then, invoke the com.newatlanta.appengine.taskqueue.Deferred.defer()
>> method to queue up your task:
>>
>>    TestDeferred testDeferred = new TestDeferred(); // implements
>> Deferrable
>>    Deferred.defer( testDeferred, "one", "two", "three", 1, 2, 3 );
>>
>> (Note that it would be possible to pass arguments as attributes to
>> your Deferrable instance, rather than using varargs; I'm not sure it
>> makes much difference which you choose).
>>
>> Just as for the Python implementation, if the arguments size exceeds
>> 10KB, the arguments are stored in a datastore entity, which is then
>> deleted when the task is executed. Also, your doTask() method can
>> throw a PermanentTaskFailure exception to halt retries; any other
>> exceptions cause the task to be retried.
>>
>> Let me know if you find this useful, have any questions or encounter
>> any problems.
>>
>> Vince
>>
>> On Fri, Oct 30, 2009 at 6:13 PM, Jason (Google) 
>> wrote:
>> > Hi David. This may be coming to Java eventually, but it hasn't been
>> > started
>

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I'm not sure about adding a TemporaryTaskFailure
(TransientTaskFailure?) exception, but regardless of whether we do or
not, it seems that the Deferrable.doTask method should be declared to
throw Exception. Otherwise, implementations will be restricted to
throwing only RuntimeException (or subclasses thereof). Think of cases
where you're using third-party libraries and you simply want
exceptions to propagate up.

My current implementation is based on the Python implementation, which
says this about task failures:

"'Failure' in the case of a deferred task is defined as your task
throwing any uncaught exception. Normally, automatic retries are what
you want - for example, if the task does datastore operations, and the
datastore threw a Timeout exception. If you want to deliberately cause
your task to fail so it will be retried, you can simply throw an
exception."

That seems reasonable: any uncaught exception causes a task retry;
normal termination or PermanentTaskFailure exception do not retry.

Vince

On Tue, Nov 10, 2009 at 1:35 AM, Roy  wrote:
>
> I've started looking at using Vince's Defer helper class.
>
> As a suggestion, could we add a TemporaryTaskFailure exception so it
> is explicit what behaviour is expected. I realise that a task retry is
> the default behaviour by propagating an exception, I just think it's
> better documentation if it's explicit.
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I've faxed over the CLA and created issue #2381 for this:

   http://code.google.com/p/googleappengine/issues/detail?id=2381

Here are the changes I've made since the original implementation:

 - The Deferrable.doTask method is now declared to throw Exception
(instead of PermanentTaskFailure). This allows implementations to
throw any subclass of Exception in order to have the task retried.
(PermanentTaskFailure remains a subclass of Exception).

 - The Deferrable.doTask method no longer takes any arguments.

 - Removed the TaskHolder nested class since it's no longer needed.

 - Allow an extra 240 bytes for task overhead; in testing the actual
overhead was 101 bytes, so this will leave some head-room. In
addition, the IllegalArgumentException that's thrown if the task size
is exceeded is caught (and logged), and the task it then written to
the datastore.

 - An exception is logged if payload is not an instance of Deferrable.

 - If the task is written to the datastore and there's a subsequent
failure to queue the task, the datastore entity is deleted.

 - Try twice for datatstore puts and deletes in case of
DatastoreTimeoutException.

 - Try twice when queuing a task in case of TransientFailureException.

 - Added comments.

Let me know if you have additional feedback.

Vince

On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
 wrote:
> Hi Vince,
>
> On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti  wrote:
>>
>> Hi Nick,
>>
>> Thanks for your feedback. Yes, I'm very interested in seeing this
>> included in the official SDK. What are the steps from here?
>
> See this doc:
> http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
>
>>
>> See responses to specific questions inserted below.
>>
>> Vince
>>
>> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> > eliminate the need to declare it on methods that throw it.
>>
>> That's fine. I generally prefer checked exceptions, but if the
>> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> change.
>
> I wouldn't say we prefer them in general - but personally, I think common
> uses of PermanentTaskFailure fit into the same sort of category as out of
> memory errors, divisions by zero, and so forth - they're not predictable at
> design-time.
>>
>> > - If you do the above, you could make the Deferrable interface extend
>> > Runnable, as suggested in the meeting. I'm not sure that's appropriate,
>> > since the implications of Runnable are different.
>>
>> I think Runnable is not appropriate. I also looked at Callable, but in
>> the end thought a new interface--Deferrable--was the best option.
>
> Fair enough.
>
>>
>> > - I'm not sure there's any need to be able to pass arguments to the
>> > Deferrable, since users will already have to declare a deferrable class
>> > and
>> > instantiate it. I'm open to convincing, though.
>>
>> I put the arguments there to mimic the Python API and then realized
>> later they're not really needed. Removing the arguments also
>> simplifies the code by eliminating the need for the TaskHolder
>> class--the Deferrable class could be serialized directly. I'm happy to
>> remove the arguments if that's what the AppEngine team decides is
>> best.
>
> Since the Java version is inevitably different due to the need for a custom
> class instead of a function object, I'd vote to drop the arguments.
>
>>
>> > - Have you tested tasks of size exactly MaxTaskSizeBytes? At least in
>> > Python, the actual limit seems to include some overhead that varies from
>> > task to task.
>>
>> No, I haven't tested this yet.
>>
>> > - If payload isn't an instance of TaskHolder, the task will silently
>> > return
>> > - it should log an error.
>>
>> I think the only way payload won't be an instance of TaskHolder is if
>> the deserialize() method returns null, in which case an error has
>> already been logged. But I'll double-check this.
>
> The other possibility is that something else serialized an object and sent
> it to your task handler - or possibly, that a future version of your own
> code did so. I think logging an error (but returning a 200 status code)
> would be the best policy, so these sorts of situations are debuggable.
>
> -Nick Johnson
>>
>> > Are you interested in having this included in the official SDK?
>> > -Nick
>> > On Sat, Oct 31, 2009 at 7:08 PM, Vince Bonfanti 
>> > wrote:
>> >>
>>

[appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Nick,

I'm taking this off-list for now.

I've visited codereview.appspot.com, but am confused. I tried creating
a new issue, but:

 - what do I specify for "SVN base"?
 - it will only let me upload a patch file (not a java source file),
but I'm not sure how to create a patch for a brand-new file?

Can you offer some help? thanks.

Vince

On Tue, Nov 10, 2009 at 5:12 PM, Nick Johnson (Google)
 wrote:
> Hi Vince,
> Thanks for doing this! Could you upload your code to codereview.appspot.com
> and link to it from the issue? We can continue the discussion there.
> -Nick
>
> On Tue, Nov 10, 2009 at 9:37 PM, Vince Bonfanti  wrote:
>>
>> I've faxed over the CLA and created issue #2381 for this:
>>
>>   http://code.google.com/p/googleappengine/issues/detail?id=2381
>>
>> Here are the changes I've made since the original implementation:
>>
>>  - The Deferrable.doTask method is now declared to throw Exception
>> (instead of PermanentTaskFailure). This allows implementations to
>> throw any subclass of Exception in order to have the task retried.
>> (PermanentTaskFailure remains a subclass of Exception).
>>
>>  - The Deferrable.doTask method no longer takes any arguments.
>>
>>  - Removed the TaskHolder nested class since it's no longer needed.
>>
>>  - Allow an extra 240 bytes for task overhead; in testing the actual
>> overhead was 101 bytes, so this will leave some head-room. In
>> addition, the IllegalArgumentException that's thrown if the task size
>> is exceeded is caught (and logged), and the task it then written to
>> the datastore.
>>
>>  - An exception is logged if payload is not an instance of Deferrable.
>>
>>  - If the task is written to the datastore and there's a subsequent
>> failure to queue the task, the datastore entity is deleted.
>>
>>  - Try twice for datatstore puts and deletes in case of
>> DatastoreTimeoutException.
>>
>>  - Try twice when queuing a task in case of TransientFailureException.
>>
>>  - Added comments.
>>
>> Let me know if you have additional feedback.
>>
>> Vince
>>
>> On Sun, Nov 8, 2009 at 3:34 PM, Nick Johnson (Google)
>>  wrote:
>> > Hi Vince,
>> >
>> > On Sun, Nov 8, 2009 at 7:58 PM, Vince Bonfanti 
>> > wrote:
>> >>
>> >> Hi Nick,
>> >>
>> >> Thanks for your feedback. Yes, I'm very interested in seeing this
>> >> included in the official SDK. What are the steps from here?
>> >
>> > See this doc:
>> >
>> > http://groups.google.com/group/google-appengine/web/how-to-submit-a-patch-to-the-sdk
>> >
>> >>
>> >> See responses to specific questions inserted below.
>> >>
>> >> Vince
>> >>
>> >> > - Perhaps make PermanentTaskFailure a runtime exception, which would
>> >> > eliminate the need to declare it on methods that throw it.
>> >>
>> >> That's fine. I generally prefer checked exceptions, but if the
>> >> AppEngine team prefers unchecked exceptions, I'm happy to make this
>> >> change.
>> >
>> > I wouldn't say we prefer them in general - but personally, I think
>> > common
>> > uses of PermanentTaskFailure fit into the same sort of category as out
>> > of
>> > memory errors, divisions by zero, and so forth - they're not predictable
>> > at
>> > design-time.
>> >>
>> >> > - If you do the above, you could make the Deferrable interface extend
>> >> > Runnable, as suggested in the meeting. I'm not sure that's
>> >> > appropriate,
>> >> > since the implications of Runnable are different.
>> >>
>> >> I think Runnable is not appropriate. I also looked at Callable, but in
>> >> the end thought a new interface--Deferrable--was the best option.
>> >
>> > Fair enough.
>> >
>> >>
>> >> > - I'm not sure there's any need to be able to pass arguments to the
>> >> > Deferrable, since users will already have to declare a deferrable
>> >> > class
>> >> > and
>> >> > instantiate it. I'm open to convincing, though.
>> >>
>> >> I put the arguments there to mimic the Python API and then realized
>> >> later they're not really needed. Removing the arguments also
>> >> simplifies the code by eliminating the need for the Task

Re: [appengine-java] Using class objects as keys in tasks for TaskQueue

[Vince Bonfanti]

You can pass an object to a task by first serializing it to a byte array,
setting the byte array as the task payload, and then deserializing it when
your task runs. The following code demonstrates how to do this:

http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

You'll
want to modify your code to do the following; note the use of the
"serialize" method from the Deferred class:


QueueFactory.getDefaultQueue().add(url("/url/to/servlet").payload(serialize(data)));

Then within your task you can deserialize the object as follows

   Object obj = deserialize( req );

You might be able to use the Deferred class directly to solve your problem.

Note that because of issue #2097, this serialization and deserialization of
objects as the task payload does not work on the development server (but
does work in production). You may want to star this issue:

  http://code.google.com/p/googleappengine/issues/detail?id=2097

Vince

On Fri, Nov 13, 2009 at 2:52 PM, edarroyo  wrote:

> Hello,
> We are trying to set an object as a parameter for a task.
> It seems the only values you can pass are strings, as we always get
> Eclipse complaining that the funciont param() does not handle non-
> string objects.
>
> What we are doing is the following:
>
>
>  QueueFactory.getDefaultQueue().add(url("/url/to/servlet").param
> ("Key", data));
>
> Where data is an object of type Test.
>
> How can we pass this object to our task and retrieve it?
>
> Thanks!
>
> --
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=.

[appengine-java] memcache memory leak on dev server

[Vince Bonfanti]

There appears to be a memory leak with the memcache implementation on the
dev server; I've opened a new issue for this:

http://code.google.com/p/googleappengine/issues/detail?id=2428

I have some testcases that make heavy use of memcache that are failing due
to this issue.

Vince

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] memcache memory leak on dev server

[Vince Bonfanti]

But in the testcase I attached to the issue report, I'm doing a put of the
same key every time--why would this take more and more memory (and never
release it)? Shouldn't the value be replaced, and the memory for the
previous value be freed?

Vince

On Tue, Nov 24, 2009 at 1:30 PM, Toby Reyelts  wrote:

> I've posted a reply to the issue, which I'm copying below:
>
> Unlike production, the dev_appserver's memcache implementation has a 100M 
> limit which
> it will happily attempt to fill, regardless of how much heap space you 
> actually have
> free in your JVM. You can either increase your heap size to accomodate the 
> limit, or
> change the limit to accomodate your heap size. You can change the limit by 
> setting
> the system property, memcache.maxsize. For example,
>
> $SDK_HOME/bin/dev_appserver.sh --jvm_flag=-Dmemcache.maxsize=50M
>
>
> On Tue, Nov 24, 2009 at 12:59 PM, Vince Bonfanti wrote:
>
>> There appears to be a memory leak with the memcache implementation on the
>> dev server; I've opened a new issue for this:
>>
>> http://code.google.com/p/googleappengine/issues/detail?id=2428
>>
>> I have some testcases that make heavy use of memcache that are failing due
>> to this issue.
>>
>> Vince
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Hi Jeff,

Thanks for the suggestions and the code. David Chandler sent me a patch to
support user-specified queue names (almost exactly the same as your
changes), and I've committed that patch to SVN. Regarding your other
changes:

  - I've probably make the  init parameter optional and have it
default to "/_ah/" rather than throw an exception from the init
method if it's missing.

  - While we're at it, the default queue name could be optionally specified
by an init parameter also.

  - I'm not sure a stacktrace from the doPost method is really going to give
you much more information, but would be open to that  change.

It looks like the Deferred class is going to be added to the official GAE/J
SDK, though I'm not sure when. Rather than make any more changes to the code
right now, I'd rather wait to see what shows up the SDK and then work
through normal Google channels to get any further modifications made. In the
mean time, I'm glad it's working for you.

Vince

On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer  wrote:

> Attached is a modified version of that class that lets you define any
> path you want for the servlet and lets you specify which queue to use
> like this: Deferred.defer(task, "queueName");
>
> (I needed this for my own purposes)
>
> Do with it as you wish :-)
>
> The only other major change I would make is to stop masking all the
> exceptions during the task processing.  Or at least, if you're going
> to log the exception and stop propagation, log the whole thing so we
> get a stacktrace in the logs.
>
> Jeff
>
> On Fri, Nov 20, 2009 at 1:03 PM, David Chandler 
> wrote:
> > Vince, this is great! I hadn't been watching my own issue, so didn't
> > see this until now. Thanks so much!
> >
> > Only enhancement I would suggest is to enable multiple deferred
> > queues. For example. I would like to defer tasks in an email throttle
> > queue separately from a general background queue. Perhaps the
> > Deferrable interface could have a getQueueName() method, or
> > Deferred.defer could have an additional signature defer(Deferrable
> > task, String queueName).
> >
> > Thanks again,
> > /dmc
>
> --
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: How to return a file from a servlet

[Vince Bonfanti]

Or, something similar, using the IOUtils class from Commons I/O (
http://commons.apache.org/io/):

 InputStream resourceAsStream =
getServletContext().getResourceAsStream.(pathToImage);
 resp.setContentType("image/gif");
 IOUtils.copy(resourceAsStream,resp.getOutputStream);

Vince

On Mon, Nov 30, 2009 at 8:31 AM, bysse  wrote:

> This should get you going:
>
>  InputStream resourceAsStream = getServletContext().getResourceAsStream
> (pathToImage);
>  // use stream to read image data
>  ...
>  resp.setContentType("image/gif");
>  resp.setContentLength(imageData.length);
>  outputStream.write(imageData, 0, imageData.length);
>
> /Erik
>
> On Nov 30, 11:23 am, Prashant  wrote:
> > Hi,
> >
> > I have a servlet with request handler */file/** . I want to return a file
> *
> > /theme/bg.gif* for all */file/*.gif *. I can check for .gif extension
> then
> > how do i send */theme/bg.gif*, for */file/*.gif* without sending a
> redirect
> > ?
> >
> > Thanks.
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: How to return a file from a servlet

[Vince Bonfanti]

I'm not absolutely sure it's required, but I think in practice you should
always specify the content type. Note that you can get most common types via
a lookup based on the file extension:

   String contentType = getServletContext().getMimeType( imagePath );

You can add additional mime types to web.xml and the above code will use
them, too (in addition to the common ones).

Vince

On Mon, Nov 30, 2009 at 12:25 PM, Prashant  wrote:

> thanks a lot guys.
>
>
> is it necessary to use resp.setContentType("image/gif"); even if url end
> with ".gif" ?
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] task queue once-only semantics

[Vince Bonfanti]

I'm trying to solve the following problem using task queues:

  1. Queue a task to perform some action "N".
  2. Until action N has been performed, don't allow any other tasks to be
queued that also perform action N.
  3. As soon as action N has been performed, immediately allow other tasks
to be queued to perform action N (repeat from step 1).

Using task names almost accomplishes what I need, except that at step 3 it
will be at least seven days before another task can be queued to perform
action N, when I need it to be allowed immediately:

  "This (task names) provides a lightweight mechanism for ensuring once-only
semantics. Once a Task with name N is written, any subsequent attempts to
insert a Task named N will fail. Eventually (at least seven days after the
task successfully executes), the task will be deleted and the name N can be
reused."

I've thought of different ways to solve this, most likely by using a
memcache flag as a semaphore. But, it would be nice if the task queue API
could support this natively, maybe by adding a "reuse time" or "exclusion
time" parameter when setting the TaskOptions name parameter.

Any thoughts on this?

Vince

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: task queue once-only semantics

[Vince Bonfanti]

BTW, I've noticed that the dev server actually behaves the way I would like
it to. It only throws a "Task name already exists" exception while the task
is queued; as soon as the task runs it allows you to queue up another task
with the same name.

I guess I'll just open a feature request on this in the issue tracker.

Vince

On Tue, Dec 1, 2009 at 11:39 AM, Vince Bonfanti  wrote:

> I'm trying to solve the following problem using task queues:
>
>   1. Queue a task to perform some action "N".
>   2. Until action N has been performed, don't allow any other tasks to be
> queued that also perform action N.
>   3. As soon as action N has been performed, immediately allow other tasks
> to be queued to perform action N (repeat from step 1).
>
> Using task names almost accomplishes what I need, except that at step 3 it
> will be at least seven days before another task can be queued to perform
> action N, when I need it to be allowed immediately:
>
>   "This (task names) provides a lightweight mechanism for ensuring
> once-only semantics. Once a Task with name N is written, any subsequent
> attempts to insert a Task named N will fail. Eventually (at least seven days
> after the task successfully executes), the task will be deleted and the name
> N can be reused."
>
> I've thought of different ways to solve this, most likely by using a
> memcache flag as a semaphore. But, it would be nice if the task queue API
> could support this natively, maybe by adding a "reuse time" or "exclusion
> time" parameter when setting the TaskOptions name parameter.
>
> Any thoughts on this?
>
> Vince
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: task queue once-only semantics

[Vince Bonfanti]

I'm not sure I follow the first part of your response--are you mixing up
"queue names" and "task names"?

But, yes, implementing a mutex via memcache works (I've already tested it),
but there are two things I don't like about this approach:

  1) There's additional overhead of using memcache (about 15ms per memcache
call, though I've measured up to 90ms).
  2) It makes my code more complex than it needs to be, considering that the
task queue API already supports almost everything I need.

Vince

On Tue, Dec 1, 2009 at 5:08 PM, James H  wrote:

> Vince, shouldn't you use an un-named Task queue (to avoid Naming rules
> mentioned) but query your own datastore semaphore prior to adding to
> the queue.  This way when task N runs it manages the semaphore (a
> Entity) to implement a mutex begin/end sequence.  I believe this
> implies that task N will in fact run 3 transactions: #1 - mutex begin
> (create Entity), #2 - work for N, and #3 - mutex end (remove Entity).
>
> Anyhow assuming your logic to trigger a new Task N can incorporate the
> criteria for mutex exists, then what would keep this from working?
>
> On Dec 1, 2:37 pm, Vince Bonfanti  wrote:
> > BTW, I've noticed that the dev server actually behaves the way I would
> like
> > it to. It only throws a "Task name already exists" exception while the
> task
> > is queued; as soon as the task runs it allows you to queue up another
> task
> > with the same name.
> >
> > I guess I'll just open a feature request on this in the issue tracker.
> >
> > Vince
> >
> >
> >
> > On Tue, Dec 1, 2009 at 11:39 AM, Vince Bonfanti 
> wrote:
> > > I'm trying to solve the following problem using task queues:
> >
> > >   1. Queue a task to perform some action "N".
> > >   2. Until action N has been performed, don't allow any other tasks to
> be
> > > queued that also perform action N.
> > >   3. As soon as action N has been performed, immediately allow other
> tasks
> > > to be queued to perform action N (repeat from step 1).
> >
> > > Using task names almost accomplishes what I need, except that at step 3
> it
> > > will be at least seven days before another task can be queued to
> perform
> > > action N, when I need it to be allowed immediately:
> >
> > >   "This (task names) provides a lightweight mechanism for ensuring
> > > once-only semantics. Once a Task with name N is written, any subsequent
> > > attempts to insert a Task named N will fail. Eventually (at least seven
> days
> > > after the task successfully executes), the task will be deleted and the
> name
> > > N can be reused."
> >
> > > I've thought of different ways to solve this, most likely by using a
> > > memcache flag as a semaphore. But, it would be nice if the task queue
> API
> > > could support this natively, maybe by adding a "reuse time" or
> "exclusion
> > > time" parameter when setting the TaskOptions name parameter.
> >
> > > Any thoughts on this?
> >
> > > Vince- Hide quoted text -
> >
> > - Show quoted text -
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: task queue once-only semantics

[Vince Bonfanti]

Yes, obviously I can't use task names because of the 7-day expiration; I
just happened to be testing something else on the dev server when I noticed
that it behaved the way I wanted.

In my case the unreliability of memcache isn't too much of a concern for two
reasons. First, I expect the tasks be executed relatively quickly--within a
few hundred milliseconds--and hopefully memcache can be relied upon for at
least that long. Second, there's no real harm if my tasks get executed more
than once.

I just opened a new issue on this, if you want to go star it:

  http://code.google.com/p/googleappengine/issues/detail?id=2459

Vince

On Tue, Dec 1, 2009 at 5:24 PM, James H  wrote:

> Sorry, I was writing my 2nd reply prior to seeing yours...but I mean't
> don't specify a Task Name in order to free yourself from the
> rediculous 7-day rule (what good is that?).  So really since memcache
> is not a good choice for sensitive mutable data such as a mutex, you'd
> have to implement your own as I suggested.  I'll be curious to hear
> other alternatives since this would be a common requirement in apps!
>
> On Dec 1, 4:17 pm, James H  wrote:
> > Also, the nature of memcache is such that it would not be reliable for
> > a mutex (since objects can be removed by the system at any time
> > regardless if Task N is running), hence the potential for duplicate
> > task entries.
> >
> > On Dec 1, 4:08 pm, James H  wrote:
> >
> >
> >
> > > Vince, shouldn't you use an un-named Task queue (to avoid Naming rules
> > > mentioned) but query your own datastore semaphore prior to adding to
> > > the queue.  This way when task N runs it manages the semaphore (a
> > > Entity) to implement a mutex begin/end sequence.  I believe this
> > > implies that task N will in fact run 3 transactions: #1 - mutex begin
> > > (create Entity), #2 - work for N, and #3 - mutex end (remove Entity).
> >
> > > Anyhow assuming your logic to trigger a new Task N can incorporate the
> > > criteria for mutex exists, then what would keep this from working?
> >
> > > On Dec 1, 2:37 pm, Vince Bonfanti  wrote:
> >
> > > > BTW, I've noticed that the dev server actually behaves the way I
> would like
> > > > it to. It only throws a "Task name already exists" exception while
> the task
> > > > is queued; as soon as the task runs it allows you to queue up another
> task
> > > > with the same name.
> >
> > > > I guess I'll just open a feature request on this in the issue
> tracker.
> >
> > > > Vince
> >
> > > > On Tue, Dec 1, 2009 at 11:39 AM, Vince Bonfanti 
> wrote:
> > > > > I'm trying to solve the following problem using task queues:
> >
> > > > >   1. Queue a task to perform some action "N".
> > > > >   2. Until action N has been performed, don't allow any other tasks
> to be
> > > > > queued that also perform action N.
> > > > >   3. As soon as action N has been performed, immediately allow
> other tasks
> > > > > to be queued to perform action N (repeat from step 1).
> >
> > > > > Using task names almost accomplishes what I need, except that at
> step 3 it
> > > > > will be at least seven days before another task can be queued to
> perform
> > > > > action N, when I need it to be allowed immediately:
> >
> > > > >   "This (task names) provides a lightweight mechanism for ensuring
> > > > > once-only semantics. Once a Task with name N is written, any
> subsequent
> > > > > attempts to insert a Task named N will fail. Eventually (at least
> seven days
> > > > > after the task successfully executes), the task will be deleted and
> the name
> > > > > N can be reused."
> >
> > > > > I've thought of different ways to solve this, most likely by using
> a
> > > > > memcache flag as a semaphore. But, it would be nice if the task
> queue API
> > > > > could support this natively, maybe by adding a "reuse time" or
> "exclusion
> > > > > time" parameter when setting the TaskOptions name parameter.
> >
> > > > > Any thoughts on this?
> >
> > > > > Vince- Hide quoted text -
> >
> > > > - Show quoted text -- Hide quoted text -
> >
> > > - Show quoted text -- Hide quoted text -
> >
> > - Show quoted text -
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: task queue once-only semantics

[Vince Bonfanti]

Yes, I received your patch and it's on my TODO list to review it and add it
to the code. Unfortunately, I've been a bit backed up with other projects.
I'm trying to get a new version of GaeVFS ready for release in the next few
weeks and will try to include your patch.

Vince

On Wed, Dec 2, 2009 at 12:08 AM, James H  wrote:

> Ah, short execution time and bullet-proof re-runs...I see how memcache
> would work in this case.  On another topic, I recall you wrote
> GaeVFS...I recently added a comment related to a patch for checking
> file timestamp to avoid redundant downloads.  On the surface it
> appears straight forward...if the http header was supplied then just
> compare with the Entity's timestamp and you're done.  Is it really
> that simple?  Are you or someone else familiar with the code
> considering adding this check?
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: AppEngine needs an AppStore

[Vince Bonfanti]

What we really, really need is something similar to Amazon DevPay:

http://aws.amazon.com/devpay/

Vince

On Tue, Dec 1, 2009 at 2:08 PM, Ikai L (Google)  wrote:

> I see what you mean. Say, an application marketplace where folks can
> download source code for applications and deploy to their own App Engine
> instances, get support, download updates, and so forth. It's an interesting
> idea. We're focusing on Apps for Google Apps for the Enterprise in a SaaS
> model rather than an a la carte software download model.
>
> Cool idea: a marketplace like what you describe could be an App Engine app
> itself ...
>
>
> On Tue, Dec 1, 2009 at 7:16 AM, Prashant  wrote:
>
>> +1
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine for Java" group.
>> To post to this group, send email to
>> google-appengine-j...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine-java+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine-java?hl=en.
>>
>
>
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
>
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

I just committed the following changes:

  - Added additional overloads to the defer() method to allow optional
TaskOptions to be specified. This lets you set the following task options:
url, countdownMIllis, etaMillis, taskName.

  - Added support for optional "queueName" and "taskUrl" init parameters.

  - The defer() method and its overloads no longer throw IOException;
instead, an unchecked QueueFailureException is thrown if there's a failure
to serialize the task object (Deferrable instance).

  - The defer() method and its overloads now return a TaskHandle.

  - A stacktrace is now logged upon failure to deserialize the task.

  - Updated javadoc comments.

For convenience, here's the direct link to the source:

http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

As always, looking forward to comments and feedback.

Vince

On Thu, Nov 26, 2009 at 5:39 AM, Nick Johnson (Google) <
nick.john...@google.com> wrote:

> Hi Vince,
>
> I haven't had a chance to integrate your code yet, so feel free to continue
> making modifications. In particular, I think it would be useful to provide
> the same set of options the Python deferred API accepts, which include 'eta'
> and 'countdown' parameters, as well as specifying the queue and URL.
>
> -Nick
>
> On Wed, Nov 25, 2009 at 7:03 PM, Vince Bonfanti wrote:
>
>> Hi Jeff,
>>
>> Thanks for the suggestions and the code. David Chandler sent me a patch to
>> support user-specified queue names (almost exactly the same as your
>> changes), and I've committed that patch to SVN. Regarding your other
>> changes:
>>
>>   - I've probably make the  init parameter optional and have
>> it default to "/_ah/" rather than throw an exception from the
>> init method if it's missing.
>>
>>   - While we're at it, the default queue name could be optionally
>> specified by an init parameter also.
>>
>>   - I'm not sure a stacktrace from the doPost method is really going to
>> give you much more information, but would be open to that  change.
>>
>> It looks like the Deferred class is going to be added to the official
>> GAE/J SDK, though I'm not sure when. Rather than make any more changes to
>> the code right now, I'd rather wait to see what shows up the SDK and then
>> work through normal Google channels to get any further modifications made.
>> In the mean time, I'm glad it's working for you.
>>
>> Vince
>>
>> On Tue, Nov 24, 2009 at 8:09 PM, Jeff Schnitzer wrote:
>>
>>> Attached is a modified version of that class that lets you define any
>>> path you want for the servlet and lets you specify which queue to use
>>> like this: Deferred.defer(task, "queueName");
>>>
>>> (I needed this for my own purposes)
>>>
>>> Do with it as you wish :-)
>>>
>>> The only other major change I would make is to stop masking all the
>>> exceptions during the task processing.  Or at least, if you're going
>>> to log the exception and stop propagation, log the whole thing so we
>>> get a stacktrace in the logs.
>>>
>>> Jeff
>>>
>>>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

The behavior you're reporting is exactly the opposite of what I'm seeing.
For me, deserialization always fails on the dev server and always works in
production. What I'm seeing on the dev server is that the bytes retrieved
from the task payload (req.getInputStream) are not the same bytes that are
sent; the result is the StreamCorruptedException. Again, though, I only see
this on the dev server and never in production. Can you send me an example
that fails in production? I'd like to understand what's going on before
applying the Base64 encoder workaround.

I'm not at all familiar with Guice, but I think I understand what you're
trying to do. One thought I've had is that we could pass the
HttpServletRequest object to the doTask() method. This would allow you to
add attributes (objects) to the HttpServletRequest within your Guice filter
and then retrieve them within your doTask() method.

Vince

On Fri, Nov 27, 2009 at 7:53 PM, David Chandler wrote:

> I've solved the serialization problems by Base64 encoding the
> serialized task before calling payload() and decoding it in the
> deserialize(HttpServletRequest) method. I'm guessing something in the
> task queue chain (either task queue payload storage or the servlet
> call when task is run) has problems transmitting the binary data, as
> the exception I was getting at one point was a
> java.io.StreamCorruptedException.
>
> I'll send another patch to Vince. My solution works, but I'm not
> completely satisfied that I've diagnosed the problem correctly. I used
> Apache commons-codec for Base64. Is it safe to use com.sun.misc
> instead? I see it in appengine.repackaged chain, as well, but don't
> see it as part of an official Google API.
>
> Somewhat related, I've wired up the Deferred servlet through the
> GuiceFilter so I can pass a Guice injector to Deferrable tasks. Since
> the task objects are not created through Guice, but rather by
> deserializing, I modified doTask() to accept an additional Injector
> argument. We probably don't want that in the official version of
> Deferrable, though.
>
> Is there a better way for my Deferrable classes to get access to a
> Guice "context"? I'm trying to avoid Guice.createInjector(), as it
> results in a duplicate PersistenceManagerFactory being created since
> other servlets are also being served through Guice.
>
> Thank you,
> /dmc
>
> On Nov 27, 5:42 pm, David Chandler  wrote:
> > Jeff,
> >
> > I'm seeing problems with deserialization, too, when deployed on
> > AppEngine. In dev, I can deserialize(serialize(task)) and it works
> > just fine, but not so in AppEngine. I get the same error whether the
> > task payload is the serialized Deferrable task itself or just the Key
> > with the task in the db, but I haven't yet figured out which is the
> > cause of the problem.
> >
> > The error I'm getting on AppEngine is invalid type code: 00 in
> > Deferred.deserialize method. I've tried Base64 encoding after
> > serialization in case it's related to special binary escape sequences
> > that happen to be in the serialized object stream but still no
> > success.
> >
> > I'm pretty sure AppEngine has written some custom code to do
> > serialization because I previously got an AppEngine acess control
> > exception when calling enableResolveObject(true) in a subclass of
> > ObjectInputStream, and I'm wondering if that's in play somehow...
> >
> > /dmc
> >
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

[Vince Bonfanti]

I just started testing with the 1.2.8 prerelease, and I'm getting the
following exception from Queue.add() in code that works in 1.2.6:

java.lang.IllegalStateException: Current enviornment must have the server
url available via the com.google.appengine.server_url_key attribute.

This only happens when Queue.add() is invoked from the servlet init() method
or from a static initializer; if Queue.add() is invoked from a "regular"
request thread, then it works properly.

Vince

On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google)  wrote:

> Hello App Engine Developers,
>
> As part of our ongoing efforts to improve release quality and
> transparency, we will start prereleasing SDKs for early testing. We
> hope this gives developers a chance to participate in our release
> process by trying out new changes and sending feedback. As of this
> morning, the prerelease SDK for our next release, 1.2.8, is available
> in the familiar download location (note that the filename ends in
> 'prerelease.zip'):
>
> http://code.google.com/p/googleappengine/downloads/list
>
> If you're interested, please download and give it a try locally with
> your favorite App Engine code. Please note that, as a prerelease, this
> SDK is not yet supported and still subject to change. Thus, please
> don't take critical dependencies or make substantial changes to
> production apps based on this SDK.
>
> Importantly, this prerelease is purely for the SDK and is intended for
> local testing and development in dev_appserver. The server-side of App
> Engine (our production environment) is not at 1.2.8, so deploying with
> this SDK is not yet supported. In the future, we might enable a
> complete SDK and server test environment for prereleases.
>
> A few notes on 1.2.8 in particular - this release is primarily for
> servicing and updates in preparation for some exciting feature
> launches we have in the pipeline. The current release notes (still
> subject to change) are included below; these release notes do include
> changes which will only be available on the server side Admin Console
> (non-local) once 1.2.8 is formally released.
>
> Please try 1.2.8 for local development and send us your feedback!
>
> Thanks,
>
> App Engine Team
>
> Version 1.2.8
> =
>  - Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
> production
>servers). If you're using JDK 1.5 with your local dev_appserver,
> you will
>need to include the JAXB libraries with your application to use
> it.
>  http://code.google.com/p/googleappengine/issues/detail?id=1267
>  - Added Quota API (com.google.appengine.api.quota) to match Python
> API.
>  - Low-level Memcache API now supports grabTail() and batchIncrement
> ().
>  - HTTPResponse object now has getFinalUrl() method for 302
> redirects.
>  http://code.google.com/p/googleappengine/issues/detail?id=1464
>  - Java Dev Appserver now automatically executes tasks.  If you
> prefer the old
>behavior where tasks do not automatically execute you can use the
>  -Dtask_queue.disable_auto_task_execution flag when starting the
> server.
>  - Additional file extensions permitted when sending mail.
>  http://code.google.com/p/googleappengine/issues/detail?id=494
>  - Fixed issue with Java mail handler not processing multipart
> messages
>correctly.
>  - Fixed agent code included in appengine-local-runtime.jar results
> in
>RuntimeException.
>  http://code.google.com/p/googleappengine/issues/detail?id=2280
>  - Fixed issue with sort orders defined on properties that allow
> multiple
>values.
>  http://code.google.com/p/googleappengine/issues/detail?id=2349
>  - Fixed problem with dropped query strings after requiring log-in.
>  http://code.google.com/p/googleappengine/issues/detail?id=2225
>  - Removed limitation preventing multiple parameters with the same
> name.
>  http://code.google.com/p/googleappengine/issues/detail?id=2090
>  - Fixed issue with local datastore incorrectly sorting results of
> ancestor queries.
>  http://code.google.com/p/googleappengine/issues/detail?id=2177
>  - New Index building status page in the Admin Console
>  - Task Queue now supports purging queues, and deleting tasks and
> queues via
>the Admin Console.
>  http://code.google.com/p/googleappengine/issues/detail?id=2159
>  http://code.google.com/p/googleappengine/issues/detail?id=1740
>  - Over Quota HTTP status code changed from 403 to 503, other to 500.
>  - Task Queue now considers all HTTP 2xx status codes to represent
> success.
>
> ORM Changes
>
>  - Explicitly disallow multiple relationships of the same type
>  http://code.google.com/p/datanucleus-appengine/issues/detail?id=154
>  - Occasional ArrayOutOfBoundsIndexException
>  http://code.google.com/p/datanucleus-appengine/issues/detail?id=156
>  - Support inheritance
>  http://code.google.com/p/datanucleus-appengine/issues/detail?id=25
>  - Support != queries
>  - Support IN queries
>  http://code.g

Re: [appengine-java] Why no GAE system property?

[Vince Bonfanti]

I agree. My workaround was to create a ServletEventListener and put this in
the contextInitialized() method:

System.setProperty( "appengine.server",
event.getServletContext().getServerInfo().contains( "Development" ) ?
"development" : "production" );

Vince

On Thu, Dec 3, 2009 at 1:15 PM, Jeff Schnitzer  wrote:

> Why isn't there a system property that can be used to detect if a
> program is running on appengine?
>
> This is really a glaring oversight.  I would expect something like
> System.getProperty("google.appengine") to be "development" or
> "production" (or of course null).
>
> Before someone says "check the servlet context", realize that this is
> grossly inadequate - library writers do not have access to the servlet
> context.
>
> I'm trying to get the Hessian RPC library working on appengine so we
> have an easy-to-use web services protocol.  One of the first things
> the hessian java client does is teardown any keepalive connections by
> making a quick request expired at 10ms.  This terminates with a caught
> IOException in the normal world, but produces an
> ApiDeadlineExceededException in appengine.
>
> I'd like to simply short-circuit this unnecessary behavior when
> running on appengine.  I found the SecurityManager check but that
> seems unreliable.  Why isn't there a nice easy system property?
>
> Thanks,
> Jeff
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

[Vince Bonfanti]

That fixed it. Thanks.

Vince

On Thu, Dec 3, 2009 at 2:24 PM, Max Ross (Google)

> wrote:

> Vince,
>
> As a workaround, try setting the missing environment attribute explicitly
> before you add the task to the queue:
>
> ApiProxy.getCurrentEnvironment().getAttributes().put("com.google.appengine.server_url_key",
> "http://localhost:8080";);
>
> This should only be necessary for tasks that are added when there is no
> "live" request and it should have no impact in prod.
>
> Sorry for the trouble,
> Max
>
>
> On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google) <
> maxr+appeng...@google.com > wrote:
>
>> Hi Vince,
>>
>> That sounds like a bug in our code that was a side effect of the automatic
>> task execution that now happens in the dev environment.  Let me see if I can
>> come up with a workaround for you.
>>
>> Max
>>
>> On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti wrote:
>>
>>> I just started testing with the 1.2.8 prerelease, and I'm getting the
>>> following exception from Queue.add() in code that works in 1.2.6:
>>>
>>> java.lang.IllegalStateException: Current enviornment must have the server
>>> url available via the com.google.appengine.server_url_key attribute.
>>>
>>> This only happens when Queue.add() is invoked from the servlet init()
>>> method or from a static initializer; if Queue.add() is invoked from a
>>> "regular" request thread, then it works properly.
>>>
>>> Vince
>>>
>>>
>>> On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) wrote:
>>>
>>>> Hello App Engine Developers,
>>>>
>>>> As part of our ongoing efforts to improve release quality and
>>>> transparency, we will start prereleasing SDKs for early testing. We
>>>> hope this gives developers a chance to participate in our release
>>>> process by trying out new changes and sending feedback. As of this
>>>> morning, the prerelease SDK for our next release, 1.2.8, is available
>>>> in the familiar download location (note that the filename ends in
>>>> 'prerelease.zip'):
>>>>
>>>> http://code.google.com/p/googleappengine/downloads/list
>>>>
>>>> If you're interested, please download and give it a try locally with
>>>> your favorite App Engine code. Please note that, as a prerelease, this
>>>> SDK is not yet supported and still subject to change. Thus, please
>>>> don't take critical dependencies or make substantial changes to
>>>> production apps based on this SDK.
>>>>
>>>> Importantly, this prerelease is purely for the SDK and is intended for
>>>> local testing and development in dev_appserver. The server-side of App
>>>> Engine (our production environment) is not at 1.2.8, so deploying with
>>>> this SDK is not yet supported. In the future, we might enable a
>>>> complete SDK and server test environment for prereleases.
>>>>
>>>> A few notes on 1.2.8 in particular - this release is primarily for
>>>> servicing and updates in preparation for some exciting feature
>>>> launches we have in the pipeline. The current release notes (still
>>>> subject to change) are included below; these release notes do include
>>>> changes which will only be available on the server side Admin Console
>>>> (non-local) once 1.2.8 is formally released.
>>>>
>>>> Please try 1.2.8 for local development and send us your feedback!
>>>>
>>>> Thanks,
>>>>
>>>> App Engine Team
>>>>
>>>> Version 1.2.8
>>>> =
>>>>  - Support for JAXB. JAXB is included in JDK 1.6 (and App Engine's
>>>> production
>>>>servers). If you're using JDK 1.5 with your local dev_appserver,
>>>> you will
>>>>need to include the JAXB libraries with your application to use
>>>> it.
>>>>  http://code.google.com/p/googleappengine/issues/detail?id=1267
>>>>  - Added Quota API (com.google.appengine.api.quota) to match Python
>>>> API.
>>>>  - Low-level Memcache API now supports grabTail() and batchIncrement
>>>> ().
>>>>  - HTTPResponse object now has getFinalUrl() method for 302
>>>> redirects.
>>>>  http://code.google.com/p/googleappengine/issues/detail?id=1464
>>>>  - Java Dev Appserver now automatical

Re: [appengine-java] Re: Photo and Video services

[Vince Bonfanti]

I just committed the code to add support for the If-Modified-Since request
header (and the Last-Modified and Content-Length response headers). It'll be
in the next GaeVFS release, which will happen after SDK 1.2.8 is released to
production.

BTW, the way this works is that the client simply takes the value from the
Last-Modified response header and sends it back as the If-Modified-Since
request header, so there's no need to worry about synchronization between
client time and server time. I tested on Chrome, IE8, and Firefox (all on
Windows).

Thanks for the info.

Vince

On Fri, Nov 27, 2009 at 12:56 PM, James H wrote:

> Here's some legwork...you can drill-down to section 14.25 of the page
> below where it describes the If-Modified-Since header.  Basically, any
> Get with this header including a timestamp can be compared to the
> timestamp for the matching Entity to determine whether to respond
> normally (return the image) or respond with a 304 (Not modified)
> response.  I think the trick is determining the compare granualarity
> since the server time and the client time are never exactly in sync.
> Perhaps the granularity for comparison should be down to the minute,
> not seconds and definitely not millisecond.  What do you think?
>
> http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
>
> On Oct 23, 10:08 am, Vince Bonfanti  wrote:
> > No, not yet. It's on the TODO list (see line 171 of the GaeVfsServlet
> > source code). If you--or anyone else--wants to add this I'll be happy
> > to accept a patch.
> >
> > Vince
> >
> >
> >
> > On Fri, Oct 23, 2009 at 11:42 AM, Diana Cruise 
> wrote:
> >
> > > In terms of performance does gaevfs accmmodate http resource timestamp
> > > checks to avoid re-downloading repeated requests for the same images?-
> Hide quoted text -
> >
> > - Show quoted text -
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] App Engine SDK 1.2.8 released including new Admin Console features

[Vince Bonfanti]

Is SDK 1.2.8 going to be added to the Eclipse update site?

Vince

On Thu, Dec 3, 2009 at 6:06 PM, App Engine Team  wrote:

> The App Engine team has been hard at work tackling our the issues on
> our tracker, tweaking APIs and closing bugs. In addition to a ton of
> bug fixes, 1.2.8 also includes:
>
> Enhanced Admin Console - Users will notice new tools for managing
> tasks and queues created with the Task Queue API, and more visibility
> into index processing.
>
> Improved Java Compatibility - This release adds support for new filter
> operators and inheritance to JPA and JDO as well as support for JAXB,
> the single most requested feature for the Java SDK.
>
> This was also the first release we "previewed" with developers before
> formally rolling out changes. Thanks very much to all the developers
> that gave us feedback on the preview release.
>
> 1.2.8 is now available for both Python and Java developers. Take a
> look at our release notes for details on all the changes included in
> this release.
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

[Vince Bonfanti]

Hi Max,

Yes, it works. Thanks very much.

Vince

P.S. While step-debugging a task, I noticed that they seem to time-out very
quickly--within about 5 seconds. Is this intentional? Here's a partial
stacktrace:

Dec 4, 2009 3:03:58 PM com.google.appengine.api.labs.taskqueue.dev.DevQueue
runTask
SEVERE: Exception executing task 24b5a6e4-c327-4bc6-bc4f-51f838f3d1a5 on
queue write-behind-task
com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError:
5: http method GET against URL
http://127.0.0.1:8080/_ah/queue/write-behind-task?watchdog=839445f1-2c77-4c2c-bf32-81b88d5e092etimed
out.
at
com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch(LocalURLFetchService.java:214)
at
com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:53)
at
com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:46)
at
com.google.appengine.api.labs.taskqueue.dev.DevQueue.runTask(DevQueue.java:221)
at
com.google.appengine.api.labs.taskqueue.dev.LocalTaskQueue.runTask(LocalTaskQueue.java:313)

On Thu, Dec 3, 2009 at 7:01 PM, Max Ross (Google)

> wrote:

> Vince,
>
> We added a last minute fix for this in the SDK that just went out.  Could
> you please try your code with the official SDK download?
>
> Thanks,
> Max
>
>
> On Thu, Dec 3, 2009 at 12:15 PM, Max Ross (Google) <
> maxr+appeng...@google.com > wrote:
>
>> Sure, sorry for the trouble, and thanks for trying out the prerelease!
>>
>> Max
>>
>>
>> On Thu, Dec 3, 2009 at 12:12 PM, Vince Bonfanti wrote:
>>
>>> That fixed it. Thanks.
>>>
>>> Vince
>>>
>>>
>>> On Thu, Dec 3, 2009 at 2:24 PM, Max Ross (Google) <
>>> maxr+appeng...@google.com > wrote:
>>>
>>>> Vince,
>>>>
>>>> As a workaround, try setting the missing environment attribute
>>>> explicitly before you add the task to the queue:
>>>>
>>>> ApiProxy.getCurrentEnvironment().getAttributes().put("com.google.appengine.server_url_key",
>>>> "http://localhost:8080";);
>>>>
>>>> This should only be necessary for tasks that are added when there is no
>>>> "live" request and it should have no impact in prod.
>>>>
>>>> Sorry for the trouble,
>>>> Max
>>>>
>>>>
>>>> On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google) <
>>>> maxr+appeng...@google.com > wrote:
>>>>
>>>>> Hi Vince,
>>>>>
>>>>> That sounds like a bug in our code that was a side effect of the
>>>>> automatic task execution that now happens in the dev environment.  Let me
>>>>> see if I can come up with a workaround for you.
>>>>>
>>>>> Max
>>>>>
>>>>> On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti 
>>>>> wrote:
>>>>>
>>>>>> I just started testing with the 1.2.8 prerelease, and I'm getting the
>>>>>> following exception from Queue.add() in code that works in 1.2.6:
>>>>>>
>>>>>> java.lang.IllegalStateException: Current enviornment must have the
>>>>>> server url available via the com.google.appengine.server_url_key 
>>>>>> attribute.
>>>>>>
>>>>>> This only happens when Queue.add() is invoked from the servlet init()
>>>>>> method or from a static initializer; if Queue.add() is invoked from a
>>>>>> "regular" request thread, then it works properly.
>>>>>>
>>>>>> Vince
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 24, 2009 at 9:00 PM, Ikai L (Google) 
>>>>>> wrote:
>>>>>>
>>>>>>> Hello App Engine Developers,
>>>>>>>
>>>>>>> As part of our ongoing efforts to improve release quality and
>>>>>>> transparency, we will start prereleasing SDKs for early testing. We
>>>>>>> hope this gives developers a chance to participate in our release
>>>>>>> process by trying out new changes and sending feedback. As of this
>>>>>>> morning, the prerelease SDK for our next release, 1.2.8, is available
>>>>>>> in the familiar download location (note that the filename ends in
>>>>>>> 'prerelease.zip'):
>>>>>>>
>>>>>>> http://code.google.com/p/googleappengine/downloads/list
>>>>>>>
>&g

Re: [appengine-java] 1.2.8 SDK Prerelease - help us verify!

[Vince Bonfanti]

That's correct--the task still completes successfully even after the timeout
exception shows up in the console window.

Vince

On Fri, Dec 4, 2009 at 5:11 PM, Max Ross (Google)

> wrote:

> We're using the UrlFetchService to invoke the task URL, and the local
> implementation of UrlFetchService times out after 5 seconds.  For task
> execution the timeout should instead be 30 seconds.  I'll get a fix
> together, but in practice the timeout shouldn't actually impact the
> completion of the task.  UrlFetchService will timeout but the task should
> still finish running.
>
> Thanks,
> Max
>
>
> On Fri, Dec 4, 2009 at 7:06 AM, Vince Bonfanti wrote:
>
>> Hi Max,
>>
>> Yes, it works. Thanks very much.
>>
>> Vince
>>
>> P.S. While step-debugging a task, I noticed that they seem to time-out
>> very quickly--within about 5 seconds. Is this intentional? Here's a partial
>> stacktrace:
>>
>> Dec 4, 2009 3:03:58 PM
>> com.google.appengine.api.labs.taskqueue.dev.DevQueue runTask
>> SEVERE: Exception executing task 24b5a6e4-c327-4bc6-bc4f-51f838f3d1a5 on
>> queue write-behind-task
>> com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError:
>> 5: http method GET against URL
>> http://127.0.0.1:8080/_ah/queue/write-behind-task?watchdog=839445f1-2c77-4c2c-bf32-81b88d5e092etimed
>>  out.
>>  at
>> com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch(LocalURLFetchService.java:214)
>> at
>> com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:53)
>>  at
>> com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:46)
>> at
>> com.google.appengine.api.labs.taskqueue.dev.DevQueue.runTask(DevQueue.java:221)
>>  at
>> com.google.appengine.api.labs.taskqueue.dev.LocalTaskQueue.runTask(LocalTaskQueue.java:313)
>>
>> On Thu, Dec 3, 2009 at 7:01 PM, Max Ross (Google) <
>> maxr+appeng...@google.com > wrote:
>>
>>> Vince,
>>>
>>> We added a last minute fix for this in the SDK that just went out.  Could
>>> you please try your code with the official SDK download?
>>>
>>> Thanks,
>>> Max
>>>
>>>
>>> On Thu, Dec 3, 2009 at 12:15 PM, Max Ross (Google) <
>>> maxr+appeng...@google.com > wrote:
>>>
>>>> Sure, sorry for the trouble, and thanks for trying out the prerelease!
>>>>
>>>> Max
>>>>
>>>>
>>>> On Thu, Dec 3, 2009 at 12:12 PM, Vince Bonfanti wrote:
>>>>
>>>>> That fixed it. Thanks.
>>>>>
>>>>> Vince
>>>>>
>>>>>
>>>>> On Thu, Dec 3, 2009 at 2:24 PM, Max Ross (Google) <
>>>>> maxr+appeng...@google.com > wrote:
>>>>>
>>>>>> Vince,
>>>>>>
>>>>>> As a workaround, try setting the missing environment attribute
>>>>>> explicitly before you add the task to the queue:
>>>>>>
>>>>>> ApiProxy.getCurrentEnvironment().getAttributes().put("com.google.appengine.server_url_key",
>>>>>> "http://localhost:8080";);
>>>>>>
>>>>>> This should only be necessary for tasks that are added when there is
>>>>>> no "live" request and it should have no impact in prod.
>>>>>>
>>>>>> Sorry for the trouble,
>>>>>> Max
>>>>>>
>>>>>>
>>>>>> On Thu, Dec 3, 2009 at 10:20 AM, Max Ross (Google) <
>>>>>> maxr+appeng...@google.com > wrote:
>>>>>>
>>>>>>> Hi Vince,
>>>>>>>
>>>>>>> That sounds like a bug in our code that was a side effect of the
>>>>>>> automatic task execution that now happens in the dev environment.  Let 
>>>>>>> me
>>>>>>> see if I can come up with a workaround for you.
>>>>>>>
>>>>>>> Max
>>>>>>>
>>>>>>> On Thu, Dec 3, 2009 at 10:06 AM, Vince Bonfanti >>>>>> > wrote:
>>>>>>>
>>>>>>>> I just started testing with the 1.2.8 prerelease, and I'm getting
>>>>>>>> the following exception from Queue.add() in code that works in 1.2.6:
>>>>>>>>
>>>>>>>> java.lang.IllegalStateException: Current enviornment must have the
>>>>>>>> server url available via the com.google.appengine.server_url_key 
>>>>>>>> attribute.
>>>>>>>>
>>>>>>>> This only happens when Queue.add() is invoked from the servlet
>>>>>>>> init() method or from a static initializer; if Queue.add() is invoked 
>>>>>>>> from a
>>>>>>>> "regular" request thread, then it works properly.
>>>>>>>>
>>>>>>>> Vince
>>>>>>>>
>>>>>>>>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] FinalizableReferenceQueue error

[Vince Bonfanti]

The exception is harmless. Add this to your logging.properties file to hide
it:

com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue.level=WARNING

There was previous discussion of this (search
for FinalizableReferenceQueue).

Vince

On Sat, Dec 5, 2009 at 1:00 PM, Rusty Wright wrote:

> I always get this exception when my app does its first datastore query.  I
> think it's causing me other problems.  Is there anything I can do to fix
> this?
>
> com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue
> : Failed to start reference finalizer thread. Reference cleanup will
> only occur when new references are created.
> java.lang.reflect.InvocationTargetException
>at
> com.google.appengine.runtime.Request.process-b854b5fdd4c39837(Request.java)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>at java.lang.reflect.Method.invoke(Method.java:40)
>at
> com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue.(FinalizableReferenceQueue.java:124)
>at
> com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools$WeakInterningPool.(InterningPools.java:104)
>at
> com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools.newWeakInterningPool(InterningPools.java:48)
>at
> com.google.appengine.repackaged.com.google.io.protocol.ProtocolSupport.(ProtocolSupport.java:55)
>at
> com.google.apphosting.api.DatastorePb$Query.(DatastorePb.java:1072)
>at
> com.google.apphosting.api.DatastorePb$Query$1.(DatastorePb.java:2529)
>at
> com.google.apphosting.api.DatastorePb$Query.(DatastorePb.java:2529)
>at
> com.google.appengine.api.datastore.QueryTranslator.convertToPb(QueryTranslator.java:28)
>at
> com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.convertToPb(DatastoreServiceImpl.java:424)
>at
> com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.runQuery(DatastoreServiceImpl.java:384)
>at
> com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl.asQueryResultIterator(DatastoreServiceImpl.java:358)
>at
> com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl$2.iterator(DatastoreServiceImpl.java:338)
>at
> com.google.appengine.api.datastore.DatastoreServiceImpl$PreparedQueryImpl$2.iterator(DatastoreServiceImpl.java:336)
>at
> org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterable.iterator(RuntimeExceptionWrappingIterable.java:42)
>at
> org.datanucleus.store.appengine.query.LazyResult.(LazyResult.java:67)
>at
> org.datanucleus.store.appengine.query.StreamingQueryResult.(StreamingQueryResult.java:63)
>at
> org.datanucleus.store.appengine.query.DatastoreQuery.newStreamingQueryResultForEntities(DatastoreQuery.java:399)
>at
> org.datanucleus.store.appengine.query.DatastoreQuery.wrapEntityQueryResult(DatastoreQuery.java:372)
>at
> org.datanucleus.store.appengine.query.DatastoreQuery.fulfillEntityQuery(DatastoreQuery.java:364)
>at
> org.datanucleus.store.appengine.query.DatastoreQuery.executeQuery(DatastoreQuery.java:265)
>at
> org.datanucleus.store.appengine.query.DatastoreQuery.performExecute(DatastoreQuery.java:228)
>at
> org.datanucleus.store.appengine.query.JDOQLQuery.performExecute(JDOQLQuery.java:85)
>at org.datanucleus.store.query.Query.executeQuery(Query.java:1489)
>at
> org.datanucleus.store.query.Query.executeWithArray(Query.java:1371)
>at org.datanucleus.store.query.Query.execute(Query.java:1344)
>at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:221)
>at
> com.objecteffects.waitlist.db.impl.dao.jdo.AbstractJdoDao.findAll(AbstractJdoDao.java:161)
>at
> com.objecteffects.waitlist.db.impl.dao.jdo.FacilityDao.findAll(FacilityDao.java:19)
>at
> com.objecteffects.waitlist.db.impl.dao.jdo.AbstractJdoDao.findAllDetached(AbstractJdoDao.java:174)
>at
> com.objecteffects.waitlist.db.impl.dao.jdo.FacilityDao.findAllDetached(FacilityDao.java:19)
>at
> com.objecteffects.waitlist.service.impl.AbstractService.findAllDetached(AbstractService.java:36)
>at
> com.objecteffects.waitlist.service.impl.FacilityService.findAllDetached(FacilityService.java:25)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>at java.lang.reflect.Method.invoke(Method.java:40)
>at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
>at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke

Re: [appengine-java] Re: FinalizableReferenceQueue error

[Vince Bonfanti]

The warning is caused by the initialization of a new JVM instance. The
slowness of your requests is also being caused by the initialization of a
new JVM instance.

The warning is not causing the slowness--the warning and the slowness are
both being caused by the same thing (the initialization of a new JVM
instance).

Vince

On Fri, Dec 11, 2009 at 6:54 AM, fhtino  wrote:

> Tony,
>
> thank you for the tips. I'll test them as soon as possible.
>
> I've a question. You said "the warning is not slowing down your code".
> But it increases the response time of the page. So the user waits for
> a longer time before receiving the page. Do you confirm?  Have I
> correctly understood?
>
>  fabrizio
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Introducing App Engine SDK 1.3.0

[Vince Bonfanti]

I had the same issue, so instead I went to "Help->Install New Software...",
then selected the Google App Engine site from the "Work with..." menu; then
it allowed me to select SDK 1.3.0.

Vince

On Wed, Dec 16, 2009 at 11:37 AM, Matt Farnell  wrote:

> Hi Jason,
>
> It could be just me but when I try and update to 1.3.0 from 1.2.8 in
> eclipse by clicking Help -> Software Updates -> Update I don't get any
> updates for google appengine, thats how I have always in the past got
> updates but I seem to be stuck on 1.2.8, 1.3.0 isn't coming through the
> eclipse updates (I know I can add the SDK manually but I'd prefer the
> automagical way)
>
> Is this a known issue or just me?
>
> thanks,
> Matt
>
>
>

--

You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: any plans for deferred.defer in Java?

[Vince Bonfanti]

Issue #2097 still exists on the development server; however, I just
committed a workaround. This workaround has a dependency on Apache Commons
Codec:

   http://commons.apache.org/codec/

I'm not aware of any issues in production. For convenience, here's the link
again to the source:


http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java

<http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlanta/appengine/taskqueue/Deferred.java>Let
me know if you have any problems (or if you're using it
successfully--positive feedback is always welcomed!).

Vince

On Fri, Jan 1, 2010 at 1:42 AM, John Howe  wrote:

> I greatly appreciate the work that has been done here. But let me ask my
> question another way. Is it known that this does not work?
>
>
> On Thu, Dec 31, 2009 at 5:19 PM, John Howe  wrote:
>
>> I can't seem to find any additional postings on this topic and I'm not
>> sure what the conclusion for being able to use the Deferred capability
>> with the latest Java SDK release. I'm just wondering if Vince's
>> solution works on both server and development environments.
>>
>> On Dec 16, 10:42 am, David Chandler  wrote:
>> > Thanks, Vince.
>> >
>> > Regarding a way to access the Guice injector, I've figured out I can
>> > simply create and access the injector via a static factory, so no
>> > changes are needed to Deferred or Deferrable. This works for AppEngine
>> > since there is only one WAR per JVM. If there were more than web app,
>> > I'm not sure whether Guice ServletModules would work, but that's not
>> > an issue for now.
>> >
>> > /dmchttp://turbomanage.wordpress.com
>> >
>> > On Dec 2, 2:47 pm, Vince Bonfanti  wrote:
>> >
>> >
>> >
>> > > The behavior you're reporting is exactly the opposite of what I'm
>> seeing.
>> > > For me, deserialization always fails on the dev server and always
>> works in
>> > > production. What I'm seeing on the dev server is that the bytes
>> retrieved
>> > > from the task payload (req.getInputStream) are not the same bytes that
>> are
>> > > sent; the result is the StreamCorruptedException. Again, though, I
>> only see
>> > > this on the dev server and never in production. Can you send me an
>> example
>> > > that fails in production? I'd like to understand what's going on
>> before
>> > > applying the Base64 encoder workaround.
>> >
>> > > I'm not at all familiar with Guice, but I think I understand what
>> you're
>> > > trying to do. One thought I've had is that we could pass the
>> > > HttpServletRequest object to the doTask() method. This would allow you
>> to
>> > > add attributes (objects) to the HttpServletRequest within your Guice
>> filter
>> > > and then retrieve them within your doTask() method.
>> >
>> > > Vince
>> >
>> > > On Fri, Nov 27, 2009 at 7:53 PM, David Chandler <
>> turboman...@gmail.com>wrote:
>> >
>> > > > I've solved the serialization problems by Base64 encoding the
>> > > > serialized task before calling payload() and decoding it in the
>> > > > deserialize(HttpServletRequest) method. I'm guessing something in
>> the
>> > > > task queue chain (either task queue payload storage or the servlet
>> > > > call when task is run) has problems transmitting the binary data, as
>> > > > the exception I was getting at one point was a
>> > > > java.io.StreamCorruptedException.
>> >
>> > > > I'll send another patch to Vince. My solution works, but I'm not
>> > > > completely satisfied that I've diagnosed the problem correctly. I
>> used
>> > > > Apache commons-codec for Base64. Is it safe to use com.sun.misc
>> > > > instead? I see it in appengine.repackaged chain, as well, but don't
>> > > > see it as part of an official Google API.
>> >
>> > > > Somewhat related, I've wired up the Deferred servlet through the
>> > > > GuiceFilter so I can pass a Guice injector to Deferrable tasks.
>> Since
>> > > > the task objects are not created through Guice, but rather by
>> > > > deserializing, I modified doTask() to accept an additional Injector
>> > > > argument. We probably don't want that in the official version of

Re: [appengine-java] SQL to Datastore Query object

[Vince Bonfanti]

I've written such a translater that implements a subset of JDOQL:

 https://code.google.com/p/lowlevelquery/

There's really no documentation
other than the source code, which is a single class:


https://code.google.com/p/lowlevelquery/source/browse/trunk/src/com/newatlanta/appengine/datastore/LowLevelQuery.java

The
LowLevelQuery class contains a number of static methods that let you execute
a JDOQL-like query string directly; for example:

List results = LowLevelQuery.asList( "select from myKind where
category = 'Java' and rating > 5" );

LowLevelQuery also implements the PreparedQuery interface; you can create a
LowLevelQuery instance from a query string and then use it just like you
would a PreparedQuery; for example:

PreparedQuery myQuery = new LowLevelQuery( "select from myKind where
category = 'Java' and rating > 5" );
List results = myQuery.asList();

Let me know what you think.

Vince

On Wed, Jan 6, 2010 at 3:06 PM, keyurva  wrote:

> This is likely a common scenario and I'm wondering if someone has
> already written some code to do this that I can steal.
>
> I'm using the low-level datastore API. In my app, users will be
> entering a SQL where clause. For instance:
>
> category = "Java" and rating > 5
>
> This translates to the com.google.appengine.api.datastore.Query object
> as such:
>
> Query q = new Query("a kind");
> q.addFilter("category", Query.FilterOperator.EQUAL, "Java");
> q.addFilter("rating", Query.FilterOperator.GREATER_THAN, 5);
>
> Has anyone written a generic library that does this translation?
>
> Thanks,
> Keyur
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.

To post to this group, send email to google-appengine-j...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] how to send data in blobstore as mail attachment?

[Vince Bonfanti]

I haven't tried this, but...the BlobstoreService.serve() method takes a
javax.servlet.http.HttpServletResponse instance as a parameter. My guess is
that BlobstoreService.serve() simply invokes
HttpServletResponse.getOutputStream() and then writes the data to the output
stream. You could create your own implementation of the HttpServletResponse
interface that simply writes the output to a byte array
(java.io.ByteArrayOutputStream). You could then retrieve the byte array and
do whatever is needed to send it as an attachment. Something like this:

   ByteArrayHttpServletResponse response = new
ByteArrayHttpServletResponse();
   BlobstoreService.serve( myBlobKey, response );
   byte[] data = response.toByteArray();

Attached is a quick-and-dirty skeleton for ByteArrayHttpServletResponse that
should get you started.

Vince

On Sun, Jan 3, 2010 at 2:32 AM, Aadith  wrote:

> Hi,
> I am trying to design an application that would require me to retrieve
> data stored in blobstore and send it as attachment. Does google app
> engine allow this?  From the documentation, i could not find a way to
> retrieve data from blobstore for processing within the app.. can
> someone please tell me how to accomplish this? Code examples and/or
> pointers to related online resources would be really helpful.
>
> Thanks,
> Aadith
>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.

To post to this group, send email to google-appengine-j...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.



ByteArrayHttpServletResponse.java
Description: Binary data

Re: [appengine-java] handling "com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request to API call datastore_v3.Put() was too large."

[Vince Bonfanti]

Try catching com.google.appengine.api.datastore.DatastoreFailureException,
as indicated in the docs:

http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/DatastoreService.html#put(com.google.appengine.api.datastore.Entity)

Vince


On Thu, Jan 7, 2010 at 11:25 PM, Prashant Gupta wrote:

> hi,
>
> "com.google.apphosting.api.ApiProxy$RequestTooLargeException: The request
> to API call datastore_v3.Put() was too large." was thrown while saving an
> entity to datastore. I want to handle this exception but I am not getting
> which exception to catch. Please help me out...
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.

To post to this group, send email to google-appengine-j...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] problem with Java Task Queue API...

[Vince Bonfanti]

Hi Larry,

There was a discussion of this some time ago; these exceptions are "harmless
but annoying":


http://www.mail-archive.com/google-appengine-java@googlegroups.com/msg00979.html

You can turn them off by setting the following in your logging.properties
file:


com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue.level=WARNING

Vince

On Tue, Jan 12, 2010 at 6:51 PM, Larry Cable  wrote:

> Using Vince Bonfanti's excellent GaeVFS pkg I am getting this
> exception trace from the Task Queue API...
>
>
> com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue
> : Failed to start reference finalizer thread. Reference cleanup
> will only occur when new references are created.
> java.lang.reflect.InvocationTargetException
>at com.google.appengine.runtime.Request.process-679c01b24082cbe2
> (Request.java)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>at java.lang.reflect.Method.invoke(Method.java:42)
>at
>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.

To post to this group, send email to google-appengine-j...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] task cancel, task insert, url fetch delay, servlet context listener

[Vince Bonfanti]

There's an open issue requesting an API to delete/cancel tasks; you can star
it if you'd like to see it implemented:

  http://code.google.com/p/googleappengine/issues/detail?id=2588

Vince

On Sat, Jan 23, 2010 at 9:36 AM, mete  wrote:

>
> Hello,
>
> It seems there is no task delete/cancel operation for a queue, is
> there any plan for it ?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.