[appengine-java] Re: Not able to write string arrays in data store

2010-03-12 Thread datanucleus
You mean you now have ArrayListArrayList, as opposed to the array
mentioned in your first post ?

ArrayListArrayList is not a supported property type. ArrayList is.
You could obviously add a dummy class (e.g MyTempClass) as persistable
with the other ArrayList in it as a field so the original field
becomes ArrayListMyTempClass

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Objectify - Twig - approaches to persistence

2010-03-12 Thread Jeff Schnitzer
On Thu, Mar 11, 2010 at 8:56 PM, John Patterson jdpatter...@gmail.com wrote:

 But for typesafe changes large or small Twig supports data migration in a
 much safer, more flexible way than Objectify.  Read on for details.

You are increasing my suspicion that you've never actually performed
schema migrations on big, rapidly changing datasets.

 Cool, the @AlsoLoad is quite a neat feature.  Although very limited to
 simple naming changes and nothing structural.  All this is based on a
 dangerous assumption that you can modify live data in place.  Hardly
 bullet proof.

Actually, @AlsoLoad (in conjunction with @LoadOnly and the @PrePersist
and @PostLoad lifecycle callbacks) provides an enormous range of
ability to transform your data.  I know, I've had to do more of it
than I would like to admit.  You can rename fields, change types,
arbitrarily munge data, split entities into multiple parts, combine
multiple entities into one, convert between child entities and
embedded parts, etc.

In most cases you can do this on a live running system.  That is the
entire point, actually - our goal is zero downtime for schema
migration.  The general approach:

 * Modify your entities to save in your new format.
 * Use Objectify's primitives so that data loads in both the old
format and the new format.
 * Test your code against your local datastore, or if you're deeply
concerned, against exported data in another appid.
 * Deploy your new code, letting the natural churn update your database.
 * Fire off a batch job at your leisure to finish it off.
 * Remove the extra loading logic from your code when you're done.

Not every migration works exactly the same way, but the tools are
there.  I know from experience that it works and works well.

 The Twig solution is to create a new version of the type (v2) and process
 your changes while leaving the live data completely isolated and safe.  Then
 after you have tested your changes you bump up the version number of your
 live app.

This is cumbersome and inelegant compared to Objectify's solution.
You require the developers to 1) create a parallel hierarchy of
classes and 2) create code (possibly scattered across the app) to
write out both formats.  You require a complete duplication of the
datastore kind - potentially billions of entities occupying hell only
knows how much space.  It could take *weeks* to do even minor schema
migrations this way.  And if you want to make another minor change
halfway through the process?  Start from scratch!  In the mean time,
your customers are wondering why the new feature isn't live yet.

Also... do you realize how slow and expensive deletes are in
appengine?  Duplicating the database is just not an option.  Not with
the Mobcast 2.0 dataset (not live yet, I should be able to talk about
it more freely in a month or two).  Certainly not with Scott's
dataset, which may end up caching a significant chunk of Flickr,
Picasa, and Facebook if it takes off.

 What is with your obsession with batch gets?  I understand they are central
 in Objectify because you are always loading keys.  As I said already - even
 though this is not as essential in Twig it will be added to a new load
 command.

Batch gets are *the* core feature of NoSQL databases, including the
GAE datastore.  Look at these graphs:

http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-get-latency
http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-query-latency

Notice that a get()'s average latency is 50ms and a query()'s average
latency is 500ms.  Last week the typical query was averaging
800-1000ms with frequent spikes into 1200ms or so.

Deep down in the fiber of its being, BigTable is a key-value store.
It is very very efficient at doing batch gets.  It wants to do batch
gets all day long.  Queries require touching indexes maintained in
alternative tablets and comparatively, the performance sucks.

I'm by no means a BigTable expert, but I have a significant
professional interest in being able to read  write a lot of data.  I
could not implement (perhaps better said I couldn't scale) Mobcast
without batch gets and sets.

To be honest, I'm not wholly thrilled with the performance of batch
get/put operations on appengine either.  Cassandra folks are claiming
10k/s writes *per machine*.  Tokyo Tyrant folks are claiming 20k+/sec
writes.  Reads are even faster!  True, these systems are not as
full-featured as the appengine datastore... but we're talking at least
two full orders of mangitude difference!  Ouch.

Why am I obsessed with batch gets?  Because they're essential for
making an application perform.  They're why there is such a thing as a
NoSQL movement in the first place.

 Oops I didn't post the CookBook page in the end.  Rest assured it is a
 trivial addition and I'll update the docs.
 It is also often better to cache above the data layer - hardly the killer
 feature you claim.

If you have a 

Re: [appengine-java] Unable to upload app: Error posting to URL:

2010-03-12 Thread Chummar Maly
How  where  to specify the proxy settings ?.


On Thu, Mar 11, 2010 at 3:25 PM, Ikai L (Google) ika...@google.com wrote:

 Yes, you will have to specify your proxy settings.

 On Tue, Mar 9, 2010 at 8:32 AM, WillSpecht willspe...@gmail.com wrote:
  I am getting the following error when trying to upload my app to app
  engine.  What does the 302 Redirected mean?  I am behind a proxy could
  this be part of the problem?
 
  Error
  Tue Mar 09 11:12:47 EST 2010
  Unable to upload app: Error posting to URL:
 
 http://appengine.google.com/api/appversion/create?app_id=black3liveversion=1302
  Redirected
 
 
  See the deployment console for more details
 
  com.google.appengine.tools.admin.AdminException: Unable to upload app:
  Error posting to URL:
 http://appengine.google.com/api/appversion/create?app_id=black3liveversion=1;
  302 Redirected
 
  at
  com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:
  59)
  at
 
 com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngineBridgeImpl.java:
  271)
  at
 
 com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(DeployProjectJob.java:
  148)
  at
 
 org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:
  38)
  at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
  Caused by: java.io.IOException: Error posting to URL:
 
 http://appengine.google.com/api/appversion/create?app_id=black3liveversion=1;
  302 Redirected
 
  at
 
 com.google.appengine.tools.admin.ServerConnection.send(ServerConnection.java:
  143)
  at
 
 com.google.appengine.tools.admin.ServerConnection.post(ServerConnection.java:
  81)
  at
 
 com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.java:
  415)
  at
 
 com.google.appengine.tools.admin.AppVersionUpload.beginTransaction(AppVersionUpload.java:
  229)
  at
 
 com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload.java:
  98)
  at
  com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:
  53)
 
  --
  You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
  To post to this group, send email to
 google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.
 
 



 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 http://googleappengine.blogspot.com | http://twitter.com/app_engine

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




-- 
Chummar Maly
http://servetube.appspot.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Objectify - Twig - approaches to persistence

2010-03-12 Thread Nacho Coloma
  You are increasing my suspicion that you've never actually performed
  schema migrations on big, rapidly changing datasets.

 You are increasing my suspicion that you like to make inflammatory  
 remarks without thinking them through just for the sake of trolling.  

I have one question more or less related to this thread. Why are both
Objectify and Twig tying the schema upgrade to the persistence
framework? As far as I can recall no persistence framework does this,
and it hasn't been that Hibernate didn't have the chance.

If I am following correctly, you are proposing to specify the
migration path in the persistence metadata, but I just don't get it.
Why?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Help with modeling JDO persistent classes

2010-03-12 Thread objectuser
One way to do this would be to duplicate A.name on the associated Bs.

class A {
  Long id;
  String name;
  ...
}

class B {
  Long id;
  Long Aid;
  Long Aname;
  ...
}

Then you'd be able to do your select on just the B entity group and it
would work.

On Mar 10, 2:59 am, kattus g.adjiashv...@gmail.com wrote:
 Hi,

 I have 2 persistent classes:

 1. class A that has a primary key (Long) and a property called
 name (String).
 2. class B that is referencing class A (one to many relationship, each
 B has one A, but A can belong to many B's)

 I need to retrieve the B's sorted by the name property of A. In
 other words if it was relational database I would make something like
 this (simplified):

 SELECT * FROM A, B WHERE A.id=B.Aid ORDER BY A.name

 The question is how to make this with JDO. I don't want to make A and
 B in the same entity group and it seams it is not necessary either, I
 think using unowned relationships may be a good direction, the help is
 too basic though:

 http://code.google.com/appengine/docs/java/datastore/relationships.ht...

 The questions are:

 Is it possible to define such a relationship between A and B? If yes
 how (which annotations)? Do I have to use the Key class in B to
 reference A?
 How to use the Google query language to write the correct query?

 Thank you,
 Gil

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Do I need unowned relationships to accomplish this...?

2010-03-12 Thread objectuser
How about something like this?

class User {
  Long id;
  ...
}

class Deck {
  Long id;
  Long userId;
  ...
}

class Card {
  Long id;
  Long deckId;
  ...
}

Then inserting a card into the deck is a simple insert and finding all
cards in a deck is a single query.  The same for adding a deck to a
user.

This structure has its own trade-offs, of course.

On Mar 10, 4:15 pm, tempy fay...@gmail.com wrote:
 Exactly, CardList is potentially very large and I want to avoid having
 to load it just to add the cardreference.

 On Mar 10, 10:43 pm, WillSpecht willspe...@gmail.com wrote:

  So if you have a reference to a new card and do

  cardList.add(cardRefference)

  all you are loading into memory is the card list and the new card.

  Is this what you are trying to avoid?

  On Mar 10, 4:06 pm, tempy fay...@gmail.com wrote:

   Actually cards can only be owned by one deck... so that's not a
   problem.  Deck--1...0toN--card.

   The thing that I am looking for is a way to add new cards without
   loading a deck's entire card collection, and to add decks without
   loading a User's entire deck collection.

   On Mar 10, 9:15 pm, WillSpecht willspe...@gmail.com wrote:

The way I understand it, if an object can be owned by more than one
object it must be unowned.  I would assume that cards can be in
multiple decks so they must be unowned.  I would assume each deck
would belong to one user so decks could be owned.  I don't know a good
way to store cards that can be queried in one query unless you have
each card store what decks they are in.  This could be even more
difficult if cards appear more than once in a deck.  If that is true I
would suggest a join table.

On Mar 10, 2:20 pm, tempy fay...@gmail.com wrote:

 I have the following datastructure:

 Users are the root entities, and each user can have one or more
 decks, and each deck can have one or more cards.

 When a user wants to add a deck, I would like to be able to add the
 deck to the user's collection of decks without first fetching all of
 the user's decks (potentially a large amount of data), then adding the
 new deck to that collection, and then persisting the user.  Rather, I
 would like to simply instantiate the deck and append it to the user's
 collection of decks, without ever retrieving the entire collection.

 Similarly, if a user wants to add a new card to an existing deck, I
 would like to add the card to the deck without first retrieving the
 entire deck (that is, the deck with all of its cards).

 I would like to preserve the option of fetching a user with a
 populated collection of all their decks and to retrieve a deck with a
 populated collection of all its cards, which is possible with owned
 relationships.  But to accomplish what I have mentioned above, would I
 be forced to use unowned relationships? (Collections of keys instead
 of collections of objects.)

 Thanks,
 Mike

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Error when deleting entities - Id cannot be zero

2010-03-12 Thread Pavel Byles
I'm trying to delete all entities in my datastore but I receive the
following error:

javax.jdo.JDOUserException: One or more instances could not be deleted...
NestedThrowablesStackTrace:
java.lang.IllegalArgumentException: id cannot be zero...

Caused by:java.lang.IllegalArgumentException: id cannot be zero


For the following code:

  public void deleteAllMyType() {
PersistenceManager pm = PMF.get().getPersistenceManager();
Query query = pm.newQuery(MyType.class);
try {
  query.deletePersistentAll();
  //ListMyType clist = (ListMyType) query.execute();
  //pm.deletePersistentAll(clist); // This doesn't work either
} finally {
  query.closeAll();
  pm.close();
}
  }

My entity class looks like this:

@PersistenceCapable(identityType = IdentityType.APPLICATION)//, detachable =
false)
public class MyType implements Serializable {
  @PrimaryKey
  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
  private Long id;

  @Persistent
  private String name;
  .
  .
  .
}

-- 
-Pav

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] App Engine Bug

2010-03-12 Thread Henning
Hello,

if I have a servlet only the admin has access to and I logged in/
registered to App Engine  with my Google apps account there is no way
to run this servlet although I am an admin because it wants me to
authenticate/login as admin with my regular Google account (it does
not support the Google apps login)

Regards,
Henning

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: How exactly do the App Engine Logs work?

2010-03-12 Thread Don Schwarz
I'm not sure that we document or guarantee this anywhere, but we currently
seem to be preserving these for 90 days.  I don't know how feasible it is to
retrieve all 90 days of data via appcfg, though.

On Thu, Mar 11, 2010 at 8:50 PM, Spines kwste...@gmail.com wrote:

 Thanks for your help Don.  Just one more question (hopefully :)).

 I think just the request logs will be good enough for my purposes, and
 I won't actually have to use the diagnostic logs.  How much past data
 is stored for the request logs?

 On Mar 11, 8:16 am, Don Schwarz schwa...@google.com wrote:
  On Wed, Mar 10, 2010 at 8:05 PM, Spines kwste...@gmail.com wrote:
   Sorry, I don't think I really understood the task queue approach. To
   ensure no loss of data, would the task have to update in the datastore
   every time?  And the benefit over just doing it directly in the
   servlet handler would be the faster response time to the user?
 
  Basically, yes.
 
   I think the memcache solution may be my best bet.  I can tolerate some
   chance of loss of data.  I'm wondering how likely loss of data would
   be? How often does the data get booted out? If I persist from memcache
   to datastore every 10 seconds would data loss be super rare?
 
  You would have to experiment with this as we don't make any guarantees,
 but
  I believe that memcache data generally survives much longer than 10
 seconds
  of inactivity, yes.
 
  On Mar 10, 4:11 pm, Spines kwste...@gmail.com wrote:
 
 
 
Thanks Don,
I thought about the task queue, but that caps at being able to
 execute
like 5 tasks per second right?
 
So, as long as the log data doesn't get full before I download it
 then
it would be fine?
 
On Mar 10, 3:42 pm, Don Schwarz schwa...@google.com wrote:
 
 Yeah, those are diagnostic logs.  They effectively go into a ring
   buffer per
 logging level, so the maximum data stored at any given time is
 capped.
The
 more you log, the more frequently you would have to download the
 logs
   to
 avoid missing any.  You would also be competing with log space with
 any
 other log messages generated by your application.
 
 What I would suggest instead is either to increment counters in
   memcache,
 and flush them to the datastore periodically if you need durability
   (I'm
 assuming you can tolerate some chance of data loss here).  If you
   cannot
 tolerate any loss of data, then I would suggest enqueueing tasks to
 a
   task
 queue for each request that maintains a summary in memcache and/or
 the
 datastore.
 
 On Wed, Mar 10, 2010 at 4:59 PM, Spines kwste...@gmail.com
 wrote:
  I'm talking about the logs that get written when I call
  Logger.info(something).
 
  Basically this is what I'm thinking: I have certain data that
 needs
   to
  get written very often, but hardly ever needs to be read (stuff
 like
  what users view what pages of my site).  The datastore is
 optimized
  for read efficiency. So, I want to output this data to the logs.
 I
  will have an offsite computer download these logs, do
 calculations on
  them, and upload the result of the calculations to the datastore.
 
  On Mar 10, 2:03 pm, Don Schwarz schwa...@google.com wrote:
   Are you talking about request logs or diagnostic logs?
  Although we
  conflate
   them a bit in both the Admin Console viewer and the appcfg
 command,
   but
  they
   are stored and tracked separately.
 
   On Wed, Mar 10, 2010 at 4:01 PM, Spines kwste...@gmail.com
   wrote:
Hmm, that is my biggest concern, log reliability.  Can
 someone
   from
Google confirm whether or not I can rely on the logs having
 all
   of the
log data? Or might certain entries just disappear?
 
On Mar 10, 1:24 pm, thierry Le conniat thlec...@euriware.fr
 
   wrote:
 Hello,
 I think google log are stored in file.
 My experience about log reliability is that when the app is
   very
 strong working, not all the log are stored.
 It's confusing, but i can't not explain it.
 
 Bye
 
 On 10 mar, 22:04, Spines kwste...@gmail.com wrote:
 
  Where does Google store the logs when you do a Logging
   statement?
  Logging statements seem to be pretty fast, so it doesn't
 seem
   like
  they are stored in the datastore.
 
  How reliable are the logs? If I do a logging statement
 and it
  succeeds, is it pretty much guaranteed that it will show
 up
   in the
  logs?
 
  How much past history of logs is stored?
 
  The reason I'm interested in this is because I'm making a
   question
  and
  answer website, and I want to keep track of views by each
   unique
  logged in user to each question, and display the view
 count
   on the
  question page. So if 10 different users visit the
 question
   page 100
  times, it still only counts 

[appengine-java] Re: Spring MVC - File upload problem

2010-03-12 Thread amit
Is there any demo in spring mvc also ?

On Mar 2, 7:22 am, yjun hu itswa...@gmail.com wrote:
 hi, i got a demo here.http://hapeblog.appspot.com/blog.shtml?id=2002

 On Tue, Mar 2, 2010 at 4:36 AM, Sebastian Cartier 
 sebi.cart...@gmail.comwrote:



  Hi
  my first solution i wrote isn't working any more. I had to add

  beans = {

  multipartResolver(is.hax.spring.web.multipart.StreamingMultipartResolver)
  }

  to /grails-app/conf/spring/resources.xml
  Now it works again

  @Markus: you can not use byte-arrays with google app engine. You have
  to use blobs! See my previous solution.
  I put the library in myApplication/lib

  On 2 Feb., 03:08, Markus Paaso markus.pa...@gmail.com wrote:
   Hi

   I tried it with Grails 1.2.0 and app-engine plugin 0.8.8 but got just
   an another error:

   java.lang.NoClassDefFoundError: Could not initialize class
   org.apache.commons.fileupload.disk.DiskFileItem
           at
  org.apache.commons.fileupload.disk.DiskFileItemFactory.createItem
   (DiskFileItemFactory.java:196)
           at org.apache.commons.fileupload.FileUploadBase.parseRequest
   (FileUploadBase.java:358)
           at
   org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest
   (ServletFileUpload.java:126)
           at

  org.springframework.web.multipart.commons.CommonsMultipartResolver.parseRequest
   (CommonsMultipartResolver.java:155)

   my controller:

           def imageInstance = new Image(imageParams)
           def f = request.getFile('imageData')
           imageInstance.imageData = f.getBytes()
           Image.withTransaction {
                   if(imageInstance.save(flush:true)) {
                       flash.message = Image ${imageInstance.id} created
                       redirect(action:show,id:imageInstance.id)
                   }
                   else {

  render(view:'create',model:[imageInstance:imageInstance])
                   }
           }

   and domain-class:

   import javax.persistence.*;
   // import com.google.appengine.api.datastore.Key;

   @Entity
   class Image implements Serializable {

       @Id
       @GeneratedValue(strategy = GenerationType.IDENTITY)
       Long id
       byte[] imageData

       static constraints = {
           id visible:false
       }

   }

   It seems like the multipart resolver is not replaced with the new one.
   I placed the jar file into myApplication/lib and GRAILS_HOME/lib
   directories.
   Maybe I didn't place the jar to the right directory?
   Would you like to tell more about how you got it to work?

   Markus

   On 2 helmi, 00:00, Sebastian Cartier sebi.cart...@gmail.com wrote:

this works also for grails!
Add the library to your lib directory and add
    bean id=multipartResolver

class=is.hax.spring.web.multipart.StreamingMultipartResolver
    /bean
to your applicationContext.xml

For saving the image in a google blob i used the following functions:
        @Persistent
        Blob imageBlob

        byte [] getImage(){
                if(imageBlob){
                        imageBlob.getBytes()
                }else{
                        null;
                }
        }

        void setImage(byte [] imageBytes){
                imageBlob = new Blob(imageBytes)
        }

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine for Java group.
  To post to this group, send email to
  google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.

 --
 dream or truth

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: App instance recycling and response times - is there solution?

2010-03-12 Thread xcdesz
This is NOT just a problem with Spring -- stop talking like
optimization is going to fix things.  It takes too much time for a
naked servlet to load (i.e; 5-10 seconds).  The only jars that I have
are for JPA.

On Jan 12, 8:32 pm, Jeff Schnitzer j...@infohazard.org wrote:
 I've been thinking about this issue a little.  It's not quite as
 straightforward as just keeping an instance warm.  Even if you have an
 app that gets multiple hits per second, there will still be cold
 starts:

  * When a new instance comes online to serve more demand.
  * When you redeploy a version of your app.

 Is appengine smart about letting new instances added to the pool warm
 up before serving requests?  It's hard to tell from my logs but it
 doesn't look like it.

 I know appengine is *not* smart about warming up an instance before
 redeploying.  When I redeploy, some large number of users must wait
 while the appserver(s) startup.

 One thing to keep in mind during these discussions is how other Java
 EE environments solve this problem:  They *don't*.  For a long time
 it's been assumed in the EE development that server initialization
 time is irrelevant, and we grew fat libraries that take tens of
 seconds to minutes to start up.  The problem is, this time has *never*
 been irrelevant - even in a production environment you must deploy new
 versions of your app, and none of the appservers I'm familiar with are
 smart enough to keep serving off the old version while the new one
 loads.  Users with unlucky timing always got screwed.

 We just didn't care because we only deployed code once a week and we
 added/removed server instances far less often than that.  Well guess
 what, now it's easy - you can deploy up to 1,000 times per day just by
 clicking a button in eclipse, and server provisioning is now not only
 trivial but 100% transparent to you.  Just try that with WebSphere!

 You aren't going to like this, but here's the only answer that isn't
 going to piss off your customers:  Stop using Spring.  Stop performing
 eager initialization.  Stop assuming that users don't see startup
 time.  Yes, change the way you write code.

 Jeff



 On Tue, Jan 12, 2010 at 1:11 PM, Don Schwarz schwa...@google.com wrote:
  Make sure you are using offline precompilation.  We are always working on
  optimizations to decrease the latency of loading requests, but here are some
  other tips:
 http://googleappengine.blogspot.com/2009/12/request-performance-in-ja...
  On Tue, Jan 12, 2010 at 3:01 PM, Locke locke2...@gmail.com wrote:

  I agree that making users wait 20 seconds for your app to load is not
  adequate for the vast majority of apps. I also agree that
  reengineering everything to try and hide load times from users is a
  poor solution in most cases.

  Using cron to keep your app loaded will not consume your quota; it
  will actually conserve your quota. Every time your app loads you will
  be billed for 20s of CPU time. If you keep it loaded, you will only be
  billed for a few milliseconds per 'keep-alive' cron execution.

  However, the Google engineers who post here have recommended against
  doing this. If everyone did it, appengine might run out of resources
  (RAM, I assume).

  I imagine that Google will need to either find a way to load apps in
  1/10th the time (the ideal solution), raise prices significantly, or
  ration  resources in some other way.

  If I may make a suggestion to the Google engineers: offer a keep my
  app loaded option and make it available ONLY for billing-enabled
  apps. Disable cron for apps which are not billing-enabled, so that
  people who just want free hosting or are merely toying with appengine
  won't be using up resources all the time.

  This way, the people who have shown that they are serious about
  appengine (by laying their cash down) won't be driven away by the
  people who are just fooling with it.

  Yes, we are seriously considering something like this.  Please star this
  issue for updates:
 http://code.google.com/p/googleappengine/issues/detail?id=2456

  On Jan 12, 1:43 pm, Konrad konradpaw...@gmail.com wrote:
   I asked same question on Stack Overflow (http://stackoverflow.com/
   questions/2051036/google-app-engine-application-instance-recycling-and-
   response-times).

   So far proposed solutions (in SO thread and found on other websites)
   do not satisfy me. Creating cron or any other kind of periodic HTTP
   requests to keep instance up and running make no sense. First - there
   is no evidence that this instance will serve next coming request (eg.
   from different network location etc.), second - it will consume Quota
   (which is less a problem).

   Other solution - refactoring app - replacing critical functionality
   with lightweight servlet - sounds better, but is GAE forcing to go
   back to CGI programming style? And I could replace let say - API
   calls, but to keep UI in better performance shape I would need to
   throw away Spring MVC.

   Can anyone 

[appengine-java] Re: Flash arcade game GAE based

2010-03-12 Thread Ahmed Khalifa
thanks for the great help ..
the problem is that GAE provided an excellent online developping
platform that allowed me to upload my code and update the versions of
the game and test it on daily basis .. it was a great disappointment
to know that sandbox restrictions prevent me from opening a socket but
any how .. i just need to move to another platform ..
the problem is, it seems that my server will need less restrictions
than the free hosts usually provide ..
i would really appreciate it if you can help me with a start .. if
there's any website that hosts web applications from the type of my
game for free (at least in the beginning of deployment ) i would be
very happy to know about it ..
best regards,
A. Khalifa

On Mar 8, 10:50 pm, nicolas melendez nfmelen...@gmail.com wrote:
 Hi, there are defferent types of games, first you should know in which
 category your game  is:

 1) Real time, like Quake, you need a Socket connection, TCP when you want
 reliable data and also UDP when reliable data is not important and will
 change soon, like position in a map. So GAE won't help you, HTTP and also
 XMPP are very slow for that kind of game where 500ms is eternity.

 2) Game with turns, like a chess game. Here Gae Can help you with HTTP or
 XMPP.
 I am making a game based on turns, i have choose the XMPP way, but i didn't
 finish yet, so i can't it was a right choice. But for the moment i can say,
 that your
 main dificult task, will be time optimization with the satastore.
 Also you should know that XMPP works right in deployment, but in
 development, you can't test your application. Here you need to do some hack,
 simulate XMPP responses.

 3) Web base games, the are very static and time isn't important,  here GAE
 also can help you. an example ishttp://www.mafia-family.com.

 Hope i Help.
 NM

 On Mon, Mar 8, 2010 at 5:11 PM, Ikai L (Google) ika...@google.com wrote:

  HTTP is just not a great protocol for real time games. Even with XMPP,
  there's a chance outgoing responses will be queued and delayed for a
  bit as capacity requires.

  If you want it to be truly real time, you'll have to develop your own
  server that is capable of maintaining an open socket connection.
  Trying to fit client/server communications in a real time game into
  the HTTP model is like trying to watch a movie by having your friend
  record it on his camera phone and MMSing it to you as quickly as
  possible.

  On Mon, Mar 8, 2010 at 9:08 AM, Ahmed Khalifa derkhal...@gmail.com
  wrote:
   I have to tell you that i had to slow down the rate by which flash
   sends requests to the server in order to give GAE some time to respond
   which consequently kills the real-time game play intended ..
   after all, even if GAE was not a good choice for a semi-real time game
   can anyone please give me valid reasons for why to disregard GAE??
   many people are keeping telling me that but no one is giving me real
   reasons for why .. plus, i have been really distracted by the amount
   of suggestions that do not converge to on direction .. some are
   suggesting using XMPP, others say long polling is the way, some say
   GraniteDS is a valid option along with memcache .. I am really
   confused about the decision to be taken ..
   regards,
   A. Khalifa

   On Mar 8, 6:21 pm, Robert Lancer robert.lan...@gmail.com wrote:
   Yeah, I certainly would not use GAE Java for anything that has to be
   semi real time. Have you checked out Red5.org or Web Orb at
   themidnightcoders.com? Those are designed to work directly with
   Flash.

   On Mar 8, 10:56 am, Ahmed Khalifa derkhal...@gmail.com wrote:

thanks a lot ..
however, i am still doubtful about the source of latency .. is it GAE
itself not supporting a certain feature that allows real-time response
or it is something that i lacked ..
thanks in advance ..

On Mar 4, 3:27 pm, Toby toby.ro...@gmail.com wrote:

 Hi Ahmed,

 take a look at GraniteDS. It is a bit like blazeds with the ability
  to
 push and syncronize data between multiple clients. It uses a very
 efficient serialization and it runs on GAE:
 http://graniteds.blogspot.com/2009/04/graniteds-20-on-google-app-engi...
 on server side you just need to make sure that the datastore updates
 do not block your clients. I would suggest using memcache and to add
 datastore updates to the task queue.
 So when you receive a data change from a client you update the
 memcache value, you propagate the change to all clients and you put
  on
 the task queue a request for the data to be saved. This will be done
 asynchronously.

 Cheers,
 Toby

 On Mar 3, 5:47 am, nicolas melendez nfmelen...@gmail.com wrote:

  I want to do the same. Is there any good XMPP Java framework to
  include in
  my Applet? recomendations?
  Thanks
  NM

  On Tue, Mar 2, 2010 at 11:31 PM, tsp...@green20now.com wrote:
   Kaz,
      Not sure about the CPU (we are still 

[appengine-java] Re: How do I write data in my Google App Engine Datastore to com.google.appengine.api.datastore.Text

2010-03-12 Thread Jake
Hey,

I presume setMethod() refers to a getter/setter.  So, your persisted
class would look like:

@Persistent
Text text;

public void setText(String s) {
   this.text = new Text(s);
}

public String getText() {
   return this.text.getValue()
}

The App Engine API is your friend:  
http://code.google.com/appengine/docs/java/javadoc/

Jake

On Mar 11, 8:37 pm, Tristan tristan.slomin...@gmail.com wrote:
 Lloyd,

 String reallyLong = It was the best of times, it was the worst of
 times. (...) ..;

 Text myText = new Text(reallyLong);

 I don't understand your reference to setMethod().

 Cheers!

 On Mar 10, 10:53 pm, Lloyd cled...@gmail.com wrote:

  I have persistent object, with a string property that often is over
  500 charachters. Google App Engine says I need to save it as a
  com.google.appengine.api.datastore.Text.

  How do I either convert a String type to a
  com.google.appengine.api.datastore.Text type so I can use a
  setMethod() on the property, or otherwise get my long sting data into
  that persistent value?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Cryptography on App Engine

2010-03-12 Thread Spines
I ended up just using javax.Crypto. If I try out bouncy castle in the
future I'll post the results here.

On Mar 12, 10:24 am, Ikai L (Google) ika...@google.com wrote:
 It's not on our Will it play page:

 http://groups.google.com/group/google-appengine-java/web/will-it-play...

 This doesn't mean it won't work, it just means no one has tried it.
 Can you try it and let us know so we can update the page?

 On Thu, Mar 11, 2010 at 8:44 PM, Spines kwste...@gmail.com wrote:
  I think bouncy castle is a good library to use, does anyone know if it
  works on the app engine?

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine for Java group.
  To post to this group, send email to google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine-java+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine-java?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App 
 Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: Objectify - Twig - approaches to persistence

2010-03-12 Thread Jeff Schnitzer
Scott:  Nacho is the author of SimpleDS.

Schema migration is something that Hibernate and RDBMSes actually do
rather poorly.  The typical process is to prepare a series of scripts
(ALTER TABLE and then any relevant data transmogrification), shut down
the application, run the scripts, then bring up new code that
understands the new schema.  Hibernate may help provide the scripts,
but it won't prevent your downtime.

With large data volumes and significant changes, this process could
take a very very long time.

The schemaless nature of AppEngine makes migration both easier and
harder.  Any entity can have any shape, but there are no bulk
operations.  Furthermore, iterating through your dataset and resaving
each entity is a *very slow* operation.  Batch jobs on my entities
(which have very few indexes) convert less than 100 instances per
second.  Want to convert 1 million entities?  Nearly 3 hours.  10
million?  100 million?  The math is easy.

We designed the Objectify schema migration tools with the assumptions:

 1) You have vast quantities of data
 2) The data is changing rapidly
 3) Any amount of downtime is unacceptable

All transformations within a single entity are pretty easy and
straightforward using @AlsoLoad (on a field, lets you load the old
name as well as the new) or on a method parameter (lets you obtain the
raw data as whatever format you need, munge it, and save it to
whatever set of fields you care).

Condensing multiple entities into a single entity is also fairly
straightforward using @PostLoad and @PrePersist.  I'll provide an
example if you want.

Splitting apart one entity into multiple is by far the hardest
transformation, and there isn't any one solution that works for
everyone.  However, I have done it.  Here is one strategy:

 Starting Entity 
class Person {
@Id Long id;
String addressStreet;
String addressCity;
}

Goal: Create an Address entity and add a foreign key reference in Person.

 Translating Entity 
class Address {
@Id Long id;
String street;
String city;
}
class Person {
@Id Long id;
@LoadOnly String addressStreet;
@LoadOnly String addressCity;
KeyAddress address;

@PrePersist void onSave() {
if (addressStreet != null || addressCity != null) {
Address addy = new Address(addressStreet, addressCity);
address = ObjectifyService.begin().put(addy);
}
}
}

You might also need to implement the getAddressStreet() and
getAddresCity() methods with a check to the address Key and a load of
the Address object.

It's somewhat convoluted, but it does work.  Another alternative,
which gets rid of the complicated getAddressStreet() implementation is
to perform the conversion onLoad():

class Person {
@Id Long id;
@LoadOnly String addressStreet;
@LoadOnly String addressCity;
KeyAddress address;

@PostLoad void onLoad) {
if (addressStreet != null || addressCity != null) {
Address addy = new Address(addressStreet, addressCity);
address = ObjectifyService.begin().put(addy);
ObjectifyService.begin().put(this);
}
}
}

It really depends on the shape of your data and your query profile -
if your app reads a *lot* of entities, you might find the performance
profile of convert-on-save to be better.  Keep in mind that this type
of conversion is a worst case scenario.  Most other schema
migrations are fairly simple.

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread Pavel Byles
Anyone?

On Fri, Mar 12, 2010 at 8:41 AM, Pavel Byles pavelby...@gmail.com wrote:

 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...

 Caused by:java.lang.IllegalArgumentException: id cannot be zero


 For the following code:

   public void deleteAllMyType() {
 PersistenceManager pm = PMF.get().getPersistenceManager();
 Query query = pm.newQuery(MyType.class);
 try {
   query.deletePersistentAll();
   //ListMyType clist = (ListMyType) query.execute();
   //pm.deletePersistentAll(clist); // This doesn't work either
 } finally {
   query.closeAll();
   pm.close();
 }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//, detachable
 = false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .
 }

 --
 -Pav




-- 
-Pav

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Memory Leak in the EntityManagerFactory?

2010-03-12 Thread Max Ross (Google)
Thanks for the report David, this certainly seems suspicious.  There is at
least one memory leak I'm aware of but it's related to transactions so
that's probably not what you're bumping into.  Have you tried taking a heap
dump to see what exactly is building up?

On Fri, Mar 12, 2010 at 1:27 PM, David Fuelling sappe...@gmail.com wrote:

 I have a JUnit test class that is attempting to test some JPA
 datastore create operations, and I'm getting results that *seem* to
 indicate a memory leak in the EntityManagerFactory (?)  Basically, if
 I use test1a (see below), the heap in use by the JUnit test process
 continually increases until the JUnit test fails with an OutOfMemory
 error.  Test1b suffers from no such problem.

 I would not expect this type of behavior from test1a because even
 though I'm creating a new EntityManager upon every for-loop iteration,
 that em should go away after every for-loop iteration since the
 variable reference is replaced with a new EntityManager each time.

 Now, one might argue that my test is just going too fast, and the GC
 isn't getting a chance to Garbage Collect.  However, Test1a takes a
 pretty long time to execute on my machine ( 120 seconds), so I
 *should* be getting some GC, right?  Unless the EntityManagerFactory
 is holding onto a reference to each created EntityManager?

 Any input here would be much appreciated...

 Thanks!

 david

 ps - my UserImpl is a standard JPA entity.


 ///
 //Begin JUnit Test #1a
 ///

 User user = null;
 EntityManager em = null;
 for (int i = 0; i  5000; i++)
 {
  //See how I get an em here:

 http://code.google.com/appengine/docs/java/datastore/usingjpa.html#Getting_an_EntityManager_Instance
  em = EMF.get().createEntityManager();
  user = new UserImpl(test + i);
  em.persist(user);
  em.close();
 }

 ///
 //End Test #1b
 ///

 ///
 //Begin JUnit Test #1b
 ///

 User user = null;
 EntityManager em = EMF.get().createEntityManager();
 for(int i = 0; i  5000; i++)
 {
  user = new UserImpl(test + i);
  em.persist(user);
 }
 em.close();

 ///
 //End Test #1b
 ///

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread Max Ross (Google)
What version of the sdk are you using?

On Fri, Mar 12, 2010 at 1:36 PM, Pavel Byles pavelby...@gmail.com wrote:

 Anyone?


 On Fri, Mar 12, 2010 at 8:41 AM, Pavel Byles pavelby...@gmail.com wrote:

 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...


 Caused by:java.lang.IllegalArgumentException: id cannot be zero


 For the following code:

   public void deleteAllMyType() {
 PersistenceManager pm = PMF.get().getPersistenceManager();
 Query query = pm.newQuery(MyType.class);
 try {
   query.deletePersistentAll();
   //ListMyType clist = (ListMyType) query.execute();
   //pm.deletePersistentAll(clist); // This doesn't work either
 } finally {
   query.closeAll();
   pm.close();
 }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//,
 detachable = false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .
 }

 --
 -Pav




 --
 -Pav

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread Pavel Byles
GWT: 2.0.3

On Fri, Mar 12, 2010 at 4:38 PM, Max Ross (Google) 
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

 What version of the sdk are you using?

 On Fri, Mar 12, 2010 at 1:36 PM, Pavel Byles pavelby...@gmail.com wrote:

 Anyone?


 On Fri, Mar 12, 2010 at 8:41 AM, Pavel Byles pavelby...@gmail.comwrote:

 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...




 Caused by:java.lang.IllegalArgumentException: id cannot be zero


 For the following code:

   public void deleteAllMyType() {
 PersistenceManager pm = PMF.get().getPersistenceManager();
 Query query = pm.newQuery(MyType.class);
 try {
   query.deletePersistentAll();
   //ListMyType clist = (ListMyType) query.execute();
   //pm.deletePersistentAll(clist); // This doesn't work either
 } finally {
   query.closeAll();
   pm.close();
 }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//,
 detachable = false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .
 }

 --
 -Pav




 --
 -Pav

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




-- 
-Pav

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread Max Ross (Google)
Which version of the App Engine SDK?


On Fri, Mar 12, 2010 at 1:43 PM, Pavel Byles pavelby...@gmail.com wrote:

 GWT: 2.0.3

 On Fri, Mar 12, 2010 at 4:38 PM, Max Ross (Google) 
 maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

 What version of the sdk are you using?

 On Fri, Mar 12, 2010 at 1:36 PM, Pavel Byles pavelby...@gmail.comwrote:

 Anyone?


 On Fri, Mar 12, 2010 at 8:41 AM, Pavel Byles pavelby...@gmail.comwrote:

 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...





 Caused by:java.lang.IllegalArgumentException: id cannot be zero


 For the following code:

   public void deleteAllMyType() {
 PersistenceManager pm = PMF.get().getPersistenceManager();
 Query query = pm.newQuery(MyType.class);
 try {
   query.deletePersistentAll();
   //ListMyType clist = (ListMyType) query.execute();
   //pm.deletePersistentAll(clist); // This doesn't work either
 } finally {
   query.closeAll();
   pm.close();
 }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//,
 detachable = false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .
 }

 --
 -Pav




 --
 -Pav

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




 --
 -Pav

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread Pavel Byles
1.3.1

On Fri, Mar 12, 2010 at 4:51 PM, Max Ross (Google) 
maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

 Which version of the App Engine SDK?



 On Fri, Mar 12, 2010 at 1:43 PM, Pavel Byles pavelby...@gmail.com wrote:

 GWT: 2.0.3

 On Fri, Mar 12, 2010 at 4:38 PM, Max Ross (Google) 
 maxr+appeng...@google.com maxr%2bappeng...@google.com wrote:

 What version of the sdk are you using?

 On Fri, Mar 12, 2010 at 1:36 PM, Pavel Byles pavelby...@gmail.comwrote:

 Anyone?


 On Fri, Mar 12, 2010 at 8:41 AM, Pavel Byles pavelby...@gmail.comwrote:

 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...







 Caused by:java.lang.IllegalArgumentException: id cannot be zero


 For the following code:

   public void deleteAllMyType() {
 PersistenceManager pm = PMF.get().getPersistenceManager();
 Query query = pm.newQuery(MyType.class);
 try {
   query.deletePersistentAll();
   //ListMyType clist = (ListMyType) query.execute();
   //pm.deletePersistentAll(clist); // This doesn't work either
 } finally {
   query.closeAll();
   pm.close();
 }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//,
 detachable = false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .
 }

 --
 -Pav




 --
 -Pav

 --
 You received this message because you are subscribed to the Google
 Groups Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




 --
 -Pav

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.




-- 
-Pav

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] DataNucleus: java.lang.IllegalArgumentException: Invalid MTJ Project

2010-03-12 Thread haole
There have been numerous posts on the subject of a
NullPointerException popping up when the DataNucleus Enhancer runs. At
one point, I was able to make this problem go away, and now, it is
AGAIN preventing me from running my project.

I've seen about 5 different explanations for what's happening. While i
can't reproduce the conditions, i do know that i removed all
references to PersistenceCapable in my code and it got rid of the
problem the last time. Something else is causing the problem this
time.

Is this going to be fixed? Shouldn't there be more verbose exception
handling within the datanucleus enhancer or plugin as to why the
enhancer is failing?

Message in Eclipse error log:
An internal error occurred during: DataNucleus Enhancer

Stack trace:
java.lang.NullPointerException
at
com.google.gdt.eclipse.core.ProcessUtilities.cleanupProcess(ProcessUtilities.java:
367)
at
com.google.gdt.eclipse.core.ProcessUtilities.launchProcessAndActivateOnError(ProcessUtilities.java:
271)
at
com.google.appengine.eclipse.core.orm.enhancement.EnhancerJob.runInWorkspace(EnhancerJob.java:
82)
at
org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:
38)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)

Session data:
eclipse.buildId=M20100211-1343
java.version=1.6.0_18
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86

And the entries from workspace/.metadata/log:
!SESSION 2010-03-10 06:29:11.124
---
eclipse.buildId=M20100211-1343
java.version=1.6.0_18
java.vendor=Sun Microsystems Inc.
BootLoader constants: OS=win32, ARCH=x86, WS=win32, NL=en_US
Command-line arguments:  -os win32 -ws win32 -arch x86

This is a continuation of log file C:\Users\Joe\workspace\.metadata
\.bak_0.log
Created Time: 2010-03-10 08:56:31.234

!ENTRY org.eclipse.mtj.core 4 0 2010-03-10 08:56:31.234
!MESSAGE Invalid MTJ Project.
!STACK 0
java.lang.IllegalArgumentException: Invalid MTJ Project.
at
org.eclipse.mtj.internal.core.build.MTJBuildProperties.init(Unknown
Source)
at
org.eclipse.mtj.internal.core.build.MTJBuildProperties.getBuildProperties(Unknown
Source)
at
org.eclipse.mtj.internal.core.util.MTJBuildPropertiesResourceListener.updateBuildProperties(Unknown
Source)
at
org.eclipse.mtj.internal.core.util.MTJBuildPropertiesResourceListener.resourceChanged(Unknown
Source)
at org.eclipse.core.internal.events.NotificationManager
$2.run(NotificationManager.java:291)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at
org.eclipse.core.internal.events.NotificationManager.notify(NotificationManager.java:
285)
at
org.eclipse.core.internal.events.NotificationManager.broadcastChanges(NotificationManager.java:
149)
at
org.eclipse.core.internal.resources.Workspace.broadcastPostChange(Workspace.java:
313)
at
org.eclipse.core.internal.resources.Workspace.checkpoint(Workspace.java:
367)
at org.eclipse.ltk.core.refactoring.PerformChangeOperation
$1.run(PerformChangeOperation.java:265)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:
1800)
at
org.eclipse.ltk.core.refactoring.PerformChangeOperation.executeChange(PerformChangeOperation.java:
308)
at
org.eclipse.ltk.internal.ui.refactoring.UIPerformChangeOperation.executeChange(UIPerformChangeOperation.java:
92)
at
org.eclipse.ltk.core.refactoring.PerformChangeOperation.run(PerformChangeOperation.java:
220)
at org.eclipse.core.internal.resources.Workspace.run(Workspace.java:
1800)
at
org.eclipse.ltk.internal.ui.refactoring.WorkbenchRunnableAdapter.run(WorkbenchRunnableAdapter.java:
87)
at org.eclipse.jface.operation.ModalContext
$ModalContextThread.run(ModalContext.java:121)

!ENTRY org.eclipse.mtj.core 4 0 2010-03-10 09:02:39.870
!MESSAGE Invalid MTJ Project.
!STACK 0
java.lang.IllegalArgumentException: Invalid MTJ Project.
at
org.eclipse.mtj.internal.core.build.MTJBuildProperties.init(Unknown
Source)
at
org.eclipse.mtj.internal.core.build.MTJBuildProperties.getBuildProperties(Unknown
Source)
at
org.eclipse.mtj.internal.core.util.MTJBuildPropertiesResourceListener.updateBuildProperties(Unknown
Source)
at
org.eclipse.mtj.internal.core.util.MTJBuildPropertiesResourceListener.resourceChanged(Unknown
Source)
at org.eclipse.core.internal.events.NotificationManager
$2.run(NotificationManager.java:291)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:42)
at
org.eclipse.core.internal.events.NotificationManager.notify(NotificationManager.java:
285)
at
org.eclipse.core.internal.events.NotificationManager.broadcastChanges(NotificationManager.java:
149)
at
org.eclipse.core.internal.resources.Workspace.broadcastPostChange(Workspace.java:
313)
  

[appengine-java] Re: DataNucleus: java.lang.IllegalArgumentException: Invalid MTJ Project

2010-03-12 Thread haole
removing all of the jars from my user-defined library for gdata and
including only those that i need (core, client, calendar) seemed to
fix the problem.

i remember reading somewhere about how long classpaths can cause this
problem.

is this going to be fixed? is the complexity of my app going to be
limited by a limit on how long classpaths can be?!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: Error when deleting entities - Id cannot be zero

2010-03-12 Thread thierry LE CONNIAT
HI,
I have try your function deleteAllMyType
it's functions well, the differenxces are in my class of object :

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Picture {
private static final Logger log =
Logger.getLogger(Picture.class.getName());

@PrimaryKey
private String fileName;

...
Picture doesn't extend Serializable and my key is not an id...



On 12 mar, 14:41, Pavel Byles pavelby...@gmail.com wrote:
 I'm trying to delete all entities in my datastore but I receive the
 following error:

 javax.jdo.JDOUserException: One or more instances could not be deleted...
 NestedThrowablesStackTrace:
 java.lang.IllegalArgumentException: id cannot be zero...

 Caused by:java.lang.IllegalArgumentException: id cannot be zero

 For the following code:

   public void deleteAllMyType() {
     PersistenceManager pm = PMF.get().getPersistenceManager();
     Query query = pm.newQuery(MyType.class);
     try {
       query.deletePersistentAll();
       //ListMyType clist = (ListMyType) query.execute();
       //pm.deletePersistentAll(clist); // This doesn't work either
     } finally {
       query.closeAll();
       pm.close();
     }
   }

 My entity class looks like this:

 @PersistenceCapable(identityType = IdentityType.APPLICATION)//, detachable =
 false)
 public class MyType implements Serializable {
   @PrimaryKey
   @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
   private Long id;

   @Persistent
   private String name;
   .
   .
   .

 }

 --
 -Pav

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



[appengine-java] Re: JPA enhancement problem (DataNucleus)

2010-03-12 Thread thierry LE CONNIAT
Hi,
I had this problem, and after try to clean your project and rebuild
it .
bye

On 12 mar, 19:34, Rajeev Dayal rda...@google.com wrote:
 Hi there,

 Can you provide the full error message that you're seeing when you get this
 error?

 Also, can you navigate to the Error Log (Window - Show View - Error Log)
 and see if there are any errors related to this problem listed there?

 Thanks,
 Rajeev



 On Thu, Mar 11, 2010 at 2:21 PM, Sekhar sek...@allurefx.com wrote:
  I'm using the Eclipse Google plugin, and every once in a while after a
  build I get the dreaded this class is not enhanced! errors for all
  my entities (even when I don't edit any of them). Any idea why this
  is? If I touch the files, they get built/enhanced again fine, but this
  is getting to be a real annoyance. I'd appreciate any pointers you can
  give!

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine for Java group.
  To post to this group, send email to
  google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B 
  unsubscr...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.



Re: [appengine-java] Objectify - Twig - approaches to persistence

2010-03-12 Thread John Patterson


On 12 Mar 2010, at 16:28, Jeff Schnitzer wrote:


Look at these graphs:

http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-get-latency
http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-query-latency

Notice that a get()'s average latency is 50ms and a query()'s average
latency is 500ms.  Last week the typical query was averaging
800-1000ms with frequent spikes into 1200ms or so.


You are increasing my suspicion that you have never worked with an  
application that queries large amounts of data.  If your queries are  
taking anywhere near 1000 ms then you must be doing something  
seriously wrong.


One of my apps query times are generally in the 200 ms range over 2  
million records.  A keys-only query can return in 50ms.


This is the time required to execute 9 parallel queries on geospatial  
data and OR merge them together.  Keep in mind that with Twig I could  
execute 90 parallel queries and expect the time to be about the same.




Deep down in the fiber of its being, BigTable is a key-value store.
It is very very efficient at doing batch gets.  It wants to do batch
gets all day long.  Queries require touching indexes maintained in
alternative tablets and comparatively, the performance sucks.


You are ignoring the fact that for many (most?) applications queries  
are essential.  I completely understand that your FaceBook app doesn't  
depend on them but assuming that other peoples apps also do not is  
just not helpful.



Why am I obsessed with batch gets?  Because they're essential for
making an application perform.  They're why there is such a thing as a
NoSQL movement in the first place.


Again, essential for your app.  Not mine and probably many other apps  
in which querying their own data is more important.  Batch gets are  
really only useful in apps that need to take a load of ids from an  
external source and do something with them.  Social network  
extension apps for example.


Just to reiterate - batch gets of external ids is a trivial feature  
that has always been planned to be a part of the new load command  
that will follow the pattern of the find and store commands.



* Fire off a batch job at your leisure to finish it off.


This partial update approach only works in cases where you are not  
adding a field that you will query on.  That needs to be an all-or- 
nothing batch job.


What is with your obsession with batch gets?  I understand they are  
central
in Objectify because you are always loading keys.  As I said  
already - even
though this is not as essential in Twig it will be added to a new  
load

command.


Batch gets are *the* core feature of NoSQL databases, including the
GAE datastore.


Querying is important.  You are ignoring a whole class of applications  
if you think that querying is not important.  I understand that your  
applications works with FaceBook and does a lot of lookups by  
external ids in a large dataset so to your mind batch get is the most  
important operation.  This is really not such a common scenario as you  
social network developers might think.


One of the applications I work on application has about 2 million  
records on which it needs to do geospatial queries sorted and  
filtered.  I guarantee you that there are many other applications that  
have different query needs so to focus only on batch gets is myopic.


It probably explains why you don't think that OR queries are so  
important.  They were one of the first things I tried on App Engine  
and one of the reasons Twig was written.  I would bet that most  
developers could not imagine working with an RDBMS that did not  
support OR and AND queries (on more than one property).  Twigs support  
for these saves time and reduces the complexity of the developers  
app.  With Objectify they are left on their own to re-invent the wheel  
every time.


The high-level design of Twigs commands means that ORs are supported  
now in the query API.  Objectifies low-level design could only help  
out by providing helper classes -  hardly user friendly or intuitive.   
The goal of Twigs design is to put these common solutions at the  
developers finger tips.  Yes there are more methods in the API, but  
they are well organised using the fluent style commands.


The command pattern used by Twig has the potential to add new high  
level functionality that Objectifies low-level query interface would  
need to rely on helper functions.  For example, supporting AND queries  
with more than one inequality filter is in development.  Just like the  
OR queries it will stream results, never keeping more than a small  
number in memory.


These are the types of common problems that take a lot of time to  
code.  Im not saying that this is impossible to code with Objectify -  
just that it is up to the developer to code these patterns again and  
again.  Re-inventing the wheel is one of the biggest wasters of a  

Re: [appengine-java] Objectify - Twig - approaches to persistence

2010-03-12 Thread John Patterson

On 13 Mar 2010, at 11:00, Jeff Schnitzer wrote:

On Fri, Mar 12, 2010 at 4:26 PM, John Patterson  
jdpatter...@gmail.com wrote:



It's hardly FUD to point out that every extra query counts.  In GAE,
you can measure the price (in $) of every single request.  In the
applications I have developed, it *matters* that you don't do multiple
queries to fetch excess data


The FUD I was referring to was you claim that uninitialised instances  
are magical.  They are just normal instances with no values set.  But  
I See below that you are coming to like the idea after all :)



For nontrivial applications, fetching a large object graph every time
you load an object just doesn't work.  This is why Hibernate added
proxies.  I'm glad you're sufficiently aware of this problem that
you're building in limits to activation, but I think you're nuts if
you think that these facilities will rarely be used.


I believe you claimed that Keys were not used all that often in  
Objectify.  I guess that would also make you nuts.  Activation would  
be used less than Keys because it can be done as a part of  
optimisation when needed.  I consider Objectifies use of Keys and  
manual loading of every relationship a premature optimisation.



Proxies are a completely different beast.  If bytecode manipulation  
were
used there are suddenly serialization problems to worry about.   
That is why

Twig uses pure plain POJOs with NO magic.  As simple as possible.


Proxies serve the exact same purpose as your uninitialized entities.


They have the same purpose (with some advantages) but different  
implications and are more complex



They allow the fetch process to halt, because in nontrivial
applications you cannot afford to load large object graphs every time
you fetch a single entity.  As a solution, proxies have advantages and
disadvantages - just like your uninitialized entities have advantages
and disadvantages.


That is why, as the docs say, a future release will include an option  
to use proxies for automatic activation and dirty detection.


For now the simple no magic approach works fine and has less gotchas.


FWIW, I myself prefer the uninitialized entity solution over proxies,
despite the - quite significant - danger.  Just remember that if
everyone was using Twig instead of Hibernate, all those



After you explained the concept of uninitialized entities (the brief
blurb in your docs really isn't enough), I actually rather like your
solution!  I might even implement something similar in Objectify.


Thanks for the recognition!  But I borrowed the concept from Db4o -  
just a shame that their implementation was completely useless for  
server apps (single threaded!).


Yes I admit that docs have so far come after features.  Twig contains  
a hell of a lot of features to save developers time and make their  
code cleaner.  Hopefully we'll get some other developers excited by  
these new features to join the effort to add documentation - and more  
features!!  Automatic activation and dirty detection are high on the  
list of non-trivial additions.



 But I really think you need to document the hell out of the issues
surrounding them.  It is very very easy to corrupt data.



I think that it is easier to corrupt data in Objectify because the  
same instance can be loaded into memory more than once at the same  
time.  Very easy to overwrite one with the other without realising.   
Twig guarantees that one entity will only have one instance in memory.



Sadly, this is where bytecode manipulation really would come in handy
- you can intercept data access on an uninitialized entity and throw
an exception.  It's too bad java dynamic proxies can't wrap concrete
classes.


Yes and too bad Guice's bytecode AOP doesn't support field access  
interceptors.


This is controllable in *very* fine detail by Activation settings.   
Any
class can have a default activation depth, any field can have an  
activation
depth and the datastore as a whole can set the depth for any  
individual
operation.  This gives *complete* control over what is loaded and  
when.


I wouldn't call it complete control.  It doesn't very gracefully
handle entities with multiple relationships - some queries you will
want to fetch some parts and not others.


Actually you can set activation per Class, Field or per datastore  
command.  Also the ActivationStrategy makes it possible to make *any*  
control you can think of in Java code.  But I don't think this will be  
hardly ever necessary.



However, the point is moot,
since you can (and IMNSHO probably should) always disable automatic
activation and refresh the graph manually.


Completely disagree.  IMNSHO it i best to make the framework work out- 
of-the box first and optimize later.  Although refreshing is handy you  
would end up in the same position as Objectify - all your code that  
uses your data models would need to reference the data layer (i.e. be  
ties to the platform)



Here's a bit of free 

Re: [appengine-java] Objectify - Twig - approaches to persistence

2010-03-12 Thread Jeff Schnitzer
On Fri, Mar 12, 2010 at 8:35 PM, John Patterson jdpatter...@gmail.com wrote:

 On 12 Mar 2010, at 16:28, Jeff Schnitzer wrote:

 Look at these graphs:

 http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-get-latency
 http://code.google.com/status/appengine/detail/datastore/2010/03/12#ae-trust-detail-datastore-query-latency

 Notice that a get()'s average latency is 50ms and a query()'s average
 latency is 500ms.  Last week the typical query was averaging
 800-1000ms with frequent spikes into 1200ms or so.

 You are increasing my suspicion that you have never worked with an
 application that queries large amounts of data.  If your queries are taking
 anywhere near 1000 ms then you must be doing something seriously wrong.
 One of my apps query times are generally in the 200 ms range over 2 million
 records.  A keys-only query can return in 50ms.

Are you debating the validity of google's statistics?  Or the loud
complaints posted to this mailing list last week?

Some queries will certainly return faster than others, and from what
I've read/watched, keys-only queries should have performance profiles
roughly similar to simple gets.  But there can be no doubt that real
queries are quite slow compared to simple gets.

But you're arguing with a straw man here.  I've never suggested that
queries are not useful.

However, you *have* suggested that batch gets aren't important.
Batch gets are really only useful in apps that need to take a load of
ids from an external source and do something with them.  That's
absolute rubbish.  A very large (and growing) number of applications
are being built on NoSQL databases that are effectively key-value
stores.  Cassandra, Tokyo Cabinet, HBase, Voldemort, and *dozens* of
other tools are being developed because they can do something that
relational systems can't:  get() and put() vast quantities of data
quickly.

There are a growing number of applications (largely defined by
staggeringly large user bases) in which the cost of maintaining
traditional indexes is not practical.  You aren't going to implement
Twitter or Facebook with a bunch of appengine queries!  But apparently
Cassandra works great.

 This is the time required to execute 9 parallel queries on geospatial data
 and OR merge them together.  Keep in mind that with Twig I could execute 90
 parallel queries and expect the time to be about the same.

You have the luxury of relatively static data, which colors your view
of the world.  I work with data that has a high churn rate, which
colors mine.

I have to ask you something though - would you need to do 9 parallel
queries if you were working with a datastore that has proper spatial
indexes?  Not that doing parallel queries isn't cool, but is it
actually necessary for your app?

I'm not doing spatial queries right now, but it's on the horizon.
I've done the research.  For my application, it's much easier and more
efficient to push my spatial queries off to a cluster of PostGIS
instances running elsewhere in the cloud.  It's also much, much
cheaper.

 * Fire off a batch job at your leisure to finish it off.

 This partial update approach only works in cases where you are not adding
 a field that you will query on.  That needs to be an all-or-nothing batch
 job.

Nonsense, this is totally dependent on the specific logic of your application.

Simple example:  You're adding a loginCount to your User entity, and
you want to add a query that selects out users that have logged in
more than N times.  No reason you can't start running those queries
right away.

You're trying to dismiss the utility of upgrading the dataset in-place
by saying that *some* application features require the dataset to be
completely transitioned before being enabled.  Ok, some do some don't.
 Your claim is still absurd.

 It probably explains why you don't think that OR queries are so important.

The reason OR queries aren't high on our priority list is because
nobody has been asking for them.  There doesn't even seem to be an
issue for it in GAE's issue tracker - or if there is, it's *pages*
down the list of priorities.

 They were one of the first things I tried on App Engine and one of the
 reasons Twig was written.  I would bet that most developers could not
 imagine working with an RDBMS that did not support OR and AND queries (on
 more than one property).  Twigs support for these saves time and reduces the
 complexity of the developers app.  With Objectify they are left on their own
 to re-invent the wheel every time.

Our conceptual model of the datastore is not an RDBMS.  It's a
key-value store that also allows limited queryability.  If you really
want an RDBMS, I'm sure the Cloud2db guys will be happy to chime in
again.

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send 

Re: [google-appengine] Re: DNS providers that support wildcard CNAME's?

2010-03-12 Thread Waleed Abdulla
Thanks Albert. I thought that might be what you were referring to. It's a
different case, though. I do need to serve the app from the naked domain (
exmaple.com) without redirecting it to www.example.com. As far as I can
tell, it's not possible now. So I'm doing a dirty hack of serving the page
from a dedicated server which pulls it's data from the app engine. I really
want to not have to do that, though. I hope that feature gets added soon.

Waleed






On Thu, Mar 11, 2010 at 2:08 AM, Albert albertpa...@gmail.com wrote:

 Hi Waleed!

 I don't think I did anything special really. I decided to take a look
 into my configurations, and I realized that I really didn't do
 anything at all. I set that app about a year ago, and I haven't
 touched it since, so I had to look into my configurations again. And
 after some testing, I think my Domain Name Provider (name.com)
 automatically redirects traffic from example.com to
 www.example.com. So I think that's the reason why it works. I don't
 have anything configured for my base domain name. This effect may not
 be the same across different Domain Name providers.

 By the way, I don't have any of my domain names pointing to any IP
 Address.

 I have no problem since it's fine with me if the same app is served
 through www and base domain.

 If I'm not mistaken, you probably want to serve one app on the www
 and another on the base domain. In that case, I don't know yet how to
 make that work.

 I hope this helps.

 Enjoy!

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Is it possible to change a datastore object key name after it was created?

2010-03-12 Thread Shedokan
Ok, thanks, I was thinking of that as my plan b.

On 11 מרץ, 21:58, Eli Jones eli.jo...@gmail.com wrote:
 You cannot change the key_name for an entity..

 At best, you could put a new entity into the datastore using the new
 key_name and the old entities properties.. and just delete the old entity.



 On Thu, Mar 11, 2010 at 2:55 PM, Shedokan shedok...@gmail.com wrote:
  In my app I create an object and put it into the datastore with a
  custom key_name based on other properties of the object for getting it
  from the datastore faster.

  what happens if I want to change later on the key name?

  THanks.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] Exception: A maximum of 16 composites can be applied in a single request.

2010-03-12 Thread Nick Johnson (Google)
Hi Patrick,

The request in question is per API call. You can make multiple calls, with
up to 16 images composited in each.

-Nick

On Thu, Mar 11, 2010 at 7:31 PM, Patrick Twohig
patr...@namazustudios.comwrote:

 Hi,

 I'm getting this error when trying to take several layers and build a
 single composite of them.  I was curious if a request meant a single call
 to the graphics API, or if that meant per HTTP request.  If it it the latter
 case, would it be possible to pay for extra graphics API usage?

 Thanks,
 Patrick.



 --
 Patrick H. Twohig.

 Namazu Studios
 P.O. Box 34161
 San Diego, CA 92163-4161

 Office: 619.862.2890 x100
 Cell: 619.453.5075
 Twitter: @svm_invictvs
 IRC: svm_invic...@irc.freenode.net ##java, #android-dev, #iphonedev,
 #appengine

 http://www.namazustudios.com/

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: object = pm.detachCopy(object);

2010-03-12 Thread Prashant Gupta
anyone ?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Do the terms of use allow serving library files?

2010-03-12 Thread Riley
I could do that, but these are static files and retrieving them from
the datastore will add unnecessary overhead, wouldn't it?  I'm under
the impression that since these are files that my server code will
never need, GAE will physically distribute them in a way that takes
advantage of that.

On Mar 10, 6:51 pm, Barry Hunter barrybhun...@googlemail.com wrote:
 Why dont youusethe datastore to save (and then serve with a simple
 script) Library L?

 Then you dont have to worry about file limits.

 On 10 March 2010 22:26, Riley rileyl...@gmail.com wrote:



  Thanks for the response, but I'm not sure I've been clear.  I want to
  create App X on GAE, but I am hitting the 3k file limit when Iuse
  Library L.  I think to myself, hmm, I could create App Y, whose sole
  purpose is to distribute the static files needed in L.  In fact, many
  people coulduseY to serve L to their own apps.  If it stays under 5M
  requests a month (free from Google) then great.  If not, I can charge
  a fee to pay the Google fees.  In that way, Y is its own app and not
  only part of X.

  It's clear that I could safely write a file-distribution app within
  thetermsofuse.  I just want to be sure that if I write a file-
  distribution app that only distributes files to be used in other GAE
  apps I won't cross some sort of specificity line into a violation of
  the TOS.

  Thanks!

  On Mar 10, 1:36 pm, Ikai L (Google) ika...@google.com wrote:
  If you're not using the application for this purpose, you should be
  fine. It just sounds like you're writing an application that provides
  an API for uploading files for distribution. I don't see any problem
  with this.

  On Wed, Mar 10, 2010 at 9:59 AM, Riley rileyl...@gmail.com wrote:
   I would like to create an app whose primary function is to serve
   static files to other services on the web.  For example, someone might
  usemy app to reduce the load on their server, or to decrease the
   number of files they have to manage in their app.  Someone using GAE
   mightusemy app to avoid using up their 3000 file quota, etc.

   Section 4.4 of the GAEtermsofusereads:

   
   4.4. You may not develop multiple Applications to simulate or act as a
   single Application or otherwise access the Service in a manner
   intended to avoid incurring fees.
   

   Does this rule out my idea?

   If this is not the proper forum for a legal question like this, who
   can I ask for a definitive answer?

   Thanks for any help!

   --
   You received this message because you are subscribed to the Google 
   Groups Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

  --
  Ikai Lan
  Developer Programs Engineer, Google App 
  Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Increase number of apps

2010-03-12 Thread Marcos Placona
Thanks a lot for that.

Have filled up the form.

Cheers

On Mar 11, 7:08 pm, Ikai L (Google) ika...@google.com wrote:
 Can you fill out a request here on the billing support page?

 http://code.google.com/support/bin/request.py?contact_type=AppEngineB...

 Include the email address you are using to create applications. Note
 that you're also allowed to delete applications.

 On Thu, Mar 11, 2010 at 5:48 AM, MarcosPlacona





 marcos.plac...@gmail.com wrote:
  Hi, I developed various apps for the app engine, and now am finding
  myself in need of some more, but I only seem to be allowed to create
  up to 10 apps.

  Is there anyway I can get these numbers increased?

  Thanks

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App 
 Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Invitation emails not sent durng the last days

2010-03-12 Thread dflorey
Hi,
I've experienced that invitation emails to invite developers to a App
Engine application is not working properly in the last days.
Only few invitation emails get sent, many get lost. It all worked fine
since ~4-5 days ago.
Anyone else?

Daniel

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] Re: Do the terms of use allow serving library files?

2010-03-12 Thread Nick Johnson (Google)
Hi Riley,

On Wed, Mar 10, 2010 at 10:26 PM, Riley rileyl...@gmail.com wrote:

 Thanks for the response, but I'm not sure I've been clear.  I want to
 create App X on GAE, but I am hitting the 3k file limit when I use
 Library L.  I think to myself, hmm, I could create App Y, whose sole
 purpose is to distribute the static files needed in L.  In fact, many
 people could use Y to serve L to their own apps.  If it stays under 5M
 requests a month (free from Google) then great.  If not, I can charge
 a fee to pay the Google fees.  In that way, Y is its own app and not
 only part of X.

 It's clear that I could safely write a file-distribution app within
 the terms of use.  I just want to be sure that if I write a file-
 distribution app that only distributes files to be used in other GAE
 apps I won't cross some sort of specificity line into a violation of
 the TOS.


This sounds perfectly fine, since your goal is not to avoid paying for
usage.

-Nick Johnson



 Thanks!


 On Mar 10, 1:36 pm, Ikai L (Google) ika...@google.com wrote:
  If you're not using the application for this purpose, you should be
  fine. It just sounds like you're writing an application that provides
  an API for uploading files for distribution. I don't see any problem
  with this.
 
 
 
 
 
  On Wed, Mar 10, 2010 at 9:59 AM, Riley rileyl...@gmail.com wrote:
   I would like to create an app whose primary function is to serve
   static files to other services on the web.  For example, someone might
   use my app to reduce the load on their server, or to decrease the
   number of files they have to manage in their app.  Someone using GAE
   might use my app to avoid using up their 3000 file quota, etc.
 
   Section 4.4 of the GAE terms of use reads:
 
   
   4.4. You may not develop multiple Applications to simulate or act as a
   single Application or otherwise access the Service in a manner
   intended to avoid incurring fees.
   
 
   Does this rule out my idea?
 
   If this is not the proper forum for a legal question like this, who
   can I ask for a definitive answer?
 
   Thanks for any help!
 
   --
   You received this message because you are subscribed to the Google
 Groups Google App Engine group.
   To post to this group, send email to google-appengine@googlegroups.com
 .
   To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
   For more options, visit this group athttp://
 groups.google.com/group/google-appengine?hl=en.
 
  --
  Ikai Lan
  Developer Programs Engineer, Google App Enginehttp://
 googleappengine.blogspot.com|http://twitter.com/app_engine

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Snow Leopard fix for - ImportError: cannot import name os_compat

2010-03-12 Thread Dr. Chuck
This morning I was using App Engine to record a video on a fresh, non-
admin user and got the following error:

*** Running dev_appserver with the following flags:
--admin_console_server= --port=8080
Python command: /usr/bin/python2.6
Traceback (most recent call last):
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
dev_appserver.py, line 68, in
run_file(__file__, globals())
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine
/dev_appserver.py, line 64, in run_file
execfile(script_path, globals_)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/
appengine/tools/dev_appserver_main.py, line 66, in
from google.appengine.tools import os_compat
ImportError: cannot import name os_compat

If you look through the forums there is a lot of talk about Python 2.5
or Python 2.6 and suggestions to go back to Python 2.5 - that seems
not to be the problem at all because it fails in 2.5 as well with the
same error.

The problem really is caused when a non-administrator user tries to
run the App Engine Launcher. The fix is to simply make the person an
administrative user.   And then it works fine on 2.6.

And here is the even stranger thing - once you make the use an Allow
to administer this computer user and start App Engine once as an the
user with admin powers and then log out and take a way admin powers
and then log back in - then it works! Crazy.

Charles Severance
http://www.dr-chuck.com/
http://oreilly.com/catalog/9780596800697/

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] @staticmethod @clasmethod

2010-03-12 Thread B
Why are these used prior to defining a method for a model?

@classmethod
@staticmethod

What is the difference between them? Are there others I should be
aware of?
Must I use them?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] @staticmethod @clasmethod

2010-03-12 Thread Ross M Karchner
They're called decorators, and here's a pretty good overview:

http://www.artima.com/weblogs/viewpost.jsp?thread=240808

Here's the definition of classmethod:
http://docs.python.org/library/functions.html#classmethod

and staticmethod:
http://docs.python.org/library/functions.html#staticmethod

On Fri, Mar 12, 2010 at 3:59 AM, B boris@gmail.com wrote:
 Why are these used prior to defining a method for a model?

 @classmethod
 @staticmethod

 What is the difference between them? Are there others I should be
 aware of?
 Must I use them?

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Outage: ClientLogin API from GAE

2010-03-12 Thread Jan Z
Hi folks.  Just FYI, the ClientLogin API is failing all requests from
GAE with a 1301: EntityDoesNotExist error.

This is happening across multiple domains, for the last hour +

Is this being looked at?

J

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] Outage: ClientLogin API from GAE

2010-03-12 Thread Ikai L (Google)
(Sometimes I forget that I should Google Search first): this API?

http://code.google.com/apis/accounts/docs/AuthForInstalledApps.html



On Fri, Mar 12, 2010 at 3:49 PM, Ikai L (Google) ika...@google.com wrote:
 Are you referring to the UserService? I just tested this and it works.
 Which ClientLogin API are you referring to?

 On Fri, Mar 12, 2010 at 3:00 PM, Jan Z jan.zawad...@gmail.com wrote:
 Hi folks.  Just FYI, the ClientLogin API is failing all requests from
 GAE with a 1301: EntityDoesNotExist error.

 This is happening across multiple domains, for the last hour +

 Is this being looked at?

 J

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.





 --
 Ikai Lan
 Developer Programs Engineer, Google App Engine
 http://googleappengine.blogspot.com | http://twitter.com/app_engine




-- 
Ikai Lan
Developer Programs Engineer, Google App Engine
http://googleappengine.blogspot.com | http://twitter.com/app_engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Task Queue issues

2010-03-12 Thread Saurabh
Hello,

I'm noticing issues with my task queues: the system seems to think
that it has enqueued tasks, but the URLs don't show up in the debug
logs and the actions that are supposed to be executed when the task
runs are not being run. This same code was working yesterday and is
not working today (March 12, 2010) so I'm curious to know if others
are experiencing any task queue issues as well.

Can Google provide some clarity on this?
Thanks,
Saurabh

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Do the terms of use allow serving library files?

2010-03-12 Thread Tim Hoffman
Hi


On Mar 12, 10:26 pm, Riley rileyl...@gmail.com wrote:
 I could do that, but these are static files and retrieving them from
 the datastore will add unnecessary overhead, wouldn't it?

Static files are quicker however if your files are accessed a lot then
caching the result
of the fetch from the datastore in memcache will speed the process up.

The big advantage is you can store millions of files in the datastore
and
only need to update single records when you need to change/add a file.

 I'm under
 the impression that since these are files that my server code will
 never need, GAE will physically distribute them in a way that takes
 advantage of that.

Thats a big assumption, I see no evidence of such.  You will find the
static files will be stored in a something like the datastore
under the hood anyway (you just don't get to see it that way, this
access looks like a file system access anyway).

If you want a distributed CDN then you would be better sticking those
files into S3, and have amazon distribute them nearer the consumer.

T

 On Mar 10, 6:51 pm, Barry Hunter barrybhun...@googlemail.com wrote:



  Why dont youusethe datastore to save (and then serve with a simple
  script) Library L?

  Then you dont have to worry about file limits.

  On 10 March 2010 22:26, Riley rileyl...@gmail.com wrote:

   Thanks for the response, but I'm not sure I've been clear.  I want to
   create App X on GAE, but I am hitting the 3k file limit when Iuse
   Library L.  I think to myself, hmm, I could create App Y, whose sole
   purpose is to distribute the static files needed in L.  In fact, many
   people coulduseY to serve L to their own apps.  If it stays under 5M
   requests a month (free from Google) then great.  If not, I can charge
   a fee to pay the Google fees.  In that way, Y is its own app and not
   only part of X.

   It's clear that I could safely write a file-distribution app within
   thetermsofuse.  I just want to be sure that if I write a file-
   distribution app that only distributes files to be used in other GAE
   apps I won't cross some sort of specificity line into a violation of
   the TOS.

   Thanks!

   On Mar 10, 1:36 pm, Ikai L (Google) ika...@google.com wrote:
   If you're not using the application for this purpose, you should be
   fine. It just sounds like you're writing an application that provides
   an API for uploading files for distribution. I don't see any problem
   with this.

   On Wed, Mar 10, 2010 at 9:59 AM, Riley rileyl...@gmail.com wrote:
I would like to create an app whose primary function is to serve
static files to other services on the web.  For example, someone might
   usemy app to reduce the load on their server, or to decrease the
number of files they have to manage in their app.  Someone using GAE
mightusemy app to avoid using up their 3000 file quota, etc.

Section 4.4 of the GAEtermsofusereads:


4.4. You may not develop multiple Applications to simulate or act as a
single Application or otherwise access the Service in a manner
intended to avoid incurring fees.


Does this rule out my idea?

If this is not the proper forum for a legal question like this, who
can I ask for a definitive answer?

Thanks for any help!

--
You received this message because you are subscribed to the Google 
Groups Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/google-appengine?hl=en.

   --
   Ikai Lan
   Developer Programs Engineer, Google App 
   Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine

   --
   You received this message because you are subscribed to the Google Groups 
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Outage: ClientLogin API from GAE

2010-03-12 Thread Jan Z
Hi Ikai - yes.  It seems to have recovered now though!

Phew.  Pretty critical service.

Jan



On Mar 13, 12:49 pm, Ikai L (Google) ika...@google.com wrote:
 Are you referring to the UserService? I just tested this and it works.
 Which ClientLogin API are you referring to?





 On Fri, Mar 12, 2010 at 3:00 PM, Jan Z jan.zawad...@gmail.com wrote:
  Hi folks.  Just FYI, the ClientLogin API is failing all requests from
  GAE with a 1301: EntityDoesNotExist error.

  This is happening across multiple domains, for the last hour +

  Is this being looked at?

  J

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

 --
 Ikai Lan
 Developer Programs Engineer, Google App 
 Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] GAE/J Performance (Slowness)

2010-03-12 Thread Patrick Twohig
Hi,

I asked once before, but I'm getting some painfully slow requests in my
app.  Even on queries that I am certain only fetch a few entities, (5-10 max
under absolutely worst case circumstances).  I also take advantage of memory
caching wherever possible, and I have confirmed (via logging) that I'm
pulling from the cache.  Some requests are taking upwards of 2-3 seconds.
And given that my app right now is mostly a web service and not running in a
browser, there's quite a few things that are perplexing me.  SO I have a few
questions:

   - I'm running JAX-RS (Jersey), a little Groovy, low level API (no JDO or
   JPA), and Guice (yep, I'm a Google fanboy). I have identified where Groovy
   is being slow, and came up with a workaround for that, so I'm not too
   concerned.
   - What's a reasonable number of entities I should fetch from the
   datastore per request?  I currently fetch at least one per request to
   validate a user's login credentials, so no matter what I'm fetching at least
   one per request.  Adding up the numbers in my head, I don't think that
   - Even with aggressive caching, (all of my data objects get put in
   memcache), it still behaves rather slowly?  What is the penalty speed-wise
   for using a transaction?  I can't use caching with a transaction obviously
   because I can't be guaranteed any consistency of data.  If I start a
   transaction and and commit it or roll it back and perform no operations in
   it, what is the penalty?  I'm certain this happens in my app, albeit very
   infrequently.
   - What's up with queries?  They just tend to be really really slow.
   - I know loading requests are a problem, but I'm not hitting many of
   those according to the logs, so I can rule those out as being a contributing
   factor (unless there's something I don't know).  I also don't use JSPs, most
   of it is static.
   - How much of a performance difference does a bulk fetch/put cost versus
   a bunch of individual ones?  I haven't seen much of a difference, but maybe
   I'm doing it wrong.
   - There are zero use cases in my application where more than two users
   write to the same entity group, except for a counter for which I use
   sharding offloaded to the task queue.  Is there a penalty for many users
   reading from a single entity group at the same time?


Thanks,

-- 
Patrick H. Twohig.

Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: GAE/J Performance (Slowness)

2010-03-12 Thread Tim Hoffman
Hi

Have you checked that startup times on a lightly used system are not
inflating your response times ?

T

On Mar 13, 3:05 pm, Patrick Twohig patr...@namazustudios.com wrote:
 Hi,

 I asked once before, but I'm getting some painfully slow requests in my
 app.  Even on queries that I am certain only fetch a few entities, (5-10 max
 under absolutely worst case circumstances).  I also take advantage of memory
 caching wherever possible, and I have confirmed (via logging) that I'm
 pulling from the cache.  Some requests are taking upwards of 2-3 seconds.
 And given that my app right now is mostly a web service and not running in a
 browser, there's quite a few things that are perplexing me.  SO I have a few
 questions:

    - I'm running JAX-RS (Jersey), a little Groovy, low level API (no JDO or
    JPA), and Guice (yep, I'm a Google fanboy). I have identified where Groovy
    is being slow, and came up with a workaround for that, so I'm not too
    concerned.
    - What's a reasonable number of entities I should fetch from the
    datastore per request?  I currently fetch at least one per request to
    validate a user's login credentials, so no matter what I'm fetching at 
 least
    one per request.  Adding up the numbers in my head, I don't think that
    - Even with aggressive caching, (all of my data objects get put in
    memcache), it still behaves rather slowly?  What is the penalty speed-wise
    for using a transaction?  I can't use caching with a transaction obviously
    because I can't be guaranteed any consistency of data.  If I start a
    transaction and and commit it or roll it back and perform no operations in
    it, what is the penalty?  I'm certain this happens in my app, albeit very
    infrequently.
    - What's up with queries?  They just tend to be really really slow.
    - I know loading requests are a problem, but I'm not hitting many of
    those according to the logs, so I can rule those out as being a 
 contributing
    factor (unless there's something I don't know).  I also don't use JSPs, 
 most
    of it is static.
    - How much of a performance difference does a bulk fetch/put cost versus
    a bunch of individual ones?  I haven't seen much of a difference, but maybe
    I'm doing it wrong.
    - There are zero use cases in my application where more than two users
    write to the same entity group, except for a counter for which I use
    sharding offloaded to the task queue.  Is there a penalty for many users
    reading from a single entity group at the same time?

 Thanks,

 --
 Patrick H. Twohig.

 Namazu Studios
 P.O. Box 34161
 San Diego, CA 92163-4161

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.