date:20100318

Something you may want to consider is that you are placing yourself
within the limitations of entity groups by sticking children under
parent keys. You may want to consider a model where the parent key is
simply a field in the child and then run a simple query testing that
the parentKey field is equal to the one you're looking for. This
makes queries easier and prevents entity group lock-in.

Cheers!

On Mar 18, 8:55 am, tempy fay...@gmail.com wrote:
 I want to retrieve all entities that are children of one other
 particular entity, by checking if the parent-key property of the
 child's key matches the parent key.  I have a reference to the parent
 entity but I want to avoid loading all of its children (as there may
 be many children, but I only need a few).  Thus I want a query that
 looks something like this:

         query = pm.newQuery(select from  + ChildClass.class.getName() +
  where :parentID.contains(ChildIDProperty.ParentID) 
 SomeOtherProperty    + filterString);

 But I'm not sure how to exactly address the parent key property of a
 key in a query.

 Thanks,
 Mike

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Why do custom indexes require single-property indexes?

Not official but been doing this for a while.

Your custom index is most likely build from the query. So, when you
do

Query query = new Query(Thing);
query.addFilter(foo, FilterOperator.EQUAL, fooValue);
query.addSort(bar, SortDirection.DESCENDING);

That is what builds your custom index.

However, when you setUnindexedProperty here

Entity ent = new Entity(Thing);
ent.setUnindexedProperty(foo, fooValue);
ent.setUnindexedProperty(bar, 123L);

You are not generating any index entries.

So the issue isn't that adding custom indexes after-the-fact [is]
really, really painful but that you are not generating any indexes
for the datastore to run the queries against when you use
setUnindexedProperty(). In other words, when you execute a query, it
checks the index to give you results. But you marked your data as
don't index me, so there is nothing for query to work with, as far
as it is concerned, there's nothing in the datastore.

Cheers!


On Mar 16, 4:07 pm, Jeff Schnitzer j...@infohazard.org wrote:
 On Mon, Mar 15, 2010 at 11:04 PM, John Patterson jdpatter...@gmail.com 
 wrote:

  On 16 Mar 2010, at 12:25, Jeff Schnitzer wrote:

  I'm puzzled by the behavior of custom indexes.  I have a simple test
  case below, a simple equality filter on one property combined with a
  descending sort on another property.  If I set the properties with
  setUnindexedProperty(), the query fails to find the result.  If I set
  the properties with setProperty(), it does.

  I also wondered why - I assume that the custom index build reads the single
  property indexes directly which must be more efficient than reading the
  Entities table.

 I guess that is possible, but seems like a poor design decision.  It
 makes adding custom indexes after-the-fact really, really painful.

 Can someone official chime in on this?  Is it intended behavior, or
 should we file an issue against it?  The documentation doesn't say
 much on the subject, and all the conceptual explanation of queries
 suggests that these extra single-property indexes will be unused.

 Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: How to query for parent key property of an entity's key?

2010-03-18 Thread Ugorji

Use an ancestor-is query. This will find all entities which have a
given ancestor as parent. It even works in a kind-less query,
returning different kinds of entities under that parent. It's also
quite efficient (especially from a pricing/CPU cost perspective) since
theoretically, all this can be done on a single database node.

I'm not sure how to do ancestor-is query with JDO (I use my own type-
safe wrapper over low-level API), but you could try something like:

select from ... where ANCESTOR=... (like __key__, ANCESTOR seems to be
a reserved name in app engine)

On Mar 18, 7:56 am, tempy fay...@gmail.com wrote:
 That's a good point, but in this case the resulting entity group
 arrangement is intentional.  Lock-in isn't a worry, I can confidently
 say that these entity-group arrangements will last as long as the
 application does.

 Though, as you point out, I can make a parentKey field on the child,
 but since this parent key will also have to appear within the child's
 key, having such a field is redundant and I would rather avoid it.

 On Mar 18, 3:51 pm, Tristan tristan.slomin...@gmail.com wrote:



  Something you may want to consider is that you are placing yourself
  within the limitations of entity groups by sticking children under
  parent keys. You may want to consider a model where the parent key is
  simply a field in the child and then run a simple query testing that
  the parentKey field is equal to the one you're looking for. This
  makes queries easier and prevents entity group lock-in.

  Cheers!

  On Mar 18, 8:55 am, tempy fay...@gmail.com wrote:

   I want to retrieve all entities that are children of one other
   particular entity, by checking if the parent-key property of the
   child's key matches the parent key.  I have a reference to the parent
   entity but I want to avoid loading all of its children (as there may
   be many children, but I only need a few).  Thus I want a query that
   looks something like this:

           query = pm.newQuery(select from  + ChildClass.class.getName() +
where :parentID.contains(ChildIDProperty.ParentID) 
   SomeOtherProperty    + filterString);

   But I'm not sure how to exactly address the parent key property of a
   key in a query.

   Thanks,
   Mike

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Why do custom indexes require single-property indexes?

Ugorji,

I don't know if they will support re-indexing things based on new
index definition, however, I did put in an issue back in August 2009
to be able to create a custom index term on the fly. See if this would
partially solve the issue (again, it doesn't address reindexing
already put entities).

http://code.google.com/p/googleappengine/issues/detail?id=1935

Cheers!

On Mar 18, 10:37 am, Ugorji ugo...@gmail.com wrote:
 +1 big-time on this. With the restriction that every index'able
 property must be indexed at put-time when put into the datastore
 (using setProperty as opposed to setUnindexedProperty), it has 2 MAJOR
 disadvantages with far-reaching repercussions:

 - you cannot index entities after they have been put (even with a
 custom index). You will have to re-write the entity to index it.
 - Each put is 4X more expensive in latency/clock-time (potentially),
 CPU-cost and storage. This seems un-necessary, especially if a single
 custom index will suffice all your querying needs.

 I was really hoping to use custom indexes to buy major performance
 savings (in latency, CPU-cost and storage), and also be able to re-
 index every entity after the fact. But this restriction is heavy
 (preventing direct solution to a problem everyone may end up facing),
 and very expensive (costing us big-time). Can one of the google guys
 respond to this, before we file a bug?

 Thanks.

 On Mar 16, 2:07 pm, Jeff Schnitzer j...@infohazard.org wrote:



  On Mon, Mar 15, 2010 at 11:04 PM, John Patterson jdpatter...@gmail.com 
  wrote:

   On 16 Mar 2010, at 12:25, Jeff Schnitzer wrote:

   I'm puzzled by the behavior of custom indexes.  I have a simple test
   case below, a simple equality filter on one property combined with a
   descending sort on another property.  If I set the properties with
   setUnindexedProperty(), the query fails to find the result.  If I set
   the properties with setProperty(), it does.

   I also wondered why - I assume that the custom index build reads the 
   single
   property indexes directly which must be more efficient than reading the
   Entities table.

  I guess that is possible, but seems like a poor design decision.  It
  makes adding custom indexes after-the-fact really, really painful.

  Can someone official chime in on this?  Is it intended behavior, or
  should we file an issue against it?  The documentation doesn't say
  much on the subject, and all the conceptual explanation of queries
  suggests that these extra single-property indexes will be unused.

  Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Why do custom indexes require single-property indexes?

Jeff,

Sounds to me like you're correct in the not only don't set single-
property indexes, but also don't include the property in any custom
indexes interpretation of setUnindexedProperty. Can you post the link
to the continuation post? I'm curious what Googlers have to say about
it.

Tristan



On Mar 18, 11:13 am, Jeff Schnitzer j...@infohazard.org wrote:
 This doesn't make sense to me.

 Every scrap of documentation I've found says that GAE queries only
 follow a single index (the one exception being zigzag merges, which
 don't apply here).  This means that to answer my query.filter(foo,
 fooValue1).sort(-bar), there must be an index that contains the
 foo and bar data sorted appropriately, no?  Ie:

 /Thing/foo:fooValue1/bar:bar9/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar7/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar7/[thekeyvalue]

 To satisfy this query, GAE should start following this custom index
 and that's pretty much it.  There's no reason for it to touch the
 single-property indexes (foo ASC, foo DESC, bar ASC, and bar DESC).

 ...and in my test, if I remove the custom index from
 datastore-indexes.xml, it doesn't work.  But also if I use
 setUnindexedProperty, it doesn't work.

 It's like setUnindexedProperty is being interpreted as not only don't
 set single-property indexes, but also don't include the property in
 any custom indexes.  This is counterintuitive - if I wanted the index
 not to be built, I can just remove the index.

 I realize now that perhaps I posted this to the wrong mailing list.
 The guys who created the I/O videos about the datastore seem to be
 python fans, so I'll retry my original post on the google-appengine
 list.

 Jeff



 On Thu, Mar 18, 2010 at 8:00 AM, Tristan tristan.slomin...@gmail.com wrote:
  Not official but been doing this for a while.

  Your custom index is most likely build from the query. So, when you
  do

         Query query = new Query(Thing);
         query.addFilter(foo, FilterOperator.EQUAL, fooValue);
         query.addSort(bar, SortDirection.DESCENDING);

  That is what builds your custom index.

  However, when you setUnindexedProperty here

         Entity ent = new Entity(Thing);
         ent.setUnindexedProperty(foo, fooValue);
         ent.setUnindexedProperty(bar, 123L);

  You are not generating any index entries.

  So the issue isn't that adding custom indexes after-the-fact [is]
  really, really painful but that you are not generating any indexes
  for the datastore to run the queries against when you use
  setUnindexedProperty(). In other words, when you execute a query, it
  checks the index to give you results. But you marked your data as
  don't index me, so there is nothing for query to work with, as far
  as it is concerned, there's nothing in the datastore.

  Cheers!

  On Mar 16, 4:07 pm, Jeff Schnitzer j...@infohazard.org wrote:
  On Mon, Mar 15, 2010 at 11:04 PM, John Patterson jdpatter...@gmail.com 
  wrote:

   On 16 Mar 2010, at 12:25, Jeff Schnitzer wrote:

   I'm puzzled by the behavior of custom indexes.  I have a simple test
   case below, a simple equality filter on one property combined with a
   descending sort on another property.  If I set the properties with
   setUnindexedProperty(), the query fails to find the result.  If I set
   the properties with setProperty(), it does.

   I also wondered why - I assume that the custom index build reads the 
   single
   property indexes directly which must be more efficient than reading the
   Entities table.

  I guess that is possible, but seems like a poor design decision.  It
  makes adding custom indexes after-the-fact really, really painful.

  Can someone official chime in on this?  Is it intended behavior, or
  should we file an issue against it?  The documentation doesn't say
  much on the subject, and all the conceptual explanation of queries
  suggests that these extra single-property indexes will be unused.

  Jeff

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine for Java group.
  To post to this group, send email to google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine-java+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine-java?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Why do custom indexes require single-property indexes?

Here's a link:
http://groups.google.com/group/google-appengine/browse_thread/thread/2f6aa695a80fd5de

Jeff

On Thu, Mar 18, 2010 at 9:22 AM, Tristan tristan.slomin...@gmail.com wrote:
 Jeff,

 Sounds to me like you're correct in the not only don't set single-
 property indexes, but also don't include the property in any custom
 indexes interpretation of setUnindexedProperty. Can you post the link
 to the continuation post? I'm curious what Googlers have to say about
 it.

 Tristan



 On Mar 18, 11:13 am, Jeff Schnitzer j...@infohazard.org wrote:
 This doesn't make sense to me.

 Every scrap of documentation I've found says that GAE queries only
 follow a single index (the one exception being zigzag merges, which
 don't apply here).  This means that to answer my query.filter(foo,
 fooValue1).sort(-bar), there must be an index that contains the
 foo and bar data sorted appropriately, no?  Ie:

 /Thing/foo:fooValue1/bar:bar9/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar7/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar7/[thekeyvalue]

 To satisfy this query, GAE should start following this custom index
 and that's pretty much it.  There's no reason for it to touch the
 single-property indexes (foo ASC, foo DESC, bar ASC, and bar DESC).

 ...and in my test, if I remove the custom index from
 datastore-indexes.xml, it doesn't work.  But also if I use
 setUnindexedProperty, it doesn't work.

 It's like setUnindexedProperty is being interpreted as not only don't
 set single-property indexes, but also don't include the property in
 any custom indexes.  This is counterintuitive - if I wanted the index
 not to be built, I can just remove the index.

 I realize now that perhaps I posted this to the wrong mailing list.
 The guys who created the I/O videos about the datastore seem to be
 python fans, so I'll retry my original post on the google-appengine
 list.

 Jeff



 On Thu, Mar 18, 2010 at 8:00 AM, Tristan tristan.slomin...@gmail.com wrote:
  Not official but been doing this for a while.

  Your custom index is most likely build from the query. So, when you
  do

         Query query = new Query(Thing);
         query.addFilter(foo, FilterOperator.EQUAL, fooValue);
         query.addSort(bar, SortDirection.DESCENDING);

  That is what builds your custom index.

  However, when you setUnindexedProperty here

         Entity ent = new Entity(Thing);
         ent.setUnindexedProperty(foo, fooValue);
         ent.setUnindexedProperty(bar, 123L);

  You are not generating any index entries.

  So the issue isn't that adding custom indexes after-the-fact [is]
  really, really painful but that you are not generating any indexes
  for the datastore to run the queries against when you use
  setUnindexedProperty(). In other words, when you execute a query, it
  checks the index to give you results. But you marked your data as
  don't index me, so there is nothing for query to work with, as far
  as it is concerned, there's nothing in the datastore.

  Cheers!

  On Mar 16, 4:07 pm, Jeff Schnitzer j...@infohazard.org wrote:
  On Mon, Mar 15, 2010 at 11:04 PM, John Patterson jdpatter...@gmail.com 
  wrote:

   On 16 Mar 2010, at 12:25, Jeff Schnitzer wrote:

   I'm puzzled by the behavior of custom indexes.  I have a simple test
   case below, a simple equality filter on one property combined with a
   descending sort on another property.  If I set the properties with
   setUnindexedProperty(), the query fails to find the result.  If I set
   the properties with setProperty(), it does.

   I also wondered why - I assume that the custom index build reads the 
   single
   property indexes directly which must be more efficient than reading the
   Entities table.

  I guess that is possible, but seems like a poor design decision.  It
  makes adding custom indexes after-the-fact really, really painful.

  Can someone official chime in on this?  Is it intended behavior, or
  should we file an issue against it?  The documentation doesn't say
  much on the subject, and all the conceptual explanation of queries
  suggests that these extra single-property indexes will be unused.

  Jeff

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine for Java group.
  To post to this group, send email to 
  google-appengine-j...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine-java+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine-java?hl=en.

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine for Java group.
 To post to this group, send email to google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine-java+unsubscr...@googlegroups.com.
 For more options, visit this group at

Re: [appengine-java] Re: Why do custom indexes require single-property indexes?

2010-03-18 Thread Tristan Slominski

Thanks :)

On Thu, Mar 18, 2010 at 11:55 AM, Jeff Schnitzer j...@infohazard.orgwrote:

Here's a link:

http://groups.google.com/group/google-appengine/browse_thread/thread/2f6aa695a80fd5de

Jeff

On Thu, Mar 18, 2010 at 9:22 AM, Tristan tristan.slomin...@gmail.com
wrote:
Jeff,

Sounds to me like you're correct in the not only don't set single-
property indexes, but also don't include the property in any custom
indexes interpretation of setUnindexedProperty. Can you post the link
to the continuation post? I'm curious what Googlers have to say about
it.

Tristan

On Mar 18, 11:13 am, Jeff Schnitzer j...@infohazard.org wrote:
This doesn't make sense to me.

Every scrap of documentation I've found says that GAE queries only
follow a single index (the one exception being zigzag merges, which
don't apply here). This means that to answer my query.filter(foo,
fooValue1).sort(-bar), there must be an index that contains the
foo and bar data sorted appropriately, no? Ie:

/Thing/foo:fooValue1/bar:bar9/[thekeyvalue]
/Thing/foo:fooValue1/bar:bar8/[thekeyvalue]
/Thing/foo:fooValue1/bar:bar7/[thekeyvalue]
/Thing/foo:fooValue2/bar:bar8/[thekeyvalue]
/Thing/foo:fooValue2/bar:bar7/[thekeyvalue]

To satisfy this query, GAE should start following this custom index
and that's pretty much it. There's no reason for it to touch the
single-property indexes (foo ASC, foo DESC, bar ASC, and bar DESC).

...and in my test, if I remove the custom index from
datastore-indexes.xml, it doesn't work. But also if I use
setUnindexedProperty, it doesn't work.

It's like setUnindexedProperty is being interpreted as not only don't
set single-property indexes, but also don't include the property in
any custom indexes. This is counterintuitive - if I wanted the index
not to be built, I can just remove the index.

I realize now that perhaps I posted this to the wrong mailing list.
The guys who created the I/O videos about the datastore seem to be
python fans, so I'll retry my original post on the google-appengine
list.

Jeff

On Thu, Mar 18, 2010 at 8:00 AM, Tristan tristan.slomin...@gmail.com
wrote:
Not official but been doing this for a while.

Your custom index is most likely build from the query. So, when you
do

Query query = new Query(Thing);
query.addFilter(foo, FilterOperator.EQUAL, fooValue);
query.addSort(bar, SortDirection.DESCENDING);

That is what builds your custom index.

However, when you setUnindexedProperty here

Entity ent = new Entity(Thing);
ent.setUnindexedProperty(foo, fooValue);
ent.setUnindexedProperty(bar, 123L);

You are not generating any index entries.

So the issue isn't that adding custom indexes after-the-fact [is]
really, really painful but that you are not generating any indexes
for the datastore to run the queries against when you use
setUnindexedProperty(). In other words, when you execute a query, it
checks the index to give you results. But you marked your data as
don't index me, so there is nothing for query to work with, as far
as it is concerned, there's nothing in the datastore.

Cheers!

On Mar 16, 4:07 pm, Jeff Schnitzer j...@infohazard.org wrote:
On Mon, Mar 15, 2010 at 11:04 PM, John Patterson
jdpatter...@gmail.com wrote:

On 16 Mar 2010, at 12:25, Jeff Schnitzer wrote:

I'm puzzled by the behavior of custom indexes. I have a simple
test
case below, a simple equality filter on one property combined with
a
descending sort on another property. If I set the properties with
setUnindexedProperty(), the query fails to find the result. If I
set
the properties with setProperty(), it does.

I also wondered why - I assume that the custom index build reads
the single
property indexes directly which must be more efficient than reading
the
Entities table.

I guess that is possible, but seems like a poor design decision. It
makes adding custom indexes after-the-fact really, really painful.

Can someone official chime in on this? Is it intended behavior, or
should we file an issue against it? The documentation doesn't say
much on the subject, and all the conceptual explanation of queries
suggests that these extra single-property indexes will be unused.

Jeff

--
You received this message because you are subscribed to the Google
Groups Google App Engine for Java group.
To post to this group, send email to
google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
.
For more options, visit this group athttp://
groups.google.com/group/google-appengine-java?hl=en.

--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To

[appengine-java] Re: java.io.IOException: Could not fetch URL problem and google spreadsheet api

2010-03-18 Thread Rahul

Any body got any clues on this ?



On Mar 17, 12:44 pm, Rahul rahul.jun...@gmail.com wrote:
 I am trying to open a connection to the following service 
 :http://translate.thoughtclicks.com/translateService/german/good
 morning which is definitely taking less then 5 seconds but i am still
 getting this exception any clue on this folks.

 My code to open the url connection is as follows.

             URL url = new URL(THIS_IS_THE_URL_MENTIONED_ABOVE);
             //make connection, use post mode, and send query
             URLConnection urlc = url.openConnection();
             urlc.setDoOutput(true);
             //retrieve result
             BufferedReader br = new BufferedReader(new
 InputStreamReader(urlc.getInputStream()));

 Any Clues on this folks ?

 Thanks,
 Rahul

 On Mar 17, 12:27 pm, Rahul rahul.jun...@gmail.com wrote:



  Did you find the solution for this as i am having the problem ?
  Any help appreciated.

  Thanks,
  Rahul

  On Mar 16, 10:31 am, dominity domin...@gmail.com wrote:

   Hi, guys.

   When my application trying to put user credentials into spreadsheet
   service, I've got this exception:

   com.google.gdata.util.AuthenticationException: Error connecting with
   login URI
           at
   com.google.gdata.client.GoogleAuthTokenFactory.getAuthToken(GoogleAuthToken
Factory.java:
   479)
           at
   com.google.gdata.client.GoogleAuthTokenFactory.setUserCredentials(GoogleAut
hTokenFactory.java:
   336)
           at
   com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java
:
   362)
           at
   com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java
:
   317)
           at
   com.google.gdata.client.GoogleService.setUserCredentials(GoogleService.java
:
   301)
   Caused by: java.io.IOException: Could not fetch 
   URL:https://www.google.com/accounts/ClientLogin
           at
   com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationExc
eption(URLFetchServiceImpl.java:
   106)
           at
   com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetch(URLFetchService
Impl.java:
   39)
           at
   com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
   $Connection.fetchResponse(URLFetchServiceStreamHandler.java:404)
           at
   com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
   $Connection.getInputStream(URLFetchServiceStreamHandler.java:283)
           at
   com.google.apphosting.utils.security.urlfetch.URLFetchServiceStreamHandler
   $Connection.getResponseCode(URLFetchServiceStreamHandler.java:136)
           at
   com.google.gdata.client.GoogleAuthTokenFactory.makePostRequest(GoogleAuthTo
kenFactory.java:
   550)
           at
   com.google.gdata.client.GoogleAuthTokenFactory.getAuthToken(GoogleAuthToken
Factory.java:
   477)

   I'm using Eclipse Plug-In and GAE SDK v1.3.1. Also, I was trying my
   code with version 1.3.0 and 1.2.9. And I can tell you that this
   happens only with development server, error disappears on production
   server. That's point - I can't even do any debugging stuff. So, could
   you give me any feedback on this issue?

   Best regards, Alexander.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Is XMPPServiceFactory service down ?

2010-03-18 Thread Don Schwarz

Can you respond privately with your app id?

On Thu, Mar 18, 2010 at 3:12 PM, Rahul rahul.jun...@gmail.com wrote:

 I am getting the following message again and again.

 Error for /_ah/xmpp/message/chat/
 java.lang.NoClassDefFoundError: com/google/appengine/api/xmpp/
 XMPPServiceFactory
 Caused by: java.lang.ClassNotFoundException:
 com.google.appengine.api.xmpp.XMPPServiceFactory
at
 com.google.appengine.runtime.Request.process-82178f5b5feb9148(Request.java)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClassInternal(Unknown Source)

 Thanks,
 Rahul

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine for Java group.
 To post to this group, send email to
 google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine-java?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Google App Engine error

2010-03-18 Thread Olifant

Ik heb een probleem met de Google App Engine.
Lokaal draait het prima, maar als ik mijn project naar de Google App
Engine server heb gedeployed en het daar wil bekijken, gaat het
verkeerd.

Het gaat om de volgende foutmelding:

/index.jsp
java.lang.ClassCastException: java.lang.String cannot be cast to
java.lang.Boolean
at
com.google.appengine.runtime.Request.process-45e9281fdbb0acf1(Request.java)
at
org.datanucleus.store.appengine.DatastoreFieldManager.fetchBooleanField(DatastoreFieldManager.java:
453)
at
org.datanucleus.state.AbstractStateManager.replacingBooleanField(AbstractStateManager.java:
1052)
at project.Project.jdoReplaceField(Project.java)
at project.Project.jdoReplaceFields(Project.java)
at
org.datanucleus.state.JDOStateManagerImpl.replaceFields(JDOStateManagerImpl.java:
2772)
at
org.datanucleus.state.JDOStateManagerImpl.replaceFields(JDOStateManagerImpl.java:
2791)
at
org.datanucleus.store.appengine.DatastorePersistenceHandler.fetchObject(DatastorePersistenceHandler.java:
461)
at
org.datanucleus.store.appengine.query.DatastoreQuery.entityToPojo(DatastoreQuery.java:
545)
at
org.datanucleus.store.appengine.query.DatastoreQuery.entityToPojo(DatastoreQuery.java:
500)
at org.datanucleus.store.appengine.query.DatastoreQuery.access
$300(DatastoreQuery.java:108)
at org.datanucleus.store.appengine.query.DatastoreQuery
$6.apply(DatastoreQuery.java:618)
at org.datanucleus.store.appengine.query.DatastoreQuery
$6.apply(DatastoreQuery.java:610)
at
org.datanucleus.store.appengine.query.LazyResult.resolveNext(LazyResult.java:
94)
at org.datanucleus.store.appengine.query.LazyResult
$LazyAbstractListIterator.computeNext(LazyResult.java:215)
at
org.datanucleus.store.appengine.query.AbstractIterator.tryToComputeNext(AbstractIterator.java:
132)
at
org.datanucleus.store.appengine.query.AbstractIterator.hasNext(AbstractIterator.java:
127)
at org.datanucleus.store.appengine.query.LazyResult
$AbstractListIterator.hasNext(LazyResult.java:169)
at project.GetProject.getProjectsDashboard(GetProject.java:33)
at org.apache.jsp.index_jsp._jspService(index_jsp.java:52)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
487)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1093)
at
cwipstoremoteuserfilter.filter.CwipsToRemoteUserFilter.doFilter(CwipsToRemoteUserFilter.java:
78)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at cwipsclient.cas.client.filter.CASFilter.doFilter(CASFilter.java:
79)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:
97)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:
35)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
43)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1084)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
360)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
181)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
712)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
405)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:
238)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
139)
at org.mortbay.jetty.Server.handle(Server.java:313)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
506)
at org.mortbay.jetty.HttpConnection
$RequestHandler.headerComplete(HttpConnection.java:830)
at
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:
76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:
135)
at
com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:
235)
at com.google.apphosting.base.RuntimePb$EvaluationRuntime
$6.handleBlockingRequest(RuntimePb.java:5485)
at

[appengine-java] Suddenly getting Unknown attribute type (com.google.appengine.api.datastore.Key) for attribute key.

2010-03-18 Thread Steve

Hi,
I've been successfully building my project each day.  Today I come to
it and attempt to start it in the debugger when I get the above
error.  Here is some of the stack trace:

org.apache.jasper.JasperException: /kickboxking.jsp(23,28) Unknown
attribute type (com.google.appengine.api.datastore.Key) for attribute
key.
at
org.apache.jasper.compiler.DefaultErrorHandler.jspError(DefaultErrorHandler.java:
39)
at
org.apache.jasper.compiler.ErrorDispatcher.dispatch(ErrorDispatcher.java:
409)
at
org.apache.jasper.compiler.ErrorDispatcher.jspError(ErrorDispatcher.java:
238)
at org.apache.jasper.compiler.Validator
$ValidateVisitor.checkXmlAttributes(Validator.java:932)
at org.apache.jasper.compiler.Validator
$ValidateVisitor.visit(Validator.java:696)
at org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1441)
at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213)
at org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2219)
at org.apache.jasper.compiler.Node$Root.accept(Node.java:456)
at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163)
at org.apache.jasper.compiler.Validator.validate(Validator.java:1475)
at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java:
214)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:470)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:451)
at org.apache.jasper.compiler.Compiler.compile(Compiler.java:439)
at
org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:
511)
...


So, it's when compiling the JSP.  I don't know if it's when resolving
the tag or the jsp that it encounters this unknown, but no matter -
neither has been changed(!) since it was compiling just fine.  Also, I
can see the appengine-api jar on the build path (as it was from when
the project was setup under Eclipse).

Here's an extract from the JSP:

%@ page contentType=text/html;charset=UTF-8 language=java %
%@ page import=java.util.List %
%@ page import=com.google.appengine.api.users.User %
%@ page import=com.google.appengine.api.users.UserService %
%@ page import=com.google.appengine.api.users.UserServiceFactory %
%@ page import=com.google.appengine.api.datastore.Key %

%@ taglib uri=/WEB-INF/kickboxking.tld prefix=kbk %

html
  body
div id=header
%
UserService userService = UserServiceFactory.getUserService();
if (session.getAttribute(activePlayerId) != null) {
%
kbk:playerName key=%=
(Key)session.getAttribute(activePlayerId) %/! (You can
a href=signOut.jspsign out/a.)/p
/div
...

It's falling over on that kbk:playerName tag   Pure bit-rot
as it was fine when I was running it yesterday :-(

Steve

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Geospatial data management with GAE Java

2010-03-18 Thread Rahul Ravikumar

There is a java port of the Geomodel project in java which you can try
out.

On Mar 17, 1:39 am, fvisticot fvisti...@gmail.com wrote:
 I know that Geomodel is a solution for python users to manage
 geospatial information.
 What is available for GAE Java users in term of Geospatial data
 management ?

 What is the best solution ?
 Is there a native approach provided by GAE Java ?

 In case no solution is available, what are the workarrounds ? Is
 Google Maps Data Api the solution ?

 Fred.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Any way to call Java Compiler directly on GAE ?

2010-03-18 Thread Didier Durand

Hi there,

Is there any way to call the java compiler itself in a GAE
application ?
I would like to port some Java code generator / compiler from regular
servers to GAE but the prereq is to be able to compile directly on
GAE.

I don't  find appropriate call in the JRE white list (http://
code.google.com/appengine/docs/java/jrewhitelist.html)

thanks
didier

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Unable to upload application

2010-03-18 Thread Jon McAlister

Yep, we had a bad admin console push and deploys were affected
from 3:10pm PST until 4:30pm PST. All better now. Apologies.

On Thu, Mar 18, 2010 at 3:50 PM, Huebi konrad.hueb...@googlemail.com wrote:
 here as well :(

 On 18 Mrz., 23:42, François Masurel fm2...@mably.com wrote:
 Same problem here too :-(

 On 18 mar, 23:30, Sandeep Sathaye sandeep.sath...@gmail.com wrote:



  I am having the same problem.

  On Thu, Mar 18, 2010 at 6:26 PM, Peter Ondruska 
  peter.ondru...@gmail.comwrote:

   I am trying to upload my application but it fails everytime:

   com.google.appengine.tools.admin.AdminException: Unable to update app:
   Error posting to URL:
  https://appengine.google.com/api/appversion/create?app_id=f72dzzn5q5q...
   500 Internal Server Error

   Server Error (500)
   A server error has occurred.

   at
   com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:
   62)
   at

   com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngin
eBridgeImpl.java:
   271)
   at

   com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(De
ployProjectJob.java:
   145)
   at

   org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorksp
aceJob.java:
   38)
   at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
   Caused by: java.io.IOException: Error posting to URL:

  https://appengine.google.com/api/appversion/create?app_id=f72dzzn5q5q...
   500 Internal Server Error

   Server Error (500)
   A server error has occurred.

   at

   com.google.appengine.tools.admin.ServerConnection.send(ServerConnection.jav
a:
   149)
   at

   com.google.appengine.tools.admin.ServerConnection.post(ServerConnection.jav
a:
   82)
   at

   com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.jav
a:
   532)
   at

   com.google.appengine.tools.admin.AppVersionUpload.beginTransaction(AppVersi
onUpload.java:
   349)
   at

   com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload
.java:
   111)
   at
   com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:
   56)
   ... 4 more

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine for Java group.
   To post to this group, send email to
   google-appengine-j...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B
unsubscr...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine-java?hl=en.

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine for Java group.
 To post to this group, send email to google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine-java+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine-java?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Does memcache get clean when new version deployed and made to be the default one?

2010-03-18 Thread opok

Hi all, does memcache get clean when new version deployed and made to
be the default one?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[google-appengine] Re: GAE getting CAPTCHA-locked by Google Apps?

2010-03-18 Thread Jan Z

Hi Tim - no, we're not.  A session might be created for a user based
on their activity, so maybe a few times a day.

We're definitely not hammering the system...

J

On Mar 18, 2:14 am, Tim Hoffman zutes...@gmail.com wrote:
 I have had problems in the past where I was using a lot of remote_api
 calls into appengine.

 What I found was, on each call I wasn't reusing an already
 authenticated session, which meant after a number of calls
 I would be hit with acaptchalock.  (it seems higher up google
 infrastructure didn't like me creating new authenticated session for
 each call - aside from the fact it isn't very efficient ;-)

 Maybe thats whats happening in your case, it might be worthing
 checking to see if you are creating a new authenticated session
 frequently.

 Rgds

 T

 On Mar 16, 4:15 am, Jan Z jan.zawad...@gmail.com wrote:



  gdata ClientLogin calls from GAE (and from GAE only) appear to get
  frequent spuriousCAPTCHAlocks.

  By spurious I mean that another ClientLogin attempt a couple of
  seconds later succeeds (with same credentials), so there is no need
  for users to actually go through the unlockcaptcha step.

  We can reproduce this on GAE reliably (as in acaptchaoccurring one
  or more times for every five attempts) using gdata (python) or
  straight protocol access with urlfetch.    Same code runs on local SDK
  without encountering a singlecaptcha.

  So it looks like the API iscaptcha-locking the GAE.

  We have an existing support case #00618963 for this, would be great to
  get someone from the GAE team to have a look please?

  Jan

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Increasing Task Queue Quota Limit

generally nothing you can do about those... i would say you can try
your code in the development server where they all queue up and you
have to execute manually (or flush them as desired). at least this
way, you can catch runaway situation before it hits you in production.
more info about this here:
http://code.google.com/appengine/docs/python/taskqueue/overview.html#Task_Queues_and_the_Development_Server

right now, you're good to go and only at 6% of your quota 55096 of 1M.

best regards,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Core Python Programming, Prentice Hall, (c)2007,2001
Python Fundamentals, Prentice Hall, (c)2009
   http://corepython.com

wesley.j.chun :: wesc+...@google.com
developer relations :: google app engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Access group info?

victor,

at this point, i don't believe there is a way to do this as our (App
Engine) interaction with Google Apps doesn't really go much beyond
making your app be restricted to your Google Apps domain and users:
http://code.google.com/appengine/articles/domains.html

this would be different, if Google Apps had an API that let you make
such a query. if anyone has been able to do what the OP requested, let
me know. in general, there isn't much interaction between the Apps and
App Engine except for what's described above.

best regards,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Core Python Programming, Prentice Hall, (c)2007,2001
Python Fundamentals, Prentice Hall, (c)2009
   http://corepython.com

wesley.j.chun :: wesc+...@google.com
developer relations :: google app engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Password For Loading Data Into My Local GAE Instance

greetings jorge,

i'm not sure why you're being prompted for the login when trying to
upload to the dev server. unfortunately, i'm not in the office this
week, but my guess is because you have login: admin in your app.yaml
file for remote_api which will require an admin login.

this is there for obvious reasons on production but perhaps you can
remove that line when trying to upload to your dev server and maybe it
won't prompt for the admin login info. let me know if this works!

thanks,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Core Python Programming, Prentice Hall, (c)2007,2001
Python Fundamentals, Prentice Hall, (c)2009
   http://corepython.com

wesley.j.chun :: wesc+...@google.com
developer relations :: google app engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Engine account activation

greetings and welcome to Google App Engine!

the email message you're referring to should be sent fairly quickly.
pls doublecheck your spam/bulk mail folder to see if it was
accidentally delivered there. if you cannot find it, can you cut-n-
paste us the error you get when you try to upload your app?

thanks,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Core Python Programming, Prentice Hall, (c)2007,2001
Python Fundamentals, Prentice Hall, (c)2009
   http://corepython.com

wesley.j.chun :: wesc+...@google.com
developer relations :: google app engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Storage for geo-points

generally we cannot change the maximum number of applications you can
unless you can demonstrate and justify a legitimate reasons why you
need more applications. in other words, it's not impossible but just
not likely. if you still want to try this, go fill out the form you
can access from: http://code.google.com/appengine/kb/billing.html#cpu

-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Core Python Programming, Prentice Hall, (c)2007,2001
Python Fundamentals, Prentice Hall, (c)2009
   http://corepython.com

wesley.j.chun :: wesc+...@google.com
developer relations :: google app engine

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] interval intersection - datastore modeling

2010-03-18 Thread jeremy

i have objects which store a 'start' and 'end' position.

given an arbitrary interval, i'd like to return all objects with which
it intersects.

can anyone think of an approach to modeling this on datastore?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] interval intersection - datastore modeling

2010-03-18 Thread Scott Ellis

are they dates? floating point values? values on a scale from 1 to 5? all of
the above?

On 18 March 2010 19:28, jeremy jeremy.a...@gmail.com wrote:

 i have objects which store a 'start' and 'end' position.

 given an arbitrary interval, i'd like to return all objects with which
 it intersects.

 can anyone think of an approach to modeling this on datastore?

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Post-mortem for February 24th, 2010 outage

2010-03-18 Thread Jan Z

Thanks for posting this.  The transparency helps greatly.

I would like to add my vote to Chris's though regarding splitting
bigtable operations.
It's a mistake.

The enormous appeal of the App Engine today is that you've done an
amazing job shielding us from needing to make these sorts of
decisions.

*THAT* is the hard problem that GAE is addressing.

By making distinctions such as this one, you're fundamentally shifting
your direction away from what ought to be (and has been?) a key design
principle.

In short - please don't make us choose.  Just make it work.

Jan / Cloudbreak

On Mar 5, 12:22 pm, App Engine Team appengine.nore...@gmail.com
wrote:
 Post-Mortem Summary

 This document details the cause and events occurring immediately after
 App Engine's outage on February 24th, 2010, as well as the steps we
 are taking to mitigate the impact of future outages like this one in
 the future.

 On February 24th, 2010, all Googe App Engine applications were in
 varying degraded states of operation for a period of two hours and
 twenty minutes from 7:48 AM to 10:09 AM PT | 15:48 to 18:09 GMT.  The
 underlying cause of the outage was a power failure in our primary
 datacenter. While the Google App Engine infrastructure is designed to
 quickly recover from these sort of failures, this type of rare
 problem, combined with internal procedural issues  extended the time
 required to restore the service.

 Link to full timeline here, which is attached below.

 What did we do wrong?

 Though the team had planned for this sort of failure, our response had
 a few important issues:

 - Although we had procedures ready for this sort of outage, the oncall
 staff was unfamiliar with them and had not trained sufficiently with
 the specific recovery procedure for this type of failure.

 - Recent work to migrate the datastore for better multihoming changed
 and improved the procedure for handling these failures significantly.
 However, some documentation detailing the procedure to support the
 datastore during failover incorrectly referred to the old
 configuration. This led to confusion during the event.

 - The production team had not agreed on a policy that clearly
 indicates when, and in what situations, our oncall staff should take
 aggressive user-facing actions, such as an unscheduled failover.  This
 led to a bad call of returning to a partially working datacenter.

 - We failed to plan for the case of a power outage that might affect
 some, but not all, of our machines in a datacenter (in this case,
 about 25%). In particular, this led to incorrect analysis of the
 serving state of the failed datacenter and when it might recover.

 - Though we were able to eventually migrate traffic to the backup
 datacenter, a small number of Datastore entity groups, belonging to
 approximately 25 applications in total,  became stuck in an
 inconsistent state as a result of the failover procedure. This
 represented considerably less than 0.2% of data stored in the
 Datastore.

 Ultimately, although significant work had been done over the past year
 to improve our handling of these types of outages, issues with
 procedures reduced their impact.

 What are we doing to fix it?

 As a result, we have instituted the following procedures going
 forward:

 - Introduce regular drills by all oncall staff of all of our
 production procedures. This will include the rare and complicated
 procedures, and all members of the team will be required to complete
 the drills before joining the oncall rotation.

 - Implement a regular bi-monthly audit of our operations docs to
 ensure that all needed procedures are properly findable, and all out-
 of-date docs are properly marked Deprecated.

 - Establish a clear policy framework to assist oncall staff to quickly
 and decisively make decisions about taking intrusive, user-facing
 actions during failures. This will allow them to act confidently and
 without delay in emergency situations.

 We believe that with these new procedures in place, last week's outage
 would have been reduced in impact from about 2 hours of total
 unavailability to about 10 to 20 minutes of partial unavailability.

 In response to this outage, we have also decided to make a major
 infrastructural change in App Engine. Currently, App Engine provides a
 one-size-fits-all Datastore, that provides low write latency combined
 with strong consistency, in exchange for lower availability in
 situations of unexpected failure in one of our serving datacenters. In
 response to this outage, and feedback from our users, we have begun
 work on providing two different Datastore configurations:

 - The current option of low-latency, strong consistency, and lower
 availability during unexpected failures (like a power outage)

 - A new option for higher availability using synchronous replication
 for reads and writes, at the cost of significantly higher latency

 We believe that providing both of these options to you, our users,
 will allow

[google-appengine] anybody success with blobstore upload via JS / iframe

2010-03-18 Thread Roberto Saccon

I am experimenting with large blob uploads. Everything works fine as
long as I submit a form to do the upload. But I would prefer to use a
page-refresh-less solution. So I tried no noswfupload.js and  iframe
approach ajaxupload.js, but none of them seem to work out of the box,
and I get the following error message.

SEVERE: Must only return a redirect from a Blobstore upload callback.

if I look at the development server admin panel, the __BlobInfo__
table entry actually gets created, but not the entry containing the
actual file

Anybody got ajax blob upload working ? Or has suggestion why it fails
in my case ?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Password For Loading Data Into My Local GAE Instance

2010-03-18 Thread Eli Jones

Appcfg and bulkloader seem to always prompt for credentials.. But you
can type anything in when uploading to dev_appserver.

Copy paste the exact command you're using and the full error you are getting.

Are you sending the --app_id flag to the command?

On 3/13/10, Jorge Lugo jlug...@gmail.com wrote:
 I tried the following command to upload files to my local GAE. I get
 prompted for an email address and password. What do I use for this
 since it is local? My Google email and password don't work.

 appcfg.py upload_data --config_file=upload.py --filename=cards.csv --
 kind=Card --url=http://localhost:8080/remote_api ..

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
Sent from my mobile device

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Email Idempotence

2010-03-18 Thread Tim Hoffman

I would pass of the email sending to a task queue, and wrap the
success of the send in a transaction
and only update to yes if the email send call returns ok.

At least you off load your work from the primary transaction.

Remember email isn't really a reliable service, and even if you do
mark it as sent it may never reach the recipient.

Pushing the send email workload and recording if you sent it into a
task should not really cost you to much
in end user response times/latency.

 As someone else put it email is not transactional in any sense.  Its
very much a
throw it over the fence and hope it gets there in the next few days

T

On Mar 18, 11:47 am, prgmratlarge yossiele...@gmail.com wrote:
 I guess all of you are technically correct. However, the solutions
 given were either: a) introduce a tremendous amount of latency/CPU
 usage in terms of putting/checking/deleting things from the datastore
 b) live with it. I think it's kind of weird for them to use this
 specific case as an example, when it clearly isn't the best example.

 I guess I'll just have to hope that everything goes okay, and write
 lots of try/except clauses in my code to make sure the little things
 don't mess stuff up.

 Thanks anyways,

 On Mar 17, 9:46 pm, Eli Jones eli.jo...@gmail.com wrote:



  Like Tim says... if you're really worried about sending out a duplicate
  e-mail from time to time... then you'll need to design your mailer task to
  do something like this:

  1.  Mailer task starts.. gets a batch of work from some datastore Model that
  serves as your queue.
  2.  Process a row of work, and after you successfully send out an e-mail for
  that row delete the row from your queue.

  Then just do step 2 over and over until you finish all of your work.. or if
  some error forces the task to retry.. then it should get a new batch of work
  from your queue.. which should now contain only entities that have not been
  processed.

  The only case where you'd get a duplicate email sent for a row of work would
  be if the email successfully was sent out.. but an error occurred when
  trying to remove the row from your queue.

  And, even in that case, you could add try: except: blocks to do everything
  possible to avoid that (just retry the db.delete() from your queue, etc).
  (Then, only the 30 second limit would be a concern.. but you can handle that
  as well).

  Then again, you really have to be adamant about avoiding the potential
  duplicate e-mail issue to do this sort of thing.

  On Wed, Mar 17, 2010 at 8:53 PM, Tim Hoffman zutes...@gmail.com wrote:
   Well could yo record if you sent any particular email, and
   not resend it ?

   On Mar 18, 6:05 am, prgmratlarge yossiele...@gmail.com wrote:
When implementing the code for Tasks (as worker URLs within your
app), it is important that you consider whether the task is
idempotent. App Engine's Task Queue API is designed to only invoke a
given task once, however it is possible in exceptional circumstances
that a Task may execute multiple times (e.g. in the unlikely case of
major system failure). Thus, your code must ensure that there are no
harmful side-effects of repeated execution. -http://code.google.com/
appengine/docs/python/taskqueue/overview.html

According to this part of the documentation, it would seem that
setting up an email-sending task-queue is impossible. After all,
sending someone the same email thousands of times would be disastrous.
Yet the documentation also gives email sending as an example use of
the Task Queue.

This seems to be contradictory. How can I create a Task Queue for
sending email without running into problems of idempotence?

Thanks for your help.

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: interval intersection - datastore modeling

sounds like the 3D graphics guys solved this problem... if you post
the solution perhaps i'll be able to suggest on how to do it in a
datastore

On Mar 18, 4:16 am, Scott Ellis sje...@gmail.com wrote:
 are they dates? floating point values? values on a scale from 1 to 5? all of
 the above?

 On 18 March 2010 19:28, jeremy jeremy.a...@gmail.com wrote:



  i have objects which store a 'start' and 'end' position.

  given an arbitrary interval, i'd like to return all objects with which
  it intersects.

  can anyone think of an approach to modeling this on datastore?

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Rollback Transaction not working?!?!?

Hi Blake,

Can you please include the code in which you enqueue the task?

-Nick Johnson

On Thu, Mar 18, 2010 at 4:00 AM, Blake blakecaldw...@gmail.com wrote:

 A bug in my code sent my tasks out of control, so I'm temporarily at
 my limit for the day.  One of my database inserts is in a transaction
 that kicks off a task, per instructions here:


 http://code.google.com/appengine/docs/java/taskqueue/overview.html#Task_Within_Transactions

 As expected, the task queuing failed because of my quota issues, but
 my database insert succeeded.  This isn't good.

 I verified that I'm calling rollback on my transaction, but my record
 it still making it to the database.  What am I doing wrong!?!?

 here's the basic logic:

 Transaction tx = pm.currentTransaction();
 try{
tx.begin()

Foo foo = pm.getObjectById(...)
foo.setBar(1);
pm.makePersistent(foo);

// queue up task (throws an exception)

tx.commit();
 }
 finally
 {
if(tx.isActive())
{
 // i verified that this is happening with a log statement
 tx.rollback();
}
 }


 I verified that rollback is happening with a lot statement in that
 last block, but like I said, the database gets the record.  My
 temporary fix will be to try to queue up the task before committing
 the record to the database, but then I'm relying on the task killing
 itself later on when a database exception occurs.

 Am I using transactions wrong?

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Using String as primary key and loading large objects?

I just want to suggest that you should probably use Long as a primary
key. Initially there won't be any issues using String, but then
eventually you'll run into something where you have the same titles
and it will be a legitimate case, and you'll end up doing all sorts of
gymnastics in order to allow them to coexist but still have unique
primary keys. I haven't ran into any issues with Long primary keys
yet.

On Mar 16, 11:45 am, Mark mar...@gmail.com wrote:
 Hi,

 Kind of new to this, two questions if anyone can help:

 1) I want to make some class that is persistent on the data store. I
 can use a primary key of either Long or String. It would be more
 convenient for me to use String as the primary key. Is this going to
 really hurt performance-wise when doing lookups for objects vs a Long?

 2) I have some objects which are somewhat large, like:

   public class Story {
      private String mId;
      private String mTitle;
      private String mFullText; -- large
   }

 if I want to get a list of Story items, I don't want to spend time
 loading the mFullText parameter on each one. I can hold off on loading
 that until a user wants to see the full story. Is there a way to
 implement two different load methods for Story to control this? Or do
 we simply break a class like this up into two separate classes:

   public class StoryDescription {
      private String mId;
      private String mTitle;
   }

   public class StoryText {
      private String mId;
      private String mFullText;
   }

 and load as needed?

 Thanks

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Using String as primary key and loading large objects?

As to question 2. It depends what you use to interface with the
datastore. I can only speak for low-level datastore api, and in that
case, the whole thing gets loaded at once. So splitting your objects
into two entities would make sense.

On the other hand, remember that there is 1MB limit in the datastore,
so you may want to explore Blobstore API, so that your object now
looks like:

public class Story {
  private String mId;
  private String mTitle;
  private something BlobstoreApiKey; - not large anymore
}

Cheers!

On Mar 18, 9:42 am, Tristan tristan.slomin...@gmail.com wrote:
 I just want to suggest that you should probably use Long as a primary
 key. Initially there won't be any issues using String, but then
 eventually you'll run into something where you have the same titles
 and it will be a legitimate case, and you'll end up doing all sorts of
 gymnastics in order to allow them to coexist but still have unique
 primary keys. I haven't ran into any issues with Long primary keys
 yet.

 On Mar 16, 11:45 am, Mark mar...@gmail.com wrote:



  Hi,

  Kind of new to this, two questions if anyone can help:

  1) I want to make some class that is persistent on the data store. I
  can use a primary key of either Long or String. It would be more
  convenient for me to use String as the primary key. Is this going to
  really hurt performance-wise when doing lookups for objects vs a Long?

  2) I have some objects which are somewhat large, like:

    public class Story {
       private String mId;
       private String mTitle;
       private String mFullText; -- large
    }

  if I want to get a list of Story items, I don't want to spend time
  loading the mFullText parameter on each one. I can hold off on loading
  that until a user wants to see the full story. Is there a way to
  implement two different load methods for Story to control this? Or do
  we simply break a class like this up into two separate classes:

    public class StoryDescription {
       private String mId;
       private String mTitle;
    }

    public class StoryText {
       private String mId;
       private String mFullText;
    }

  and load as needed?

  Thanks

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Why do custom indexes require single-property indexes?

For those of you reading this thread on appengine-java, I apologize.
However, I realized that this is really a general appengine question
and might get a better response from someone in the know here.
Here's the appengine-java thread:

http://groups.google.com/group/google-appengine-java/browse_thread/thread/efed35cabf60f6ee

--

I'm puzzled by the behavior of custom indexes.  I have a simple test
case below, a simple equality filter on one property combined with a
descending sort on another property.  If I set the properties with
setUnindexedProperty(), the query fails to find the result.  If I set
the properties with setProperty(), it does.

Why?  I have a custom index, therefore the query should not need or
touch the single-property indexes on these fields, right?

With this requirement, adding a single custom index means the
datastore must now update (at least) five indexes on every put() - the
single-value ASC and DESC indexes of both properties as well as my
custom index.  That's gotta hurt.

Here's a test case using the low-level API:

/** */
@Test
public void lowLevelTest() throws Exception
{
   DatastoreService service = DatastoreServiceFactory.getDatastoreService();

   Entity ent = new Entity(Thing);
   ent.setUnindexedProperty(foo, fooValue);
   ent.setUnindexedProperty(bar, 123L);
   // switching to this works
   //ent.setProperty(foo, fooValue);
   //ent.setProperty(bar, 123L);
   service.put(ent);

   Query query = new Query(Thing);
   query.addFilter(foo, FilterOperator.EQUAL, fooValue);
   query.addSort(bar, SortDirection.DESCENDING);

   PreparedQuery pq = service.prepare(query);
   int count = 0;
   for (Entity fetched: pq.asIterable())
   {
   count++;
   }

   assert count == 1;
}

The last assertion fails.  The query doesn't find any results.  The
automatic datastore index seems to be fine:

!-- Indices written at Mon, 15 Mar 2010 21:49:01 PDT --
datastore-indexes
   !-- Used 1 time in query history --
   datastore-index kind=Thing ancestor=false source=auto
   property name=foo direction=asc/
   property name=bar direction=desc/
   /datastore-index
/datastore-indexes

What's up?  Is this just a bug in the dev mode, or is there a real
requirement that all properties must have single-value indexes in
order to be part of a custom index?

Thanks,
Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Does datastore cache for ReferenceProperty?

Hi Shinichi,

Referenced entities are cached against the entity that's doing the
referencing - so if you have multiple entities referencing the same entity,
it'll be fetched multiple times. You can work around this by passing it in,
as you suggest, or by using a prefetch recipe such as this one:
http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine

In general, Guido's appstats is an excellent tool for diagnosing these sort
of things: https://sites.google.com/site/appengineappstats/

-Nick Johnson

On Tue, Mar 16, 2010 at 7:39 AM, Shinichi Nakanishi stouton...@gmail.comwrote:

 Does datastore run query for Instructor each time?  Or does it some
 sort of cache thing?

 If I have these models:

 class Instructor(db.Model)
firstName = db.StringProperty()
lastName = db.StringProperty()

 class Participant(db.Model)
firstName = db.StringProperty()
lastName = db.StringProperty()
workshop = db.ReferenceProperty(Workshop, collection_name =
 participants)

 class Workshop(db.Model)
instructor = db.ReferenceProperty(Instructor)

 Then if I do the following,

 workshop = Workshop.all().get()
 for participant in workshop.participants:
someMethod(workshop, participant)

 def someMethod(self, workshop, participant)
if (workshop.instructor...  # does it run query every time or does it
 cache?

 If datastore queries Instructor every time when I do
 workshop.instructor, I thought I should pass instructor as an
 argument.

 Shinichi

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Why do custom indexes require single-property indexes?

Hi Jeff,

Unindexed properties are excluded from all indexing, not just from indexing
by the built-in indexes.

-Nick Johnson

On Thu, Mar 18, 2010 at 4:27 PM, Jeff Schnitzer j...@infohazard.org wrote:

 For those of you reading this thread on appengine-java, I apologize.
 However, I realized that this is really a general appengine question
 and might get a better response from someone in the know here.
 Here's the appengine-java thread:

 http://groups.google.com/group/google-appengine-java/browse_thread/thread/efed35cabf60f6ee

 --

 I'm puzzled by the behavior of custom indexes.  I have a simple test
 case below, a simple equality filter on one property combined with a
 descending sort on another property.  If I set the properties with
 setUnindexedProperty(), the query fails to find the result.  If I set
 the properties with setProperty(), it does.

 Why?  I have a custom index, therefore the query should not need or
 touch the single-property indexes on these fields, right?

 With this requirement, adding a single custom index means the
 datastore must now update (at least) five indexes on every put() - the
 single-value ASC and DESC indexes of both properties as well as my
 custom index.  That's gotta hurt.

 Here's a test case using the low-level API:

 /** */
 @Test
 public void lowLevelTest() throws Exception
 {
   DatastoreService service =
 DatastoreServiceFactory.getDatastoreService();

   Entity ent = new Entity(Thing);
   ent.setUnindexedProperty(foo, fooValue);
   ent.setUnindexedProperty(bar, 123L);
   // switching to this works
   //ent.setProperty(foo, fooValue);
   //ent.setProperty(bar, 123L);
   service.put(ent);

   Query query = new Query(Thing);
   query.addFilter(foo, FilterOperator.EQUAL, fooValue);
   query.addSort(bar, SortDirection.DESCENDING);

   PreparedQuery pq = service.prepare(query);
   int count = 0;
   for (Entity fetched: pq.asIterable())
   {
   count++;
   }

   assert count == 1;
 }

 The last assertion fails.  The query doesn't find any results.  The
 automatic datastore index seems to be fine:

 !-- Indices written at Mon, 15 Mar 2010 21:49:01 PDT --
 datastore-indexes
   !-- Used 1 time in query history --
   datastore-index kind=Thing ancestor=false source=auto
   property name=foo direction=asc/
   property name=bar direction=desc/
   /datastore-index
 /datastore-indexes

 What's up?  Is this just a bug in the dev mode, or is there a real
 requirement that all properties must have single-value indexes in
 order to be part of a custom index?

 Thanks,
 Jeff

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Why do custom indexes require single-property indexes?

From one of my followup posts:

Every scrap of documentation I've found says that GAE queries only
follow a single index (the one exception being zigzag merges, which
don't apply here).  This means that to answer my query.filter(foo,
fooValue1).sort(-bar), there must be an index that contains the
foo and bar data sorted appropriately, no?  Ie:

/Thing/foo:fooValue1/bar:bar9/[thekeyvalue]
/Thing/foo:fooValue1/bar:bar8/[thekeyvalue]
/Thing/foo:fooValue1/bar:bar7/[thekeyvalue]
/Thing/foo:fooValue2/bar:bar8/[thekeyvalue]
/Thing/foo:fooValue2/bar:bar7/[thekeyvalue]

To satisfy this query, GAE should start following this custom index
and that's pretty much it.  There's no reason for it to touch the
single-property indexes (foo ASC, foo DESC, bar ASC, and bar DESC).

...and in my test, if I remove the custom index from
datastore-indexes.xml, it doesn't work.  But also if I use
setUnindexedProperty, it doesn't work.

It's like setUnindexedProperty is being interpreted as not only don't
set single-property indexes, but also don't include the property in
any custom indexes.  This is counterintuitive - if I wanted the index
not to be built, I can just remove the index.

Jeff

On Thu, Mar 18, 2010 at 9:27 AM, Jeff Schnitzer j...@infohazard.org wrote:
 For those of you reading this thread on appengine-java, I apologize.
 However, I realized that this is really a general appengine question
 and might get a better response from someone in the know here.
 Here's the appengine-java thread:
    
 http://groups.google.com/group/google-appengine-java/browse_thread/thread/efed35cabf60f6ee

 --

 I'm puzzled by the behavior of custom indexes.  I have a simple test
 case below, a simple equality filter on one property combined with a
 descending sort on another property.  If I set the properties with
 setUnindexedProperty(), the query fails to find the result.  If I set
 the properties with setProperty(), it does.

 Why?  I have a custom index, therefore the query should not need or
 touch the single-property indexes on these fields, right?

 With this requirement, adding a single custom index means the
 datastore must now update (at least) five indexes on every put() - the
 single-value ASC and DESC indexes of both properties as well as my
 custom index.  That's gotta hurt.

 Here's a test case using the low-level API:

 /** */
 @Test
 public void lowLevelTest() throws Exception
 {
       DatastoreService service = 
 DatastoreServiceFactory.getDatastoreService();

       Entity ent = new Entity(Thing);
       ent.setUnindexedProperty(foo, fooValue);
       ent.setUnindexedProperty(bar, 123L);
       // switching to this works
       //ent.setProperty(foo, fooValue);
       //ent.setProperty(bar, 123L);
       service.put(ent);

       Query query = new Query(Thing);
       query.addFilter(foo, FilterOperator.EQUAL, fooValue);
       query.addSort(bar, SortDirection.DESCENDING);

       PreparedQuery pq = service.prepare(query);
       int count = 0;
       for (Entity fetched: pq.asIterable())
       {
               count++;
       }

       assert count == 1;
 }

 The last assertion fails.  The query doesn't find any results.  The
 automatic datastore index seems to be fine:

 !-- Indices written at Mon, 15 Mar 2010 21:49:01 PDT --
 datastore-indexes
   !-- Used 1 time in query history --
   datastore-index kind=Thing ancestor=false source=auto
       property name=foo direction=asc/
       property name=bar direction=desc/
   /datastore-index
 /datastore-indexes

 What's up?  Is this just a bug in the dev mode, or is there a real
 requirement that all properties must have single-value indexes in
 order to be part of a custom index?

 Thanks,
 Jeff


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Why do custom indexes require single-property indexes?

On Thu, Mar 18, 2010 at 4:32 PM, Jeff Schnitzer j...@infohazard.org wrote:

 From one of my followup posts:

 Every scrap of documentation I've found says that GAE queries only
 follow a single index (the one exception being zigzag merges, which
 don't apply here).  This means that to answer my query.filter(foo,
 fooValue1).sort(-bar), there must be an index that contains the
 foo and bar data sorted appropriately, no?  Ie:

 /Thing/foo:fooValue1/bar:bar9/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue1/bar:bar7/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar8/[thekeyvalue]
 /Thing/foo:fooValue2/bar:bar7/[thekeyvalue]

 To satisfy this query, GAE should start following this custom index
 and that's pretty much it.  There's no reason for it to touch the
 single-property indexes (foo ASC, foo DESC, bar ASC, and bar DESC).

 ...and in my test, if I remove the custom index from
 datastore-indexes.xml, it doesn't work.  But also if I use
 setUnindexedProperty, it doesn't work.

 It's like setUnindexedProperty is being interpreted as not only don't
 set single-property indexes, but also don't include the property in
 any custom indexes.  This is counterintuitive - if I wanted the index
 not to be built, I can just remove the index.


That's correct. Entities are evaluated individually, not as a group, and any
unindexed properties are ignored for the purposes of all indexing, not just
for built-in indexes.

-Nick Johnson



 Jeff

 On Thu, Mar 18, 2010 at 9:27 AM, Jeff Schnitzer j...@infohazard.org
 wrote:
  For those of you reading this thread on appengine-java, I apologize.
  However, I realized that this is really a general appengine question
  and might get a better response from someone in the know here.
  Here's the appengine-java thread:
 
 http://groups.google.com/group/google-appengine-java/browse_thread/thread/efed35cabf60f6ee
 
  --
 
  I'm puzzled by the behavior of custom indexes.  I have a simple test
  case below, a simple equality filter on one property combined with a
  descending sort on another property.  If I set the properties with
  setUnindexedProperty(), the query fails to find the result.  If I set
  the properties with setProperty(), it does.
 
  Why?  I have a custom index, therefore the query should not need or
  touch the single-property indexes on these fields, right?
 
  With this requirement, adding a single custom index means the
  datastore must now update (at least) five indexes on every put() - the
  single-value ASC and DESC indexes of both properties as well as my
  custom index.  That's gotta hurt.
 
  Here's a test case using the low-level API:
 
  /** */
  @Test
  public void lowLevelTest() throws Exception
  {
DatastoreService service =
 DatastoreServiceFactory.getDatastoreService();
 
Entity ent = new Entity(Thing);
ent.setUnindexedProperty(foo, fooValue);
ent.setUnindexedProperty(bar, 123L);
// switching to this works
//ent.setProperty(foo, fooValue);
//ent.setProperty(bar, 123L);
service.put(ent);
 
Query query = new Query(Thing);
query.addFilter(foo, FilterOperator.EQUAL, fooValue);
query.addSort(bar, SortDirection.DESCENDING);
 
PreparedQuery pq = service.prepare(query);
int count = 0;
for (Entity fetched: pq.asIterable())
{
count++;
}
 
assert count == 1;
  }
 
  The last assertion fails.  The query doesn't find any results.  The
  automatic datastore index seems to be fine:
 
  !-- Indices written at Mon, 15 Mar 2010 21:49:01 PDT --
  datastore-indexes
!-- Used 1 time in query history --
datastore-index kind=Thing ancestor=false source=auto
property name=foo direction=asc/
property name=bar direction=desc/
/datastore-index
  /datastore-indexes
 
  What's up?  Is this just a bug in the dev mode, or is there a real
  requirement that all properties must have single-value indexes in
  order to be part of a custom index?
 
  Thanks,
  Jeff
 

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
 .
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Auth with google apps acc ounts — depends on billing status?

Hi Denis,

Auth via appspot.com will only work if you've added that app to your Google
Apps control panel. The error you're seeing is because you haven't added
this second app to your Apps control panel.

-Nick Johnson

On Mon, Mar 15, 2010 at 10:48 PM, Denis Moskalets denya@gmail.comwrote:

Both apps have auth through the same google apps domain.

Copy-paste from from both apps - appspot.com - Application Settings:

Authentication Options:
When this application was created it was configured to allow anyone
with a valid xxx.xx Google Apps domain to sign in if the Google
Accounts API is used for authentication. Learn more

Only difference between two apps: one have billing enabled, another
— doesn't have.

On Mar 15, 4:57 pm, Wooble geoffsp...@gmail.com wrote:
On Mar 15, 3:06 am, Denis Moskalets denya@gmail.com wrote:

I have two appspot applications.
One — fot tests, second — production.
Not important, why it is so.

Two apps have same source code.
Auth throw Google Apps Accounts enabled on both.
Billing enabled only for one app.

Apps are closed for public, so only registred and authorized users can
access it. When u try to open page, you will get redirect to default
google apps login page.

So.
When i try to open production app — all right, i've got redirect to
login page:http://ctms-release.appspot.com/

The second app generate 500 eror.http://ctms-medms-test.appspot.com/

In logs:
File /base/data/home/apps/ctms-medms-test/.../googletags.py, line
11, in google_login_url
return escape(users.create_login_url(redirect))
File /base/python_lib/versions/1/google/appengine/api/users.py,
line 179, in create_login_url
raise NotAllowedError

Billing is turned off for this non-workin' app.

I've looked up manual, docs and FAQs, but there is nothing about
difference in login procedure between free/billed apps.

If an application has login restricted to a Google Apps domain, you
cannot create login URLs when the page is being visited through
appspot; it's only possible on the actual domain to which login is
restricted. I'd guess you only actually restricted logins on the test
application and not the real application; you can check this (but,
alas, not change it) under Application Settings in the appengine
dashboard.

Billing should not have any effect on the login process, as far as I
know.

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Does datastore cache for ReferenceProperty?

2010-03-18 Thread Shinichi Nakanishi

Hi Nick,

Thank you for your response.  That was exactly what I wanted to know.
AppStats looks pretty neat and I will try to use that.

Shinichi

On Thu, Mar 18, 2010 at 09:29, Nick Johnson (Google)
nick.john...@google.com wrote:
 Hi Shinichi,
 Referenced entities are cached against the entity that's doing the
 referencing - so if you have multiple entities referencing the same entity,
 it'll be fetched multiple times. You can work around this by passing it in,
 as you suggest, or by using a prefetch recipe such as this
 one: http://blog.notdot.net/2010/01/ReferenceProperty-prefetching-in-App-Engine
 In general, Guido's appstats is an excellent tool for diagnosing these sort
 of things: https://sites.google.com/site/appengineappstats/
 -Nick Johnson

 On Tue, Mar 16, 2010 at 7:39 AM, Shinichi Nakanishi stouton...@gmail.com
 wrote:

 Does datastore run query for Instructor each time?  Or does it some
 sort of cache thing?

 If I have these models:

 class Instructor(db.Model)
    firstName = db.StringProperty()
    lastName = db.StringProperty()

 class Participant(db.Model)
    firstName = db.StringProperty()
    lastName = db.StringProperty()
    workshop = db.ReferenceProperty(Workshop, collection_name =
 participants)

 class Workshop(db.Model)
    instructor = db.ReferenceProperty(Instructor)

 Then if I do the following,

 workshop = Workshop.all().get()
 for participant in workshop.participants:
    someMethod(workshop, participant)

 def someMethod(self, workshop, participant)
    if (workshop.instructor...  # does it run query every time or does it
 cache?

 If datastore queries Instructor every time when I do
 workshop.instructor, I thought I should pass instructor as an
 argument.

 Shinichi

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




 --
 Nick Johnson, Developer Programs Engineer, App Engine
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Why do custom indexes require single-property indexes?

On Thu, Mar 18, 2010 at 9:32 AM, Nick Johnson (Google)
nick.john...@google.com wrote:
 Hi Jeff,
 Unindexed properties are excluded from all indexing, not just from indexing
 by the built-in indexes.
 -Nick Johnson

Ah ha.  The next question then is - why?

This current arrangement has some particularly nasty consequences:

1) If you want a custom index across 3 properties, you are now forced
to maintain seven indexes total - the three single-property ASC
indexes, the three single-property DESC indexes, and the actual custom
index.

2) If you want to create a custom index across a property that does
not currently have a single-property index, you must manually go
through and reprocess you entire dataset.  The automatic index
building isn't automatic.

On the other hand, there doesn't seem to be much benefit to the
current approach.  It does provide the ability to make rdbms-like
partial indexes in custom indexes as well as single-property indexes,
but it seems like any minor gain is going to be overshadowed by the
fact that you're now maintaining quite a large number of additional
indexes.

Furthermore, the only way to access this partial index functionality
is with the low-level API.  At the higher level APIs, you can only
flag a property as unindexed or not.  If the user wants a property to
be excluded from a custom index... they can easily remove it from the
datastore-indexes.xml/yaml.  So this default doesn't buy you anything
and actually costs you a lot of flexibility and extra index overhead.

Unless I'm missing something?

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Why do custom indexes require single-property indexes?