[google-appengine] backends returning 50x errors

2012-01-26 Thread Ben Nevile
Hi,

I tried to start a backend on an app (id: butterpool) 1d7h ago, but
I have been consistently getting the service you requested is not
available yet error. I tried both dynamic and resident backends.

I see that other people have had this problem, and that in at least
one case it was solved by some kind of magic fix by a Google admin to
the application's state. Could someone at Google please help me out?

Ben

PS. I posted this same question to google-appengine-python, but
haven't received a response. I find the GAE support extremely
frustrating.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Weird Upload Errors

2010-10-18 Thread Ben Nevile
Same problems here. And even worse, the app seems to be stuck in a
state of non-responsiveness.  Regular URL requests are returning 404:

htmlhead
meta http-equiv=content-type content=text/html;charset=utf-8
title404 Not Found/title
/head
body text=#00 bgcolor=#ff
h1Error: Not Found/h1
h2The requested URL code//code was not found on this server./
h2
h2/h2
/body/html


eg: http://butterpool.appspot.com


Terrible timing for me, as it's the busiest time of the year and I
just launched a new product yesterday. Please help!

Ben



On Oct 18, 1:49 pm, Bart Thate bth...@gmail.com wrote:
 Same here, can take ages for an upload to finish. Just loops the 60 second
 dance.





 On Mon, Oct 18, 2010 at 10:38 PM, dloomer dloo...@gmail.com wrote:
  I should note I'm not getting the 403 or 409 error, just the repeated
  will check again in 60 seconds until it gives up with version not
  ready.

  On Oct 18, 3:35 pm, Amir  Michail amich...@gmail.com wrote:
   On Oct 18, 4:18 pm, J j.si...@earlystageit.com wrote:

Hi,

Hopefully someone can shed some light on this: all of a sudden, I
started receiving '409' errors. For example,

   I can't deploy a new version of an app.  It just keeps saying Will
   check again in 60 seconds.

   I don't see the 409 error.

   Amir

...
...
Will check again in 32 seconds.
Checking if new version is ready to serve.
Will check again in 60 seconds.
Checking if new version is ready to serve.
Will check again in 60 seconds.
Checking if new version is ready to serve.
Will check again in 60 seconds.
Checking if new version is ready to serve.
Will check again in 60 seconds.
Checking if new version is ready to serve.
2010-10-18 16:14:18,997 INFO appcfg.py:1814 HTTP Error (HTTP Error
409: Conflict)
Rolling back the update.
Error 409: --- begin server output ---
No transaction is in progress.
--- end server output ---

It has happened a couple of times already.

Thanks in advance.

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 @jsonbot Heerhugowaard, Netherlands
 programming schizofrenic -http://jsonbot.appspot.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Dashboard - Current Load could be more useful

2010-10-11 Thread Ben Nevile
I have several app engine apps that uses the following URL structure:

/:object/:key

eg:

/player/:key
/contest/:key
/team/:key

etc.

The Current Load section of the dashboard is not useful with this
URL structure. App Engine expects that the URLs will be parametrized
(eg /:object?key=:key ).  That's fine, but a bit old-fashioned and
verbose.

Anyway, is there any way to work around this issue?

Ben


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: MapReduce - less than half of the shards have items

2010-07-07 Thread Ben Nevile
I have a job with 8 shards but only one of the shards is processing
any entities. Think this is related?

mapreduce_id=159013376839436

Ben




On Jun 16, 8:28 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Jason,

 It's possible to do that right now. You simply need to define your own
 InputReader - see the existing ones for examples 
 how:http://code.google.com/p/appengine-mapreduce/source/browse/trunk/pyth...

 The best option is probably to extend the existing DatastoreInputReader, and
 override the split_input method.

 -Nick Johnson





 On Wed, Jun 16, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com wrote:
  Nick,

  It would be a great feature to allow us (developers) to define a
  custom partitioning function for the shards - unless you guys have
  some better magic in mind.

  It is very normal for us to see very poor distribution across our
  shards in practise - in fact, we've only seen poor distributions.

  j

  On Jun 15, 8:04 am, Nick Johnson (Google) nick.john...@google.com
  wrote:
   Hi Jason,

   The current implementation of the datastore mapper uses lexicographical
   sharding over keys to assign datastore shards. Unfortunately, this can
  lead
   to very inconsistent shard sizes, as you observe.

   -Nick Johnson

   On Fri, Jun 11, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com
  wrote:
We've been using MapReduce for App Engine for a couple of different
jobs.

Typically, we use 8 shards (the default), but it seems that only 3,
sometime 4, of the shards have any items in them? E.g., we're
currently running one job and three of the shards have 218,000 items
processed, but the other 5 shards appear to have zero.

I can understand that a particular key distribution would have
different amounts in each shard, but with so many at zero, I suspect
there is something else happening?

BTW, we have applied the mapreduce-recommended __key__ DESC index, but
we still see this strange shard distribution.

Is anyone else seeing this?

j

--
You received this message because you are subscribed to the Google
  Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
  .
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
 e...@googlegroups.comgoogle-appengine%2Bunsubscrib
  e...@googlegroups.com
.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

   --
   Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd.
  ::
   Registered in Dublin, Ireland, Registration Number: 368047
   Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
  Number:
   368047

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib 
  e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. ::
 Registered in Dublin, Ireland, Registration Number: 368047
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: MapReduce - less than half of the shards have items

2010-07-07 Thread Ben Nevile
Just found the thread that talks about needing the descending index
for __key__ --- definitely seems like something that should be added
to the documentation!

Ben


On Jul 7, 12:56 pm, Ben Nevile ben.nev...@gmail.com wrote:
 I have a job with 8 shards but only one of the shards is processing
 any entities. Think this is related?

 mapreduce_id=159013376839436

 Ben

 On Jun 16, 8:28 am, Nick Johnson (Google) nick.john...@google.com
 wrote:



  Hi Jason,

  It's possible to do that right now. You simply need to define your own
  InputReader - see the existing ones for examples 
  how:http://code.google.com/p/appengine-mapreduce/source/browse/trunk/pyth...

  The best option is probably to extend the existing DatastoreInputReader, and
  override the split_input method.

  -Nick Johnson

  On Wed, Jun 16, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com wrote:
   Nick,

   It would be a great feature to allow us (developers) to define a
   custom partitioning function for the shards - unless you guys have
   some better magic in mind.

   It is very normal for us to see very poor distribution across our
   shards in practise - in fact, we've only seen poor distributions.

   j

   On Jun 15, 8:04 am, Nick Johnson (Google) nick.john...@google.com
   wrote:
Hi Jason,

The current implementation of the datastore mapper uses lexicographical
sharding over keys to assign datastore shards. Unfortunately, this can
   lead
to very inconsistent shard sizes, as you observe.

-Nick Johnson

On Fri, Jun 11, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com
   wrote:
 We've been usingMapReducefor App Engine for a couple of different
 jobs.

 Typically, we use 8 shards (the default), but it seems that only 3,
 sometime 4, of the shards have any items in them? E.g., we're
 currently running one job and three of the shards have 218,000 items
 processed, but the other 5 shards appear to have zero.

 I can understand that a particular key distribution would have
 different amounts in each shard, but with so many at zero, I suspect
 there is something else happening?

 BTW, we have applied themapreduce-recommended __key__ DESC index, but
 we still see this strange shard distribution.

 Is anyone else seeing this?

 j

 --
 You received this message because you are subscribed to the Google
   Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com
   .
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
  e...@googlegroups.comgoogle-appengine%2Bunsubscrib
   e...@googlegroups.com
 .
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine Google Ireland 
Ltd.
   ::
Registered in Dublin, Ireland, Registration Number: 368047
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
   Number:
368047

   --
   You received this message because you are subscribed to the Google Groups
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to
   google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
   .
   For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

  --
  Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. ::
  Registered in Dublin, Ireland, Registration Number: 368047
  Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
  368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: MapReduce - less than half of the shards have items

2010-07-07 Thread Ben Nevile
I reduced the number of shards to 1, and it uses an ascending index,
but still only 44399 out of 500k+ entities are processed. Confused!

Ben


On Jul 7, 1:02 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Just found the thread that talks about needing the descending index
 for __key__ --- definitely seems like something that should be added
 to the documentation!

 Ben

 On Jul 7, 12:56 pm, Ben Nevile ben.nev...@gmail.com wrote:



  I have a job with 8 shards but only one of the shards is processing
  any entities. Think this is related?

  mapreduce_id=159013376839436

  Ben

  On Jun 16, 8:28 am, Nick Johnson (Google) nick.john...@google.com
  wrote:

   Hi Jason,

   It's possible to do that right now. You simply need to define your own
   InputReader - see the existing ones for examples 
   how:http://code.google.com/p/appengine-mapreduce/source/browse/trunk/pyth...

   The best option is probably to extend the existing DatastoreInputReader, 
   and
   override the split_input method.

   -Nick Johnson

   On Wed, Jun 16, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com 
   wrote:
Nick,

It would be a great feature to allow us (developers) to define a
custom partitioning function for the shards - unless you guys have
some better magic in mind.

It is very normal for us to see very poor distribution across our
shards in practise - in fact, we've only seen poor distributions.

j

On Jun 15, 8:04 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Jason,

 The current implementation of the datastore mapper uses 
 lexicographical
 sharding over keys to assign datastore shards. Unfortunately, this can
lead
 to very inconsistent shard sizes, as you observe.

 -Nick Johnson

 On Fri, Jun 11, 2010 at 4:17 PM, Jason C jason.a.coll...@gmail.com
wrote:
  We've been usingMapReducefor App Engine for a couple of different
  jobs.

  Typically, we use 8 shards (the default), but it seems that only 3,
  sometime 4, of the shards have any items in them? E.g., we're
  currently running one job and three of the shards have 218,000 
  items
  processed, but the other 5 shards appear to have zero.

  I can understand that a particular key distribution would have
  different amounts in each shard, but with so many at zero, I suspect
  there is something else happening?

  BTW, we have applied themapreduce-recommended __key__ DESC index, 
  but
  we still see this strange shard distribution.

  Is anyone else seeing this?

  j

  --
  You received this message because you are subscribed to the Google
Groups
  Google App Engine group.
  To post to this group, send email to 
  google-appengine@googlegroups.com
.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
   e...@googlegroups.comgoogle-appengine%2Bunsubscrib
e...@googlegroups.com
  .
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Nick Johnson, Developer Programs Engineer, App Engine Google Ireland 
 Ltd.
::
 Registered in Dublin, Ireland, Registration Number: 368047
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
 368047

--
You received this message because you are subscribed to the Google 
Groups
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib
 e...@googlegroups.com
.
For more options, visit this group at
   http://groups.google.com/group/google-appengine?hl=en.

   --
   Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. 
   ::
   Registered in Dublin, Ireland, Registration Number: 368047
   Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
   368047

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Entity not updating

2009-09-13 Thread Ben Nevile

No, this is wrong.  Your GQL query wasn't working because you didn't
get() or fetch() a result.

uProfile = PROFILES.gql('...').get()



On Sep 12, 4:13 pm, Larkin2 michael.mer...@gmail.com wrote:
 I think I answered my own question - I'm retrieving a result set
 with .gql - to update an entity I need to specifically grab it with
 get_by_id() or get_by_key()...

 so for example:

 profileToUpdate = PROFILES.gql(WHERE user = :1, user)
 updatedProfile = profileToUpdate[0].key().id()
 uProfile = PROFILES.get_by_id(updatedProfile)
 uProfile.name = self.request.get('name2')
 db.put(uProfile)

 On Sep 12, 6:53 pm, Larkin2 michael.mer...@gmail.com wrote:



  uProfile = PROFILES.gql(WHERE user = :1, user)
  uProfile[0].name = self.request.get('name2')
  db.put(uProfile)

  I've been racking my brain for several hours on this - is there
  anything in the code above that would prevent this particular entity
  in PROFILES from not being updated? I can't get the entity to update
  for the life of me.

  Thanks

  Larkin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Google Checkout Rate Your Experience emails

2009-08-27 Thread Ben Nevile

Hi,

Is there any way to get Google Checkout to stop sending me Rate your
shopping experience emails?  I've answered them, but they keep
coming.  I have a handful of apps and each of them generates a
checkout charge once a day, which means I get a handful of rate your
shopping experience emails every day.

Ben


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Maintenance Timing

2009-08-24 Thread Ben Nevile

I second Jeff's call for some transparency here.

Ben

On Aug 18, 10:17 am, Jeff j...@jgyoung.com wrote:
 Sometimes more warning isn't possible if maintenance is necessary to
 avoid the system tipping over because of a problem.  But of course, if
 it wasn't an emergency maintenance, then 22 hours of warning is
 definitely insufficient.

 But in spite of the comments above, I'll be quiet about this when
 Google says they scheduled the downtime at the best time they could
 based on customer usage and not for their own convenience.  I can
 hardly imagine that noon EST and 9am PST could possibly be the optimum
 time ... except for the convenience of the west coast Google employees
 that did not want to work during the night.  Yes I know there is usage
 worldwide, but ...

 How about it Google, a response defending this seemingly very poor
 scheduling choice?  Tell us why it wasn't possible to provide more
 notice or schedule this at a lower usage time and why we can expect in
 the future you are striving to minimize downtime impact to your
 customers.

 On Aug 18, 9:54 am, BenNevileben.nev...@gmail.com wrote:



  +1 for more advance warning, please.

  Ben

  On Aug 17, 11:52 am, Joshua Smith joshuaesm...@charter.net wrote:

   I really appreciate that you are warning us about the maintenance.
   However, next time, I'd ask:

   1) More warning, if possible; and,

   2) Do it sometime other than the middle of the day in the United
   States!

   It is customary for service providers to do maintenance activities in
   the wee hours, to minimize impacting users.

   -Joshua- Hide quoted text -

  - Show quoted text -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Maintenance Timing

2009-08-18 Thread Ben Nevile

+1 for more advance warning, please.

Ben



On Aug 17, 11:52 am, Joshua Smith joshuaesm...@charter.net wrote:
 I really appreciate that you are warning us about the maintenance.
 However, next time, I'd ask:

 1) More warning, if possible; and,

 2) Do it sometime other than the middle of the day in the United
 States!

 It is customary for service providers to do maintenance activities in
 the wee hours, to minimize impacting users.

 -Joshua
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: All Read Please: Geographical Request Latency

2009-08-17 Thread Ben Nevile

Plugged in to DSL I get ~150ms.

Ben


On Aug 14, 10:13 am, Martyn martyn.cutc...@googlemail.com wrote:
 Hi Ben,

 Clearly all parts of the network add latency. The bit that adds the
 most is the most significant - wherever it may be.  The claimed
 AppEngine latency in the status figures (for Java and Python) is based
 on a round-trip http request, I guess that is not necessarily a
 browser test.  Hopefully, it is not an http request from somewhere
 else on Googles network.

 But no, any one figure cannot indicate a problem with app engine
 responsiveness, which is why I am looking for a number of responses
 worldwide to provide some useful info.

 I've yet to see anyone outside the states to report a sub 100ms
 response.

 In your case, your cellular modem may be responsible for a hefty
 slice.  It would be interesting if you could also plug into a decent
 network and do the same test.

 - Martyn
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: All Read Please: Geographical Request Latency

2009-08-14 Thread Ben Nevile

Am I stupid, or does this test depend as much or more on the last
mile and its transport as it does on the underlying network
topology?  I'm in Vancouver, Canada, and I'm getting results between
300ms and 600ms, but I'm also using a cellular modem right now.

Ben

On Aug 14, 7:09 am, ajacks504 adam.p.jack...@gmail.com wrote:
 New Orleans, LA : 170 - 185 ms

 On Aug 14, 7:12 am, Martyn martyn.cutc...@googlemail.com wrote:

  Okay, just spoke with a network expert.

  He suggests that Google are any-casting the DNS.  So the pet-
  software ping is not really doing a round-trip to the app-server.
  Where perhaps the appspot.com ping is.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Sharing an app between different Google apps domains

2009-08-11 Thread Ben Nevile

Hi Nick,

Hooks are cool!  Thanks for bringing my attention to them.

Can you expand a bit on how one might handle a multi-account scenario
using hooks?  All I can imagine is a datastore hook that is smart
enough to insert the proper filter on a query before it is executed --
is that what you had in mind?

Ben


On Aug 10, 1:30 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 On Mon, Aug 10, 2009 at 9:02 AM, Sudhirsudhi...@gmail.com wrote:

  Heres the background - I'm developing an app, lets say accounting
  software, that I'd like to sell to organizations using Google Apps.

  How do I manage deployments? If I just create one app, it is possible
  for all my clients to import it into their dashboard and use it like
  their own app?

 Yes, though currently their users will have to have Google accounts
 (you can create a Google account for any email address by going 
 tohttps://www.google.com/accounts/NewAccount) in order to sign in, since
 you can only set your app to work with Google accounts or one specific
 domain's accounts. Alternately, you can work around this limitation by
 using OpenID, but this is rather cumbersome.



  I don't want to create, maintain and update a separate app for each
  and every one of my customers... how can I share the app with my
  client domains keeping a separate datastore and auth setup for each of
  them?

 If you're using the approach outlined above, you'll need to enforce
 separation in your datastore yourself - for example, by restricting
 records based on the Host header the site is accessed through, and/or
 the hostname of the user who's logged in. One way to do this is with
 Hooks:http://code.google.com/appengine/articles/hooks.html

 -Nick Johnson



  Sudhir

 --
 Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: URIs and the Current Load Table

2009-07-28 Thread Ben Nevile

done.  here's the link:

http://code.google.com/p/googleappengine/issues/detail?id=1912



On Jul 27, 7:02 pm, jonathan jricket...@gmail.com wrote:
 I also have the same problem. I remember asking about it a while ago
 and was told that it was a known shortcoming. I forgot to log it at
 the time as a defect. If you log it, I'm definitely interested and
 will star it.

 j

 On Jul 28, 10:43 am, BenNevileben.nev...@gmail.com wrote:



  Hi,

  I have an app where users create an account, and then can access the
  service throughwww.mywebsite.com/account_id.  Awesome!  But not so
  awesome for the Current Load table that's on my dashboard, since
  having the account_id as part of the URI and not a parameter means
  that it doesn't aggregate stats in a meaningful way.  No one URI has
  much of a meaningful load, even though something like /.*/settings
  might account for 50% of all the requests.

  Am I stupid to be structuring my URIs this way?  It seems reasonable
  to me.  It would be boss if this URI table was based on the mappings
  we declare in our app.

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] URIs and the Current Load Table

2009-07-27 Thread Ben Nevile

Hi,

I have an app where users create an account, and then can access the
service through www.mywebsite.com/account_id.  Awesome!  But not so
awesome for the Current Load table that's on my dashboard, since
having the account_id as part of the URI and not a parameter means
that it doesn't aggregate stats in a meaningful way.  No one URI has
much of a meaningful load, even though something like /.*/settings
might account for 50% of all the requests.

Am I stupid to be structuring my URIs this way?  It seems reasonable
to me.  It would be boss if this URI table was based on the mappings
we declare in our app.

Ben


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: memcache in remote api

2009-07-13 Thread Ben Nevile

So I finally decided I needed to figure out how to make it happen.
Here's a recipe for anyone else who wants to be able to access
memcache via remote_api:

1. copy the contents of /google/appengine/ext/remote_api/handler.py
into a new local file.  i called mine remote_api_handler.py.  then
change your app.yaml to point to this new local file.

2. add the following import to remote_api_handler.py:
from google.appengine.api.memcache import memcache_service_pb

3. in remote_api_handler.py, find the SERVICE_PB_MAP dictionary.  add
the following as a new service:
'memcache': {
  'Get': (memcache_service_pb.MemcacheGetRequest,
memcache_service_pb.MemcacheGetResponse),
  'Delete': (memcache_service_pb.MemcacheDeleteRequest,
memcache_service_pb.MemcacheDeleteResponse),
  'Increment': (memcache_service_pb.MemcacheIncrementRequest,
memcache_service_pb.MemcacheIncrementResponse),
  'FlushAll': (memcache_service_pb.MemcacheFlushRequest,
memcache_service_pb.MemcacheFlushResponse),
  'Stats': (memcache_service_pb.MemcacheStatsRequest,
memcache_service_pb.MemcacheStatsResponse),
  'Set': (memcache_service_pb.MemcacheSetRequest,
memcache_service_pb.MemcacheSetResponse),
}

4. run your tests to make sure you haven't introduced any errors into
your code.  deploy onto your server.

5. open /google/appengine/ext/remote_api/remote_api_stub.py.  In the
ConfigureRemoteDatastore method, add the following two lines at the
end:
  stub = RemoteStub(server, path)
  apiproxy_stub_map.apiproxy.RegisterStub('memcache', stub)

(feel free to reimplement this in a less hacky way if you like... the
method could be renamed to include datastore and memcache
configuration.)

6. launch your remote_api console. enjoy access to memcache!



Hope that helps someone out.
Ben

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Request: Task Queue quota increase

2009-06-24 Thread Ben Nevile

Seconded.  For the queues to be useful for some of the applications I
currently use client-side polling to accomplish, the daily number of
tasks would have to be several orders of magnitude larger (100M).

Ben

On Jun 24, 10:33 am, vivpuri vivpu...@gmail.com wrote:
 Right now the Task Queue quota is set to 10k tasks, which is rather
 small. Can this be bumped to at least 100k?

 Thanks
 Vivek
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity at the front of index, should not be there, WILL NOT MOVE.

2009-06-17 Thread Ben Nevile

Great, yes, now it is fixed.  Can you provide any insight into the
nature of the problem?  Should I be on guard for this type of thing in
the future?  What level of trust should I have for the indices?

Ben



On Jun 17, 8:29 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Ben,

 We've taken another look; _now_ it should be fixed. :)

 -Nick Johnson





 On Tue, Jun 16, 2009 at 1:02 AM, Ben Nevile ben.nev...@gmail.com wrote:

  Hi Nick,

  Sadly it is NOT fixed. Here is the URL again:

 http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

  Ben

  On Jun 15, 10:14 am, Nick Johnson (Google) nick.john...@google.com
  wrote:
   Hi Ben,

   Thanks for your patience, and for your consideration in leaving the app
   as-is. We have all the data we need, and your index should now be fixed.

   -Nick Johnson

   On Mon, Jun 15, 2009 at 4:21 PM, Ben Nevile ben.nev...@gmail.com
  wrote:

bumping. app still in a broken state, but I need some resolution wrt
this fundamental problem.

Ben

On Jun 11, 10:23 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Okay, I'm just going to leave my app in a broken state so that
  someone
 from Google can look at this.  I've been dealing with these broken
 indices for a long time.  Shouldn't we be able to trust the indices?

 Here's a link in the datastore explorer to the GQL query that's
  messed
 up.  Hopefully a Googler will have the admin access to see this view.

 http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

 Note that the entity with ID/Name 128120 is the first entity in the
 index even though there are thousands of other entities that should
  be
 in front of it.  I have tried re-saving this entity many times, with
 the same and then different values, but it is stuck at this position
 in the index.  In the past the only way I've been able to fix the
 index was by deleting the entity -- not cool.

 Please would someone from Google tell us what's up here, and if
  you're
 really kind, copy the message to my via email as well?

 Thanks in advance.  I 3 GAE and want to 3 it even more.
 Ben

 --
 Nick Johnson, App Engine Developer Programs Engineer
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity at the front of index, should not be there, WILL NOT MOVE.

2009-06-17 Thread Ben Nevile

In fact, on second look, the problem has NOT been fixed.

It has been fixed for the entity with id 128120, but now a different
entity with id 71134 has taken its place. like 128120, 71134 does not
belong at the top of this index.

The application relies on the integrity of this index.  Please help me
understand why this is happening.

Ben





On Jun 17, 8:29 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Ben,

 We've taken another look; _now_ it should be fixed. :)

 -Nick Johnson





 On Tue, Jun 16, 2009 at 1:02 AM, Ben Nevile ben.nev...@gmail.com wrote:

  Hi Nick,

  Sadly it is NOT fixed. Here is the URL again:

 http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

  Ben

  On Jun 15, 10:14 am, Nick Johnson (Google) nick.john...@google.com
  wrote:
   Hi Ben,

   Thanks for your patience, and for your consideration in leaving the app
   as-is. We have all the data we need, and your index should now be fixed.

   -Nick Johnson

   On Mon, Jun 15, 2009 at 4:21 PM, Ben Nevile ben.nev...@gmail.com
  wrote:

bumping. app still in a broken state, but I need some resolution wrt
this fundamental problem.

Ben

On Jun 11, 10:23 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Okay, I'm just going to leave my app in a broken state so that
  someone
 from Google can look at this.  I've been dealing with these broken
 indices for a long time.  Shouldn't we be able to trust the indices?

 Here's a link in the datastore explorer to the GQL query that's
  messed
 up.  Hopefully a Googler will have the admin access to see this view.

 http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

 Note that the entity with ID/Name 128120 is the first entity in the
 index even though there are thousands of other entities that should
  be
 in front of it.  I have tried re-saving this entity many times, with
 the same and then different values, but it is stuck at this position
 in the index.  In the past the only way I've been able to fix the
 index was by deleting the entity -- not cool.

 Please would someone from Google tell us what's up here, and if
  you're
 really kind, copy the message to my via email as well?

 Thanks in advance.  I 3 GAE and want to 3 it even more.
 Ben

 --
 Nick Johnson, App Engine Developer Programs Engineer
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity at the front of index, should not be there, WILL NOT MOVE.

2009-06-15 Thread Ben Nevile

bumping. app still in a broken state, but I need some resolution wrt
this fundamental problem.

Ben


On Jun 11, 10:23 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Okay, I'm just going to leave my app in a broken state so that someone
 from Google can look at this.  I've been dealing with these broken
 indices for a long time.  Shouldn't we be able to trust the indices?

 Here's a link in the datastore explorer to the GQL query that's messed
 up.  Hopefully a Googler will have the admin access to see this view.

 http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

 Note that the entity with ID/Name 128120 is the first entity in the
 index even though there are thousands of other entities that should be
 in front of it.  I have tried re-saving this entity many times, with
 the same and then different values, but it is stuck at this position
 in the index.  In the past the only way I've been able to fix the
 index was by deleting the entity -- not cool.

 Please would someone from Google tell us what's up here, and if you're
 really kind, copy the message to my via email as well?

 Thanks in advance.  I 3 GAE and want to 3 it even more.
 Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity at the front of index, should not be there, WILL NOT MOVE.

2009-06-15 Thread Ben Nevile

Hi Nick,

Sadly it is NOT fixed. Here is the URL again:

http://appengine.google.com/datastore/explorer?app_id=friendmonthviewby=gqlkind=FacebookUserquery=SELECT+*+FROM+FacebookUser+where+expired_fotm%3DTrue+order+by+last_nudged_at


Ben



On Jun 15, 10:14 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Ben,

 Thanks for your patience, and for your consideration in leaving the app
 as-is. We have all the data we need, and your index should now be fixed.

 -Nick Johnson



 On Mon, Jun 15, 2009 at 4:21 PM, Ben Nevile ben.nev...@gmail.com wrote:

  bumping. app still in a broken state, but I need some resolution wrt
  this fundamental problem.

  Ben

  On Jun 11, 10:23 pm, Ben Nevile ben.nev...@gmail.com wrote:
   Okay, I'm just going to leave my app in a broken state so that someone
   from Google can look at this.  I've been dealing with these broken
   indices for a long time.  Shouldn't we be able to trust the indices?

   Here's a link in the datastore explorer to the GQL query that's messed
   up.  Hopefully a Googler will have the admin access to see this view.

  http://appengine.google.com/datastore/explorer?app_id=friendmonthvie...

   Note that the entity with ID/Name 128120 is the first entity in the
   index even though there are thousands of other entities that should be
   in front of it.  I have tried re-saving this entity many times, with
   the same and then different values, but it is stuck at this position
   in the index.  In the past the only way I've been able to fix the
   index was by deleting the entity -- not cool.

   Please would someone from Google tell us what's up here, and if you're
   really kind, copy the message to my via email as well?

   Thanks in advance.  I 3 GAE and want to 3 it even more.
   Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] entity at the front of index, should not be there, WILL NOT MOVE.

2009-06-11 Thread Ben Nevile

Okay, I'm just going to leave my app in a broken state so that someone
from Google can look at this.  I've been dealing with these broken
indices for a long time.  Shouldn't we be able to trust the indices?

Here's a link in the datastore explorer to the GQL query that's messed
up.  Hopefully a Googler will have the admin access to see this view.

http://appengine.google.com/datastore/explorer?app_id=friendmonthviewby=gqlkind=FacebookUserquery=SELECT+*+FROM+FacebookUser+where+expired_fotm%3DTrue+order+by+last_nudged_at

Note that the entity with ID/Name 128120 is the first entity in the
index even though there are thousands of other entities that should be
in front of it.  I have tried re-saving this entity many times, with
the same and then different values, but it is stuck at this position
in the index.  In the past the only way I've been able to fix the
index was by deleting the entity -- not cool.

Please would someone from Google tell us what's up here, and if you're
really kind, copy the message to my via email as well?

Thanks in advance.  I 3 GAE and want to 3 it even more.
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: super high failure rate for datastore

2009-06-02 Thread Ben Nevile

tick tock. four and a half hours of these failures now.  I've had to
take the site down and put up a sorry! message.

ben



On Jun 1, 9:35 pm, Ben Nevile ben.nev...@gmail.com wrote:
 About 90 minutes ago one of my applications began failing on somewhere
 close to 100% of its datastore API calls.  The app is nowhere close to
 exceeding any quotas, and I haven't made any changes in the code in
 the last ... I don't know, at least 12 hours.

 Que pasa Googlito?  The app id is butterpicks.

 Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: super high failure rate for datastore

2009-06-02 Thread Ben Nevile

Hi Brett,

I'm happy to provide you with some details. I think the most important
detail though is that the application has been running for months with
more or less the same access pattern, and last night's behaviour was
anomalous.

The exceptions were all Timeouts.

The application gets between 1 and 25 req/second, depending on the
time of day.  The number of datastore requests ranges from 0 to 10
with the median probably around 1.  The amount of data being returned
is always small... just simple html or short json strings.  There's no
contention.  I am using memcache yes, but the timeouts always came
from datastore operations.

Ben



On Jun 2, 11:58 am, Brett (Google) brett-appeng...@google.com
wrote:
 Hi Ben,

 On Jun 1, 9:35 pm, Ben Nevile ben.nev...@gmail.com wrote:

  About 90 minutes ago one of my applications began failing on somewhere
  close to 100% of its datastore API calls.  The app is nowhere close to
  exceeding any quotas, and I haven't made any changes in the code in
  the last ... I don't know, at least 12 hours.

 Could you provide some more information about the source of your
 Datastore issues? What exceptions are you getting (what do you see in
 your application logs)? What's your access pattern like (requests per
 second, data size per request, number of Datastore calls per request)?
 Do you have contention on any entities? Do you have a high rate of
 querying on a single set of entities? Are you using memcache at all?

 All of these issues can affect your application individually while the
 rest of the system works just fine. It's important to keep track of
 your application's errors in its logs. It's also useful to load test
 your application before you get load to find the source of any sources
 of contention before they happen.

 -Brett
 App Engine Team
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] super high failure rate for datastore

2009-06-01 Thread Ben Nevile

About 90 minutes ago one of my applications began failing on somewhere
close to 100% of its datastore API calls.  The app is nowhere close to
exceeding any quotas, and I haven't made any changes in the code in
the last ... I don't know, at least 12 hours.

Que pasa Googlito?  The app id is butterpicks.


Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] inconsistent index after building

2009-05-27 Thread Ben Nevile

hey there,

the other night I added a couple of indexes to an model that has
hundreds of thousands of entities.  I was pleasantly surprised when I
woke up the next morning to find the magic serving beside the new
indexes!  hooray!

this afternoon I was debugging some code and I discovered that the
created indices are pretty spotty.  they're downright bad actually.
it appears as though many many entities were not inserted into the new
index.  I don't have an accurate estimate of what percentage, but if I
had to guess I would say north of 20%.  not so good.

Here's one of the indices I added:

- kind: FacebookUser
  properties:
  - name: authorized
  - name: expired_fotm
  - name: last_nudged_at


authorized and expired_fotm are boolean, and last_nudged_at is
datetime.

so I'm sweeping through now and db.put()ting the whole collection of
these guys to get them added to the index.  hopefully this will fix
things.  but isn't that what is supposed to happen already when adding
an index?

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: memcache in remote api

2009-05-27 Thread Ben Nevile

Hi Nick,

Thanks for your reply.

Here's a typical use case for me.  I often have methods like this:


def facebook_user_cache_key(uid):
  return str(uid) + _fbuser

def get_facebook_user(uid):
  u = memcache.get(facebook_user_cache_key(uid))
  if u == None:
uid = int(uid)
u = FacebookUser.all().filter('uid = ',uid).get()
if not u:
  u = FacebookUser(uid=uid)
  u.put()
memcache.set(facebook_user_cache_key(uid),u)
  return u


and I often get support emails from users, and would often like to be
able to dive in to the remote api console and load up their entity by
typing in get_facebook_user(THEIR_UID).

this is a trivial example - in some of the apps I have several layers
of entity that need to be loaded.  I already have great convenience
methods for dealing with this process, but I can't use them in remote
api because they rely on memcache.  so either I implement a
dont_use_memcache param, or else I write parallel methods that don't
use memcache.

or, another even simpler use case -- sometimes I update an entity, and
then want to delete any affected cache.


but HEY, are you calling me not dedicated?  them's fighting words.
will break open the source.

Ben



On May 26, 4:31 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Ben,

 Out of curiosity, what's your use case for accessing Memcache over
 remote_api? The main reason we didn't support it automatically is
 because there didn't seem to be any obvious reason why you'd want to
 use Memcache remotely.

 Also, note that a dedicated user can actually add support for extra
 services to remote_api relatively easily. :)

 -Nick Johnson



 On Mon, May 25, 2009 at 10:17 PM, BenNevileben.nev...@gmail.com wrote:

  Just wanted to mention that I created a new issue requesting support
  for memcache in the remote api console.  I searched but didn't find
  any existing similar requests.

 http://code.google.com/p/googleappengine/issues/detail?id=1596

  Would be a big help.  If this is of interest to you, please give 'er a
  star.

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: google account frustrations

2009-05-27 Thread Ben Nevile

ps. was never able to get any consistent behaviour here.  have
resorted to using Firefox with just one single google account and
using Safari for everything else.  (I jump between three google
accounts all day - 2 related to an apps domain, one a regular
gmail.com.)

Ben


On May 5, 7:54 am, Ben Nevile ben.nev...@gmail.com wrote:
 Hi Jason,

 I am running Safari 4.  Clearing cookies doesn't work.  I will test
 Firefox today.

 I don't think I'm doing anything in particular when I'm redirected.
 I'd estimate that the amount of time before I'm redirected is about
 5-10 mins on average.

 On May 4, 4:13 pm, Jason (Google) apija...@google.com wrote:



  Hi Ben. Which browser are you running? Have you tried using a different
  browser or clearing your cookies to see if this flippant behavior stops?

  Just to get a better idea of your problem, are you doing anything in
  particular between when you're able to work in the Admin Console and when
  you're being redirected to the standard sign-in? Or is it just one click
  you're fine and the next you're redirected? How many minutes on average
  before this happens?

  - Jason

  On Sat, May 2, 2009 at 1:09 PM, BenNevileben.nev...@gmail.com wrote:

   I administer most of my apps through a Google Apps account:
   b...@mainsocial.com.  Recently the application console started asking
   me FREQUENTLY to log in.  It bounces me to the regular sign in with
   your google account page.  If I just go directly back to
  http://appengine.google.com/a/mainsocial.com, it doesn't ask me to log
   in, and I can get back to the app console for a few minutes.

   I used to be able to mess around all day in the application console
   without this happening.  What changed?

   Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] memcache in remote api

2009-05-25 Thread Ben Nevile

Just wanted to mention that I created a new issue requesting support
for memcache in the remote api console.  I searched but didn't find
any existing similar requests.

http://code.google.com/p/googleappengine/issues/detail?id=1596


Would be a big help.  If this is of interest to you, please give 'er a
star.

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: 'Entity' object has no attribute '_Entity__unindexed_properties'

2009-05-06 Thread Ben Nevile

I am getting the same bug. 950 times, so almost 0% of the 323.35k
requests for that handler served since midnight. but a bug
nonetheless.

Nothing too complicated is happening. An entity is being loaded into
memory, through a standard memcache call that falls back to a
datastore call.  The values are being updated, then a put() is
called.  It's on the put() that I'm seeing the error.

Here's the trace:

Traceback (most recent call last):
  File /base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.py, line 501, in __call__
handler.get(*groups)
  File controllers/rankings.py, line 114, in get
friend_team.put()
  File models/team.py, line 31, in put
db.Model.put(self)
  File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 695, in put
return datastore.Put(self._entity)
  File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 155, in Put
req.entity_list().extend([e._ToPb() for e in entities])
  File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 513, in _ToPb
name in self.__unindexed_properties):
AttributeError: 'Entity' object has no attribute
'_Entity__unindexed_properties'



Ben




On May 6, 5:19 am, DarkCoiote darkcoi...@gmail.com wrote:
 Doing some more tests, and now the requests handler that
 I was testing stopped giving errors...

 My best guess is that someone (you?) is fixing the bug.
 My fears is that the bug is really nasty and difficult to find/
 reproduce and mutable.

 Anyway, thanks a lot!

 On May 6, 9:11 am, DarkCoiote darkcoi...@gmail.com wrote:



  Actually, the problem still exists... it is only occurring in others
  requests handlers...

  And the new point of error is also a updating put

  What does this error actually means?

  On May 6, 9:00 am, DarkCoiote darkcoi...@gmail.com wrote:

   Well, It was working before, and now is working again!

   Actually, theentityis being updated. But here is the code anyway

     person = get_person(owner)
       if person is None:
         self.response.out.write(minjson.write(None))
         return

       # converting to dict()                        - to do a
   json.write at the end: is there a json.write to datastore models?
       tmpPerson['id'] = person.id
       
       tmpPerson['shipsLost'] = person.shipsLost

       # update access and reload
       if owner == viewer:
           person.name = urllib.unquote(self.request.get('name'))
           person.rank = get_rank(person.score, person.rank)
     Crazy math to get some new numbers
           now = datetime.now() + timedelta(hours=-3)
           ..
           reloadShots = 0 +  ((now.hour+(24*dayDiff))*2)-(before.hour*2)
   # 2 per hour

           person.lastAccess = datetime.now()

           person.availableShots = person.availableShots + reloadShots
           if person.availableShots  person.maxShots:
             person.availableShots = person.maxShots
           if person.availableShots  0:
             person.availableShots = 0

           person.put()

   I guess is all simple stuff that should not give any errors (as it
   didn't give before and is not giving now)

   Thanks a lot for the quick - almost instant - reply

   On May 6, 7:10 am, Nick Johnson (Google) nick.john...@google.com
   wrote:

Hi,

Can you please paste the code showing how you are constructing the
   entityyou're trying to put?

-Nick Johnson

On Wed, May 6, 2009 at 11:06 AM, DarkCoiote darkcoi...@gmail.com 
wrote:

 All of sudden I started getting this error for all of the request to
 app. that requires datastore puts.

 Someonehastweaked something. lol

 The full log:

 'Entity'objecthasnoattribute'_Entity__unindexed_properties'
 Traceback (most recent call last):
  File /base/python_lib/versions/1/google/appengine/ext/webapp/
 __init__.py, line 501, in __call__
    handler.get(*groups)
  File /base/data/home/apps/naval-war/1.333185504277530133/main.py,
 line 268, in get
    person.put()
  File /base/python_lib/versions/1/google/appengine/ext/db/
 __init__.py, line 695, in put
    return datastore.Put(self._entity)
  File /base/python_lib/versions/1/google/appengine/api/
 datastore.py, line 155, in Put
    req.entity_list().extend([e._ToPb() for e in entities])
  File /base/python_lib/versions/1/google/appengine/api/
 datastore.py, line 513, in _ToPb
    name in self.__unindexed_properties):
 AttributeError: 'Entity'objecthasnoattribute
 '_Entity__unindexed_properties'

 Line of main.py 268:
  person.put()

 Person model:

 class Person(db.Model):
  id = db.StringProperty()
  name = db.StringProperty(default=) # for use with global rank
  score = db.IntegerProperty(default=0)
  rank = db.IntegerProperty(default=0) # military rank - from 0 to 17
  globalRankingPos = db.IntegerProperty(default=0) # global ranking
 position
  

[google-appengine] Re: google account frustrations

2009-05-05 Thread Ben Nevile

Hi Jason,

I am running Safari 4.  Clearing cookies doesn't work.  I will test
Firefox today.

I don't think I'm doing anything in particular when I'm redirected.
I'd estimate that the amount of time before I'm redirected is about
5-10 mins on average.


On May 4, 4:13 pm, Jason (Google) apija...@google.com wrote:
 Hi Ben. Which browser are you running? Have you tried using a different
 browser or clearing your cookies to see if this flippant behavior stops?

 Just to get a better idea of your problem, are you doing anything in
 particular between when you're able to work in the Admin Console and when
 you're being redirected to the standard sign-in? Or is it just one click
 you're fine and the next you're redirected? How many minutes on average
 before this happens?

 - Jason



 On Sat, May 2, 2009 at 1:09 PM, Ben Nevile ben.nev...@gmail.com wrote:

  I administer most of my apps through a Google Apps account:
  b...@mainsocial.com.  Recently the application console started asking
  me FREQUENTLY to log in.  It bounces me to the regular sign in with
  your google account page.  If I just go directly back to
 http://appengine.google.com/a/mainsocial.com, it doesn't ask me to log
  in, and I can get back to the app console for a few minutes.

  I used to be able to mess around all day in the application console
  without this happening.  What changed?

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Your application is exceeding a quota: Workflow Backend Index Task Count

2009-05-03 Thread Ben Nevile

bumpty bump.  i have the same quota error message coming up now.  app
seems to be working jes fine.  i created and vacuumed indexes today as
well.

Ben

On Apr 13, 11:45 am, Paul Kinlan paul.kin...@gmail.com wrote:
 Hi,

 I am just curious as to what the following message in the dashboard means:
 Your application is exceeding a quota: Workflow Backend Index Task Count?

 It doesn't appear to be affecting my app. Today I have vacuumed my indexes
 (twice as I noticed that one index was had an = in the one of the property
 names) and I have also added another index.

 Paul
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] google account frustrations

2009-05-02 Thread Ben Nevile

I administer most of my apps through a Google Apps account:
b...@mainsocial.com.  Recently the application console started asking
me FREQUENTLY to log in.  It bounces me to the regular sign in with
your google account page.  If I just go directly back to
http://appengine.google.com/a/mainsocial.com, it doesn't ask me to log
in, and I can get back to the app console for a few minutes.

I used to be able to mess around all day in the application console
without this happening.  What changed?

Ben

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: GAEUnit problems w/ SDK 1.2.1?

2009-04-24 Thread Ben Nevile

Hi Matthew -

GAEUnit is trying to create a separate dataspace for the tests so that
data for the development env isn't affected by running tests.  This
isn't an issue for me since I don't really use the dev server for
anything but tests, so I'll just comment out that business, but I
think for some people it will be a big deal.

thanks for your answer!
Ben



On Apr 24, 10:03 am, Matthew Blain matthew.bl...@google.com wrote:
 Hi Ben,

 We've added this restriction to make the dev_appserver work more like
 the live App Engine datastore. I don't know exactly what GAEUnit is
 doing, but it looks like there's a line like this in there:

 datastore_file_stub.DatastoreFileStub('GAEUnitDataStore', None, None)

 It should create the stub using the correct app id (e.g.
 'butterpool'), which may be os.environ['APPLICATION_ID'] at that
 point.

 --Matthew

 On Apr 23, 1:27 pm, Ben Nevile ben.nev...@gmail.com wrote:



  since updating to SDK 1.2.1, GAEUnit is not able to access the
  datastore.  I was using GAEunit 1.2.2 before - just updated to 1.2.6,
  but no love.  Below is the error I get.  Any tips?

  Ben

  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/ext/db/__init__.py, line 1346, in get
      results = self.fetch(1)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/ext/db/__init__.py, line 1390, in fetch
      raw = self._get_query().Get(limit, offset)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/datastore.py, line 942, in Get
      return self._Run(limit, offset)._Next(limit)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/datastore.py, line 883, in _Run
      apiproxy_stub_map.MakeSyncCall('datastore_v3', 'RunQuery', pb,
  result)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/apiproxy_stub_map.py, line 68, in MakeSyncCall
      apiproxy.MakeSyncCall(service, call, request, response)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/apiproxy_stub_map.py, line 240, in MakeSyncCall
      stub.MakeSyncCall(service, call, request, response)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/datastore_file_stub.py, line 413, in
  MakeSyncCall
      response)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/apiproxy_stub.py, line 80, in MakeSyncCall
      method(request, response)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/datastore_file_stub.py, line 518, in
  _Dynamic_RunQuery
      self.__ValidateAppId(app)
    File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
  GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
  google/appengine/api/datastore_file_stub.py, line 237, in
  __ValidateAppId
      'app %s cannot access app %s\'s data' % (self.__app_id, app_id))
  BadRequestError: app GAEUnitDataStore cannot access app butterpool's
  data
  ERROR
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] GAEUnit problems w/ SDK 1.2.1?

2009-04-23 Thread Ben Nevile

since updating to SDK 1.2.1, GAEUnit is not able to access the
datastore.  I was using GAEunit 1.2.2 before - just updated to 1.2.6,
but no love.  Below is the error I get.  Any tips?

Ben


File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/db/__init__.py, line 1346, in get
results = self.fetch(1)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/db/__init__.py, line 1390, in fetch
raw = self._get_query().Get(limit, offset)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore.py, line 942, in Get
return self._Run(limit, offset)._Next(limit)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore.py, line 883, in _Run
apiproxy_stub_map.MakeSyncCall('datastore_v3', 'RunQuery', pb,
result)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/apiproxy_stub_map.py, line 68, in MakeSyncCall
apiproxy.MakeSyncCall(service, call, request, response)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/apiproxy_stub_map.py, line 240, in MakeSyncCall
stub.MakeSyncCall(service, call, request, response)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore_file_stub.py, line 413, in
MakeSyncCall
response)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/apiproxy_stub.py, line 80, in MakeSyncCall
method(request, response)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore_file_stub.py, line 518, in
_Dynamic_RunQuery
self.__ValidateAppId(app)
  File /Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/api/datastore_file_stub.py, line 237, in
__ValidateAppId
'app %s cannot access app %s\'s data' % (self.__app_id, app_id))
BadRequestError: app GAEUnitDataStore cannot access app butterpool's
data
ERROR
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: dumb guy trying to access datastore remotely with remote_api

2009-04-21 Thread Ben Nevile

more info about this issue: if I am logged into my google apps account
I am not able to successfully add my regular gmail account as a
developer.  If I fire up another browser it works.  I guess it's hard
to program around all these account edge cases.

Ben


On Apr 20, 4:40 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Hi Nick - yes, this is the problem.  I suspected this, and yesterday I
 tried to add ben.nev...@gmail.com as a developer.  I went through the
 whole song and dance, and then after agreeing to the terms of service
 it took me to the newbie page (create an app!), and the pending
 developer account of ben.nev...@gmail.com was removed.

 So I just went to do it again to reproduce the steps exactly for you,
 and this time it worked!  and I can successfully access my data using
 the remote API!  Awesome, thank you.

 One other follow-up question: in my efforts to get this working, I
 took the admin restriction off of the /remote_api handler in
 app.yaml.  But it still asked for a login and rejected my attempts.
 Maybe that's just a problem with the client script?

 Ben

 On Apr 20, 12:42 pm, Nick Johnson nick.john...@google.com wrote:



  Hi Ben,

  Is your admin account a Google Apps one (eg, not @gmail.com or
  @googlemail.com)? Authentication using these accounts is somewhat
  broken. What you can do is create a gmail.com account specifically for
  this, and add it as another admin for the app, then authenticate using
  that.

  -Nick Johnson

  2009/4/18 BenNevileben.nev...@gmail.com:

   I was able to get appengine_console.py to launch in OS X with the help
   of this message:

  http://groups.google.com/group/google-appengine/browse_thread/thread/...

   When I try and fetch data, it asks for my name and password.  My admin
   name and pass come back as invalid.  Is there anything clever I need
   to do to get this to work?  I only have one admin account on this
   particular app, sanity checked by logging in and out to the admin
   console.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: dumb guy trying to access datastore remotely with remote_api

2009-04-20 Thread Ben Nevile

Hi Nick - yes, this is the problem.  I suspected this, and yesterday I
tried to add ben.nev...@gmail.com as a developer.  I went through the
whole song and dance, and then after agreeing to the terms of service
it took me to the newbie page (create an app!), and the pending
developer account of ben.nev...@gmail.com was removed.

So I just went to do it again to reproduce the steps exactly for you,
and this time it worked!  and I can successfully access my data using
the remote API!  Awesome, thank you.

One other follow-up question: in my efforts to get this working, I
took the admin restriction off of the /remote_api handler in
app.yaml.  But it still asked for a login and rejected my attempts.
Maybe that's just a problem with the client script?

Ben



On Apr 20, 12:42 pm, Nick Johnson nick.john...@google.com wrote:
 Hi Ben,

 Is your admin account a Google Apps one (eg, not @gmail.com or
 @googlemail.com)? Authentication using these accounts is somewhat
 broken. What you can do is create a gmail.com account specifically for
 this, and add it as another admin for the app, then authenticate using
 that.

 -Nick Johnson

 2009/4/18 BenNevileben.nev...@gmail.com:





  I was able to get appengine_console.py to launch in OS X with the help
  of this message:

 http://groups.google.com/group/google-appengine/browse_thread/thread/...

  When I try and fetch data, it asks for my name and password.  My admin
  name and pass come back as invalid.  Is there anything clever I need
  to do to get this to work?  I only have one admin account on this
  particular app, sanity checked by logging in and out to the admin
  console.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-19 Thread Ben Nevile

Hi Ryan,

So I guess what you're saying here is that a put() is not atomic with
respect to all indices.  I will keep that in mind for sure, but it's
unfortunate - adds a lot of code if you have to check all the time to
actually see if an index is correct!

Ben


On Apr 16, 9:08 am, ryan ryanb+appeng...@google.com wrote:
 the discussion here is pretty on target. thanks, all!

 beyond the points already raised, i'd guess this is caused either by
 writes (puts or deletes) that returned errors, and therefore didn't
 update all indices, or an index consistency bug that we're currently
 working on:

 http://code.google.com/p/googleappengine/issues/detail?id=901

 in both cases, a successful put() on the entity in question will fix
 the problem. we're still working on issue 901, and we definitely
 expect to have that resolved soon. as for put() and delete() calls
 that return errors (like timeouts), we're always working to reduce
 those, but they'll unfortunately always be possible, so you'll want to
 plan for that.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-19 Thread Ben Nevile

Hi Matthew,

Thanks for these great ideas!

Ben


On Apr 15, 8:30 pm, Matthew Blain matthew.bl...@google.com wrote:
 Hi Ben,
 One possible way to handle this is to retrieve the top 10 items in
 your 'queue', then loop through them in a transaction until you find
 one which is not recent.

 Better yet, you can cache this query--query a bit more, say 100 items,
 save the keys into some simple structure in memcache, and consider
 using memcache.incr to track where you are in those 100 items as you
 test for the ability to pull off the queue--I assume you don't need to
 assign these in exact order to your users.

 something like this pseudocode:

 get top item from the queue():
  possiblekeys = memcache.get(queuekeys)
  index = memcache.incr(queueindex)
  if index is None or index  len(possiblekeys):
    fallback to current code which does the query of the top 1 item
  else:
    key = possiblekeys[index]
    in transaction:
      check that date is not recent
      update date
    if transaction succeeded:
      return db.get(key)
    else:
      return gettopitemfromqueue()

 then some code to fill the queue:
 fillqueue():
   possibleEntities query(500 most recent items)
   possibleKeys = [keys of possibleEntities]
   memcache.set_multi({queuekeys: possibleKeys, queueindex: 0})

 then you need to fill the queue somehow--either some sort of mechanism
 when the cached queue is almost empty (which has mild locking issues),
 or some external mechanism like cron.

 --Matthew

 On Apr 15, 1:31 pm, BenNevileben.nev...@gmail.com wrote:



  This is happening frequently today.  I think it must expose a weakness
  in the transaction code.  Hey, I know that what I'm doing with this
  code is totally lame -- basically causing many many transaction
  collisions - but clearly there's a problem here with the transaction
  code that creates the possibility of corrupting an entity.  Not not
  not not cool.

  I'm working under a harsh time constraint right now (start of hockey
  playoffs) and my users need this crappy queue I've set up, so I'm not
  currently at liberty to leave one of these entities in its corrupted
  state for very long.  But I would like to work with Google to get this
  sorted out.

  Ben

  On Apr 15, 8:04 am, BenNevileben.nev...@gmail.com wrote:

   To add another twist, now I have an entity at the top of the stack
   that is found using

         q = FacebookUser.all()
         q.filter('authorized = ',True)
         q.order('last_nudged_at')
         u = q.get()

   but, if you try to load this entity by key, as you need to do for a
   transaction, you get None back.   Trying to access that entity's
   record in the Data Viewer also threw an exception.  I was, however,
   able to delete it from the top of that index listing using the Data
   Viewer's GQL option.

   Ben

   On Apr 14, 3:38 pm, BenNevileben.nev...@gmail.com wrote:

Hi Jeff,

To clarify, the last_nudged_at property *is* changed, but the entity
stays at the top of the stack.

Ben

On Apr 14, 12:26 pm, Jeff S j...@google.com wrote:

 Hi Ben,

 If you are experiencing write contention on some of these updates,
 then that may explain why the last_nudged_at  property is not changed
 and an entity tends to stay at the top of the stack. Do you see
 datastore timeout errors in these cases (I imagine the exception may
 be hidden somewhere in the JavaScript -- server exchange)?

 Thank you,

 Jeff

 On Apr 14, 11:48 am, BenNevileben.nev...@gmail.com wrote:

  bump.  I have observed this many times over the past few days.  
  Maybe
  has to do with transactions and/or write contention, since there is 
  a
  lot of that in my app.

  Ben

  On Apr 1, 8:23 pm, BenNevileben.nev...@gmail.com wrote:

   Hi GAE team and devotees,

   I have a entity that looks like

   class FacebookUser(db.Model):
     expired_fotm =          db.BooleanProperty()
     last_nudged_at =        db.DateTimeProperty()

   and an index that looks like

   - kind: FacebookUser
     properties:
     - name: expired_fotm
     - name: last_nudged_at

   Unbeknownst to the users of my application, some javascript is
   constantly polling a worker handler that finds the first entity 
   in the
   index, operates on it, then sets last_nudged_at to the current
   time.  There are more than a million of these entities so it takes
   some time to go through them all.

   This afternoon using the Data Viewer and the GQL query SELECT * 
   FROM
   FacebookUser where expired_fotm=True order by last_nudged_at I
   observed several times that an entity would remain stuck in the
   first position in the index after it was processed.  Usually it 
   would
   remain there only for 10-30 seconds.  However this evening one 
   entity
   stayed in an incorrect index position for several hours.  I put() 
  

[google-appengine] dumb guy trying to access datastore remotely with remote_api

2009-04-18 Thread Ben Nevile

I was able to get appengine_console.py to launch in OS X with the help
of this message:

http://groups.google.com/group/google-appengine/browse_thread/thread/6bc2f759188476b2/967e636458681da0?lnk=gstq=interactive+console+remote+api#967e636458681da0


When I try and fetch data, it asks for my name and password.  My admin
name and pass come back as invalid.  Is there anything clever I need
to do to get this to work?  I only have one admin account on this
particular app, sanity checked by logging in and out to the admin
console.

thanks in advance.
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: urlfetch and another type of communication

2009-04-17 Thread Ben Nevile

And hello again.

So I woke up this morning to see that the issue had been marked
resolved, and that you determined that the spike did not affect your
customers.  I was furious!  But then I actually took the time to
carefully debug my application, and I see that in fact the problem was
mine.  I was trying to fetch a URL with an unescaped space in it.

I'm sorry that I'm such a tool.  My cred is out the window at this
point I know, but if you want to stem future frustration from and for
idiots like me, a more helpful error message would be a good place to
start.  Here's the trace that pops out now:

preTraceback (most recent call last):
  File quot;/base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.pyquot;, line 501, in __call__
handler.get(*groups)
  File quot;controllers/migration.pyquot;, line 187, in get
x = p.points100_in_between_dates(datetime(2009,4,14),datetime
(2009,6,14))
  File quot;models/player.pyquot;, line 49, in
points100_in_between_dates
stats = self.stats_in_between_dates(start_date,end_date)
  File quot;models/player.pyquot;, line 41, in
stats_in_between_dates
stats = json.read(urlfetch.fetch(url).content)
  File quot;/base/python_lib/versions/1/google/appengine/api/
urlfetch.pyquot;, line 267, in fetch
raise DownloadError(str(e))
DownloadError: ApplicationError: 2



How about BadUrlError or something like that?

Ben




On Apr 16, 10:21 pm, Ben Nevile ben.nev...@gmail.com wrote:
 So I'm aware that Google is aware that there are urlfetch issues
 tonight.  At least, that's what I'm lead to believe at the GAE System
 Status page, where the problem has been marked as Anomaly for the
 last 5 hours or so.  But I have no feedback other than that.

 Please Google, I'm sorry to whine at you, but I need a little
 feedback.  Should I be making adjustments to my software to reduce
 dependency on urlfetch?  Is there a team of people figuring out what's
 wrong, or just one lonely, frantic person?  Was there a fire?

 You are a big animal and I am just a small gnat, trying to make a
 living by riding on your back.  Perhaps that was an unwise choice.  It
 cost me more money to host at my last place, but at least in a
 situation like this I could make a phone call.

 Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] urlfetch and another type of communication

2009-04-16 Thread Ben Nevile

So I'm aware that Google is aware that there are urlfetch issues
tonight.  At least, that's what I'm lead to believe at the GAE System
Status page, where the problem has been marked as Anomaly for the
last 5 hours or so.  But I have no feedback other than that.

Please Google, I'm sorry to whine at you, but I need a little
feedback.  Should I be making adjustments to my software to reduce
dependency on urlfetch?  Is there a team of people figuring out what's
wrong, or just one lonely, frantic person?  Was there a fire?

You are a big animal and I am just a small gnat, trying to make a
living by riding on your back.  Perhaps that was an unwise choice.  It
cost me more money to host at my last place, but at least in a
situation like this I could make a phone call.

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-15 Thread Ben Nevile

To add another twist, now I have an entity at the top of the stack
that is found using

  q = FacebookUser.all()
  q.filter('authorized = ',True)
  q.order('last_nudged_at')
  u = q.get()

but, if you try to load this entity by key, as you need to do for a
transaction, you get None back.   Trying to access that entity's
record in the Data Viewer also threw an exception.  I was, however,
able to delete it from the top of that index listing using the Data
Viewer's GQL option.

Ben



On Apr 14, 3:38 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Hi Jeff,

 To clarify, the last_nudged_at property *is* changed, but the entity
 stays at the top of the stack.

 Ben

 On Apr 14, 12:26 pm, Jeff S j...@google.com wrote:



  Hi Ben,

  If you are experiencing write contention on some of these updates,
  then that may explain why the last_nudged_at  property is not changed
  and an entity tends to stay at the top of the stack. Do you see
  datastore timeout errors in these cases (I imagine the exception may
  be hidden somewhere in the JavaScript -- server exchange)?

  Thank you,

  Jeff

  On Apr 14, 11:48 am, Ben Nevile ben.nev...@gmail.com wrote:

   bump.  I have observed this many times over the past few days.  Maybe
   has to do with transactions and/or write contention, since there is a
   lot of that in my app.

   Ben

   On Apr 1, 8:23 pm, Ben Nevile ben.nev...@gmail.com wrote:

Hi GAE team and devotees,

I have a entity that looks like

class FacebookUser(db.Model):
  expired_fotm =          db.BooleanProperty()
  last_nudged_at =        db.DateTimeProperty()

and an index that looks like

- kind: FacebookUser
  properties:
  - name: expired_fotm
  - name: last_nudged_at

Unbeknownst to the users of my application, some javascript is
constantly polling a worker handler that finds the first entity in the
index, operates on it, then sets last_nudged_at to the current
time.  There are more than a million of these entities so it takes
some time to go through them all.

This afternoon using the Data Viewer and the GQL query SELECT * FROM
FacebookUser where expired_fotm=True order by last_nudged_at I
observed several times that an entity would remain stuck in the
first position in the index after it was processed.  Usually it would
remain there only for 10-30 seconds.  However this evening one entity
stayed in an incorrect index position for several hours.  I put() it
again and it properly indexed itself.

Has anyone ever observed anything like this? Let me know if I'm not
making myself clear.

Ben 3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Facebook performance issues

2009-04-15 Thread Ben Nevile

Hi Joe,

I use GAE to host a variety of Facebook Apps.

When you say the app is showing an average HTTP response time of
1300ms, where are you getting that number from?

Ben



On Apr 14, 4:44 pm, Joe joewalt...@gmail.com wrote:
 I recently launched a Facebook application using Google AE as the
 backend (http://apps.facebook.com/lawoflargenumbers/).

 Unfortunately, the app is showing an average HTTP response time of
 approx 1300 ms...which is quite poor.  Can anybody tell me if this is
 normal for Facebook apps using AE as the backend?

 thanks,

 Joe
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-15 Thread Ben Nevile

This is happening frequently today.  I think it must expose a weakness
in the transaction code.  Hey, I know that what I'm doing with this
code is totally lame -- basically causing many many transaction
collisions - but clearly there's a problem here with the transaction
code that creates the possibility of corrupting an entity.  Not not
not not cool.

I'm working under a harsh time constraint right now (start of hockey
playoffs) and my users need this crappy queue I've set up, so I'm not
currently at liberty to leave one of these entities in its corrupted
state for very long.  But I would like to work with Google to get this
sorted out.

Ben



On Apr 15, 8:04 am, Ben Nevile ben.nev...@gmail.com wrote:
 To add another twist, now I have an entity at the top of the stack
 that is found using

       q = FacebookUser.all()
       q.filter('authorized = ',True)
       q.order('last_nudged_at')
       u = q.get()

 but, if you try to load this entity by key, as you need to do for a
 transaction, you get None back.   Trying to access that entity's
 record in the Data Viewer also threw an exception.  I was, however,
 able to delete it from the top of that index listing using the Data
 Viewer's GQL option.

 Ben

 On Apr 14, 3:38 pm, BenNevileben.nev...@gmail.com wrote:



  Hi Jeff,

  To clarify, the last_nudged_at property *is* changed, but the entity
  stays at the top of the stack.

  Ben

  On Apr 14, 12:26 pm, Jeff S j...@google.com wrote:

   Hi Ben,

   If you are experiencing write contention on some of these updates,
   then that may explain why the last_nudged_at  property is not changed
   and an entity tends to stay at the top of the stack. Do you see
   datastore timeout errors in these cases (I imagine the exception may
   be hidden somewhere in the JavaScript -- server exchange)?

   Thank you,

   Jeff

   On Apr 14, 11:48 am, BenNevileben.nev...@gmail.com wrote:

bump.  I have observed this many times over the past few days.  Maybe
has to do with transactions and/or write contention, since there is a
lot of that in my app.

Ben

On Apr 1, 8:23 pm, BenNevileben.nev...@gmail.com wrote:

 Hi GAE team and devotees,

 I have a entity that looks like

 class FacebookUser(db.Model):
   expired_fotm =          db.BooleanProperty()
   last_nudged_at =        db.DateTimeProperty()

 and an index that looks like

 - kind: FacebookUser
   properties:
   - name: expired_fotm
   - name: last_nudged_at

 Unbeknownst to the users of my application, some javascript is
 constantly polling a worker handler that finds the first entity in the
 index, operates on it, then sets last_nudged_at to the current
 time.  There are more than a million of these entities so it takes
 some time to go through them all.

 This afternoon using the Data Viewer and the GQL query SELECT * FROM
 FacebookUser where expired_fotm=True order by last_nudged_at I
 observed several times that an entity would remain stuck in the
 first position in the index after it was processed.  Usually it would
 remain there only for 10-30 seconds.  However this evening one entity
 stayed in an incorrect index position for several hours.  I put() it
 again and it properly indexed itself.

 Has anyone ever observed anything like this? Let me know if I'm not
 making myself clear.

 Ben 3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Entire site down now..

2009-04-15 Thread Ben Nevile

I went to the Versions pane of the Administration section and no
version was highlighted as a default.  So I chose one and made it the
default.  Now pages are serving again.   

scared to deploy...
Ben

On Apr 15, 3:39 pm, bvelasquez bvelasq...@gmail.com wrote:
 Ok, I'm not happy it happened to you, but now know it's not something
 I did.

 Bah!

 Help Google!

 Barry

 On Apr 15, 3:38 pm, Ben Nevile ben.nev...@gmail.com wrote:



  It's not just you.  One of my apps is down too.  I was getting errors
  deploying, and now it has completely shut off.

  The timing of this is TERRIBLE for me.  Biggest few hours of the year
  for my particular application are about to happen between 4pm and 10pm
  PST tonight.  PLEASE Google help!

  Ben

  On Apr 15, 3:32 pm, bvelasquez bvelasq...@gmail.com wrote:

   Error: Not Found
   The requested URL / was not found on this server.

   My entire site is gone now.

  http://www.tapingya.comorhttp://tapingya.appspot.com

   Is this just me or a GAE issue?

   Barry
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-14 Thread Ben Nevile

bump.  I have observed this many times over the past few days.  Maybe
has to do with transactions and/or write contention, since there is a
lot of that in my app.

Ben



On Apr 1, 8:23 pm, Ben Nevile ben.nev...@gmail.com wrote:
 Hi GAE team and devotees,

 I have a entity that looks like

 class FacebookUser(db.Model):
   expired_fotm =          db.BooleanProperty()
   last_nudged_at =        db.DateTimeProperty()

 and an index that looks like

 - kind: FacebookUser
   properties:
   - name: expired_fotm
   - name: last_nudged_at

 Unbeknownst to the users of my application, some javascript is
 constantly polling a worker handler that finds the first entity in the
 index, operates on it, then sets last_nudged_at to the current
 time.  There are more than a million of these entities so it takes
 some time to go through them all.

 This afternoon using the Data Viewer and the GQL query SELECT * FROM
 FacebookUser where expired_fotm=True order by last_nudged_at I
 observed several times that an entity would remain stuck in the
 first position in the index after it was processed.  Usually it would
 remain there only for 10-30 seconds.  However this evening one entity
 stayed in an incorrect index position for several hours.  I put() it
 again and it properly indexed itself.

 Has anyone ever observed anything like this? Let me know if I'm not
 making myself clear.

 Ben 3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: entity stuck at the wrong end of the index

2009-04-14 Thread Ben Nevile

Hi Jeff,

To clarify, the last_nudged_at property *is* changed, but the entity
stays at the top of the stack.

Ben



On Apr 14, 12:26 pm, Jeff S j...@google.com wrote:
 Hi Ben,

 If you are experiencing write contention on some of these updates,
 then that may explain why the last_nudged_at  property is not changed
 and an entity tends to stay at the top of the stack. Do you see
 datastore timeout errors in these cases (I imagine the exception may
 be hidden somewhere in the JavaScript -- server exchange)?

 Thank you,

 Jeff

 On Apr 14, 11:48 am, Ben Nevile ben.nev...@gmail.com wrote:



  bump.  I have observed this many times over the past few days.  Maybe
  has to do with transactions and/or write contention, since there is a
  lot of that in my app.

  Ben

  On Apr 1, 8:23 pm, Ben Nevile ben.nev...@gmail.com wrote:

   Hi GAE team and devotees,

   I have a entity that looks like

   class FacebookUser(db.Model):
     expired_fotm =          db.BooleanProperty()
     last_nudged_at =        db.DateTimeProperty()

   and an index that looks like

   - kind: FacebookUser
     properties:
     - name: expired_fotm
     - name: last_nudged_at

   Unbeknownst to the users of my application, some javascript is
   constantly polling a worker handler that finds the first entity in the
   index, operates on it, then sets last_nudged_at to the current
   time.  There are more than a million of these entities so it takes
   some time to go through them all.

   This afternoon using the Data Viewer and the GQL query SELECT * FROM
   FacebookUser where expired_fotm=True order by last_nudged_at I
   observed several times that an entity would remain stuck in the
   first position in the index after it was processed.  Usually it would
   remain there only for 10-30 seconds.  However this evening one entity
   stayed in an incorrect index position for several hours.  I put() it
   again and it properly indexed itself.

   Has anyone ever observed anything like this? Let me know if I'm not
   making myself clear.

   Ben 3 GAE
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: cron job length

2009-04-09 Thread Ben Nevile

 Indeed there is, and it's even on our roadmap. We're not done with
 offline processing, not by a long shot.

That's great Jon, because I've not yet eliminated all the silly
javascript polling I'm doing, not by a long shot!

Ben 3 GAE


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] cron job length

2009-04-08 Thread Ben Nevile

big sloppy wet kisses to the GAE team for your lovely cron feature.
much javascript polling has been saved.

HOWEVER, since it is always beholden on the user to make a new feature
request immediately after the launch of a new feature, it would be
really great if either

1. requests made by a cron job weren't subject to the same timeout as
regular requests.  I imagine this would mess with the nice
implementation you've worked out, so another option is

2. enable us to schedule a job more often than once a minute.


on the other hand, if there is an asynchronous worker / queue api
being released shortly I may not need this functionality.

thanks again,
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: GQL join question

2009-04-07 Thread Ben Nevile

Hi Marcel,

the SQL join world saves you the hassle of thinking about all of this
stuff at write-time, but would not scale beyond a few thousand users.
if you implement everything in SQL and then end up with a very busy
site, you end up re-implementing all of your data access to work in
the BigTable way.

one nice thing about BigTable is that when you do a db.put(entities)
where entities is an array/list, it sends them away to write in
multiple threads.  so the access time is not linear with len
(entities).

sharded counter = awesome

Ben


On Apr 7, 2:48 am, Marcel Overdijk marceloverd...@gmail.com wrote:
 This would mean really a lot of updates.

 E.g.
 - User1, 1 jan 2009, 1000
 - User1, 1 feb 2009, 1100

 This would mean craeting an avarage usage of 100 for jan 2009.

 Now next time the user enters a meter reading is e.g. 1 jan 2010 with
 value 2300.
 This would mean in this case createing avaerage use valuues for feb,
 mar, apr, may, jun, jul, aug, sep, okt, nov and dec 2009.

 And als updating average usage of all users.
 In a SQL join this would be less dificult and error prone to
 implement.

 I'm wondering if my use case fits the BigTable principles and thus
 GAE, or that I would be better of with a realtional database.
 But this would mean I can't use GAE and have to go for another cloud
 solution like mor.ph

 On 7 apr, 06:36, GregF g.fawc...@gmail.com wrote:



  On Apr 7, 6:53 am, Marcel Overdijk marceloverd...@gmail.com wrote:

   - User1, 1 jan 2009, 1000
   - User1, 1 feb 2009, 1100
   - User1, 1 mar 2009, 1200
   The avarage will be 1100 ***

   But maybe the user misses registering the meter reading on 1 feb:
   - User1, 1 jan 2009, 1000
   - User1, 1 mar 2009, 1200
   The average will still be 1100 ***
   I guess the answer will be to store average values when creating (and
   updaing existing) meter reading records.
   But how should this be implemented?

  *** Corrected.

  Yes, calculate averages when you update the table. If you don't need
  to search for particular readings, you might want to store the meter
  readings in a db.ListProperty(int), with an associated db.ListProperty
  (int) of timestamps.

  I need to count various things in my app, so I define a model and
  helper functions like this:

  class Counter(db.Model):
    count=db.IntegerProperty(required=True,default=0)

  # Increment a counter
  # NB: These are functions, not class members
  def _incrementCounter(counterId,amount):
          counter=Counter.get_by_id(counterId)
          counter.count+=amount
          counter.put()
  def incrementCounter(counterId,amount=1):
          db.run_in_transaction(_incrementCounter,counterId,amount)

  I'm sure you can do something similar for your averages.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] entity stuck at the wrong end of the index

2009-04-01 Thread Ben Nevile

Hi GAE team and devotees,


I have a entity that looks like

class FacebookUser(db.Model):
  expired_fotm =  db.BooleanProperty()
  last_nudged_at =db.DateTimeProperty()

and an index that looks like

- kind: FacebookUser
  properties:
  - name: expired_fotm
  - name: last_nudged_at

Unbeknownst to the users of my application, some javascript is
constantly polling a worker handler that finds the first entity in the
index, operates on it, then sets last_nudged_at to the current
time.  There are more than a million of these entities so it takes
some time to go through them all.

This afternoon using the Data Viewer and the GQL query SELECT * FROM
FacebookUser where expired_fotm=True order by last_nudged_at I
observed several times that an entity would remain stuck in the
first position in the index after it was processed.  Usually it would
remain there only for 10-30 seconds.  However this evening one entity
stayed in an incorrect index position for several hours.  I put() it
again and it properly indexed itself.

Has anyone ever observed anything like this? Let me know if I'm not
making myself clear.

Ben 3 GAE

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Data Storage Size - multiplier per object

2009-03-18 Thread Ben Nevile

Hi Marzia,

Just want to add my voice to the chorus of people looking for a little
more transparency in terms of data storage and entities.  In my ideal
fantasy world I'd be able to see a pie chart that would break down the
percentage of storage that each of my entities was using as a fraction
of the total storage used.  Clicking on an entity's slice in the pie
would bring up another pie chart that would show the fraction of
storage used by that entity's primary store and each of its indices.
(This second chart may not be necessary once you guys have published
some more info on exactly how different properties and indices
manifest themselves on disk.)

Thanks in advance,
Ben



On Mar 13, 9:57 am, Marzia Niccolai ma...@google.com wrote:
 Hi,

 There is no 'multiplier' per se on datastore storage.  The issue is that we
 account for both the size of the data stored and the space taken by the
 indices for this data.  As such, the amount of storage you use depends
 specifically on the types of indexes your application has.

 We are working on getting better documentation together that will give you a
 good idea on how you can account for the amount of storage an entity will
 take.

 Please note that the FAQ on this subject currently is _not_ correct and we
 will be updating it.

 -Marzia

 On Tue, Mar 10, 2009 at 9:28 AM, Jonathan Ultis 
 jonathan.ul...@gmail.comwrote:





  I created a model with fixed content that requires ~250b serialized,
  including all field names, the key, and the kind name, and parent
  (None). I added 312000 of those to the datastore, for 75 megs of raw
  data. There are 8 indexable fields, The indices should require no more
  than 176 megs of additional space, if the indices don't do any sort of
  column compression. That's 250 megs of raw space.

  But, the data store reports 1GB of space used.

  That suggests perhaps 2x redundancy, plus a 50% fill rate in big
  table. Or, maybe just 4x redundancy. No idea.

  Anyhow, for now, take your raw object size including kind, key, field
  names, and field content, and multiply by 10x-15x, depending on how
  many indexable properties you have, to get your final storage size.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] scheduled maintenance

2009-03-02 Thread Ben Nevile

Hi GAE team,

I'm writing to express frustration about the scheduled maintenances of
March 2 and March 9.  I'm using GAE to host several time-dependent
games being played by tens of thousands of people.  Having datastore
writes unavailable for 30 minutes during the peak traffic time is
frustrating and expensive.  The cost to my small business will be
significant.

I acknowledge that with a globally distributed service every hour of
the day is equally bad.  I also acknowledge that downtime is a
statistical inevitability. However I would like to believe that
*scheduled* downtime is not inevitable. There must be a way for a
company with Google's resources and talent to upgrade server hardware
without taking the entire system offline for up to 30 minutes.

As a customer who's investing considerably in this platform, I'd
appreciate some insight into the mechanics of these scheduled
maintenance decisions. How often can we expect these outages? If I
were a provider of a social service based around baseball, for
instance, what recourse would I have if Google decided to perform a
scheduled maintenance during the seventh game of the World Series?

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] need more apps please.

2009-02-27 Thread Ben Nevile

Hi - I have been using GAE for a few months now and really love it.  I
am porting my collection of Ruby apps over.

Of course I'm running up against the 10-application limit.  I have
transferred some of my apps over so that the only developer is now at
a different email account.  I seem to still be at my limit.  Perhaps
this will change after some period of time?

I can't enable another account to create apps because it won't accept
my phone number (has already been used).

Can someone on the GAE team please help me out here?  I'm sure people
are working on this feature.  Time is an important detail for me - if
I can't port these Ruby apps to GAE soon I'll probably instead have to
port them to another Ruby host due to an impending end of contract
with my current hosting provider.

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: New App - wikiaata

2009-02-26 Thread Ben Nevile

really fun game!!  Nice work.

Ben



On Feb 25, 4:08 pm, indra b.indran...@gmail.com wrote:
 Hello everyone,

 Play the wikipedia game at

 http://www.wikiaata.com

 The app is powered by AppEngine and GWT was used for building the UI.
 Any feedback is very welcome.

 Cheers.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] ReferenceProperty key without loading the entity

2009-02-13 Thread Ben Nevile

I love GAE!

I know it has been discussed in the past, but I bring it up again
because I keep running into it over and over: please can we have a way
to access the key for a ReferenceProperty without loading the entity?
It's my understanding that currently the only way to do this is create
a separate StringProperty to hold the key?

I note from the documentation that An application can explicitly
db.get() the value of a ReferenceProperty (which is a Key) to test
whether the referenced entity exists.  99% of the time this is what I
want to do, but not with db.get() -- with memcache.get().

I love GAE!
Ben


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: request: versions accessed by URL param rather than prefix?

2009-01-06 Thread Ben Nevile

Dave,

When dealing with Facebook and FBML you can't do a 304 redirect.
Facebook is making the request to your server on behalf of the user.

Ben



On Jan 6, 1:35 am, David Symonds dsymo...@gmail.com wrote:
 On Tue, Jan 6, 2009 at 10:15 AM, Ben Nevile ben.nev...@gmail.com wrote:
  Sadly I cannot redirect before returning to Facebook.  I can ask
  Facebook to redirect, but if I redirect to a non-canvas page it won't
  render within the Facebook environment.  Does that make sense?

  For example, if I wanted to redirect to 2.latest.myapp.appspot.com I
  can send Facebook this directive:

  fb:redirect url='http://2.latest.myapp.appspot.com'/

 Why can't you do a normal HTTP 304 redirect?

 Dave.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] request: versions accessed by URL param rather than prefix?

2009-01-05 Thread Ben Nevile

Hi,

Much of my work with App Engine is on Facebook applications.  Facebook
acts as a proxy through which all requests flow. Every application
registers a callback URL that Facebook directs traffic at.  So for
instance

http://apps.facebook.com/my_application/foo

becomes

http://my_application.appspot.com/foo



One of my favorite GAE features is that I can access and test
different versions of my App Engine code.  Under most circumstances
the URL subdomain-based access (eg,
1.latest.my_application.appspot.com,
2.latest.my_application.appspot.com, etc) is very convenient, but this
convention doesn't fit well with Facebook's idea of the fixed callback
URL.  I can set up a dummy application to access the version of code
that I want, but this is A. kludgy and B. inconvenient because all of
the social context is absent in the dummy application (other friends
using the app, etc.)

I would very much like to be able to access different versions of my
code using a query parameter.  Then I could test my Facebook apps with
URLs like

http://apps.facebook.com/my_application/foo?APP_VERSION=2

which would be translated into

http://my_application.appspot.com/foo?APP_VERSION=2


I'm hoping something like this already possible.  If not, consider
this a feature request!  :)

Ben

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: exceeding quota grrrrrrrr

2008-12-17 Thread Ben Nevile

Thanks for the reply Marzia.  Did something change yesterday related
to my quotas?  I had never exceeded that quota before, my usage was
normal yesterday, but for some reason I was wa over.

Ben


On Dec 17, 10:34 am, Marzia Niccolai ma...@google.com wrote:
 Hi Ben,

 All of these quota warnings are related to your use bandwidth quota.

 Since you had consumed your bandwidth quota, secure bandwidth and URLFetch
 where both unavailable since these necessarily use bandwidth quota.

 Somewhat currently confusingly, bandwidth quota was listed as 'Okay' because
 it had been entirely consumed, and so everything was being denied.  We are
 re-working the 'rate' messaging to clarify the current status, as well as
 working on updating our documentation so that the relationships between
 quota are clear.

 Also, please follow up or file a quota request for additional bandwidth if
 you are consistently using your quota.

 Please let me know if you have any questions.

 Thanks,
 Marzia

 On Tue, Dec 16, 2008 at 10:42 PM, Ben Nevile ben.nev...@gmail.com wrote:

  I should also mention that when I last checked the dashboard at about
  7-7:30pm PST the outgoing bandwidth was at 56%.

  Ben

  On Dec 16, 10:40 pm, Ben Nevile ben.nev...@gmail.com wrote:
   the new quota details dashboard looks very nice.  thanks for your hard
   work on this mr. and ms. googs.

   but hey, how come i've never had a problem with outgoing bandwidth
   before, and tonight my app is stuck in the red zone?  the graph beside
   outgoing bandwidth is in the red (10.00 of 10.00 GBytes) but the
   rate is okay, whereas the Secure Outgoing Bandwidth graph is at
   zero, yet the rate is high.  There's also a high rate for
   UrlFetch's Data Sent to API, but I don't use UrlFetch.

   sigh. pissed off users again for me.
   Ben

   ps. app is n3xt
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] exceeding quota grrrrrrrr

2008-12-16 Thread Ben Nevile

the new quota details dashboard looks very nice.  thanks for your hard
work on this mr. and ms. googs.

but hey, how come i've never had a problem with outgoing bandwidth
before, and tonight my app is stuck in the red zone?  the graph beside
outgoing bandwidth is in the red (10.00 of 10.00 GBytes) but the
rate is okay, whereas the Secure Outgoing Bandwidth graph is at
zero, yet the rate is high.  There's also a high rate for
UrlFetch's Data Sent to API, but I don't use UrlFetch.

sigh. pissed off users again for me.
Ben

ps. app is n3xt
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: GAE: Beyond computer science

2008-12-16 Thread Ben Nevile

Agree completely Amir.  I'm hopeful that more APIs will be released
soon that will make the process even lighter for developers.  Next
hurdles to come down significantly will be identity (Google Friend
Connect, FB Connect) and micropayments (?) .

Ben



On Dec 16, 2:42 pm, Amir  Michail amich...@gmail.com wrote:
 Hi,

 I think the GAE is a real game-changer.  It will result in an
 explosion of creativity that will dwarf what we have seen in Web 2.0
 thus far.

 There's a lot of interesting computer science and software engineering
 involved in building the GAE, but I suspect less so in terms of
 building apps for it.  This is particularly true since GAE apps will
 by necessity need to be simple so that they are scalable:  there's
 only so much you can do in the time available for a request.

 And so IMO, the GAE is less about computer science and more about a
 new creative field -- more art than science.  I think that's great.
 This is long overdue.  See  
 http://weblog.fortnow.com/2006/07/science-and-art-of-computation.html

 Amir
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Task Queues and Asynchronous Processing TODAY

2008-12-10 Thread Ben Nevile

Hi Dennis -

I am using an architecture similar to what you describe.  Rather than
use a separate server to do the pinging, I use client-side JS to do
this dirty work.  You need to have a fairly steady stream of users for
this technique to be reliable.  :)

Ben



On Dec 9, 9:27 pm, Dennis [EMAIL PROTECTED] wrote:
 I'm architecting my system to workaround the limited write ability in
 each gae request.

 I'd like to use task queues (as in issue 
 109http://code.google.com/p/googleappengine/issues/detail?id=109),
 but that feature is not implemented yet.

 I'm thinking about using the following:
 -the initial GAE request accepts the user's request and queues the
 task into Amazon Simple Queue Service (Amazon SQS) using a call to an
 external url.
 -a hosted system (external to GAE) polls the SQS queues.
 -when the hosted system finds a task, it makes API calls into GAE to
 do the actual asynchronous work inside GAE and it's datastore.

 Just wondering if anyone else has used this type of architecture (or
 thought about it) and what their experience / opinion is.

 The only post I could find about using Amazon's queues 
 is:http://groups.google.com/group/google-appengine/msg/86368530077bc548
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: New app for finding wallpaper images - feedback welcome :)

2008-11-27 Thread Ben Nevile

Cool app Bonstio - my quick suggestion would be to re-organize the
page so that the image is at the top rather than below your
explanatory paragraph.  I have to scroll down to see it on my small
laptop screen (1280x800).

Ben



On Nov 27, 2:07 am, Bonstio [EMAIL PROTECTED] wrote:
 Oh dear - how very embarrassing! My apologies!!! That would be a copy/
 paste error

    http://feeling-lucky.appspot.com/

 On Nov 26, 5:36 pm, Stephen Judd [EMAIL PROTECTED] wrote:

  It's impossible for anyone to browse your localhost url.

  On Wed, Nov 26, 2008 at 8:25 AM, Bonstio [EMAIL PROTECTED] wrote:

   Hi all,

   My first post here..

   I'd be grateful if you could offer any feedback you might have on my
   first App Engine app. I've never written Python before so it's been
   interesting learning it.

  http://localhost:8080/?size=o

   Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: New Article on How Index Building Works

2008-11-26 Thread Ben Nevile

Great article, thanks GAE team!



On Nov 25, 1:30 pm, Marzia Niccolai [EMAIL PROTECTED] wrote:
 Hi,

 There have been a lot of questions on this group about index building and
 Ryan Barrett, the tech lead for the datastore, took some time to write an
 article on how index building 
 works:http://code.google.com/appengine/articles/index_building.html

 The article describes what happens when you upload new index definitions for
 your application.  It's definitely a worthwhile read if you would like more
 information on how indexes are built to serve queries with App Engine.

 -Marzia
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Deleting stuff

2008-11-21 Thread Ben Nevile

Hi Chris - thanks for your response. A 2 second delay?  Really?  I
don't think that would help.

1. Another of my apps is serving 100k req / sec.
2. if I delete 100 entities every two seconds it would take me a few
months to get through them all.

Ben




On Nov 21, 11:11 am, Chris Marasti-Georg [EMAIL PROTECTED]
wrote:
 How long are you waiting between requests?  I add a 2 second delay in the
 non-gae loop code when I do things like this.

 On Fri, Nov 21, 2008 at 12:10 PM, Ben Nevile [EMAIL PROTECTED] wrote:

  Greetings fellow GAE dorks,

  So I'm trying to clean up an existing app.  I want to delete all the
  entities of a certain class.  Since there's no automagic way to do
  this, I made a handler that deletes them a block at a time.  Here it
  is:

  class DeleteHandler(BaseRequestHandler):
   def get(self):
     size = int(self.request.get('size',default_value='10'))
     touches = Touch.gql('ORDER BY time desc').fetch(size)
     db.delete(touches)
     self.response.out.write(ok)

  Pretty simple.  So I called this in a loop for a while, and it started
  timing out at the query.  Hm.  Perhaps a problem with the index?

  I then changed the query to this:
     touches = Touch.all().fetch(size)

  So that the first index wasn't involved.  This worked for a while, but
  then started timing out again.  Here's the error:

  Traceback (most recent call last):
   File /base/python_lib/versions/1/google/appengine/ext/webapp/
  __init__.py, line 499, in __call__
     handler.get(*groups)
   File /base/data/home/apps/showmeflow/1.329447312090117530/main.py,
  line 58, in get
     most_recent_touches = Touch.all().fetch(size)
   File /base/python_lib/versions/1/google/appengine/ext/db/
  __init__.py, line 1377, in fetch
     raw = self._get_query().Get(limit, offset)
   File /base/python_lib/versions/1/google/appengine/api/
  datastore.py, line 938, in Get
     return self._Run(limit, offset)._Next(limit)
   File /base/python_lib/versions/1/google/appengine/api/
  datastore.py, line 882, in _Run
     _ToDatastoreError(err)
   File /base/python_lib/versions/1/google/appengine/api/
  datastore.py, line 1636, in _ToDatastoreError
     raise errors[err.application_error](err.error_detail)
  Timeout

  What's a dork to do?

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Is Memcache really needed?

2008-11-17 Thread Ben Nevile

Hi Anders,

In my experience with standard RMDBS, memcache is more useful for
caching arbitrary data and less useful for caching model objects.  If
you have a really tuned database layer a lot of your queries will
already be resident in the database's cache, and memcache provides
only a modest efficiency gain.  With GAE that seems to be less true -
in one of my apps, caching my User model reduced response time by
about 150ms.

However, where I have found memcache most valuable is in caching more
arbitrary data constructs.  Caching fragments of your rendered HTML,
for instance, can be really effective.  Or perhaps there's a big chunk
of JSON that is often requested, but really only needs to be updated
once a minute.  Removing the memcache API would be really really
unfortunate.

re: whether or not you should implement it, I would say NO.  until you
have a whole whack of users, concentrate on more important things.

Ben



On Nov 17, 12:26 am, Anders [EMAIL PROTECTED] wrote:
 Yes, removing Memcache from the GAE API is probably not a good idea.
 There must have been some reason why it was added.

 What I'm skeptic about is if it will be a good idea for me to use it
 or if I will only be doing something unnecessary or even worse,
 shooting myself in the foot if the Memcache does not scale well and if
 my site will start to get massive traffic (my application still has
 very low traffic but I want to be prepared for the eventuality of an
 exponential traffic increase :-).

 On Nov 17, 9:13 am, Greg [EMAIL PROTECTED] wrote:

   I don't know if this means that each server instance has its own
   Memcache or that all the instances access a single Memcache.

  All instances access the same memcache - although this may be
  distributed behind the scenes, I don't know about that.

  I agree with you that it would be elegant to have automatic caching,
  but that would impose some limitations - currently you can memcache
  anything (including complex objects), but you can only store limited
  data types in the datastore; and you can do (limited) querys on the
  datastore but not on memcache.

  Ideally we'd have a transparently cached queryable object store to
  replace the datastore and memcache, but I guess this would be a
  significant amount of development. Maybe Google should hire the Zope
  guys to build it.

  Cheers!
  Greg.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Observing GAE Scalability (or trying to ...)

2008-11-16 Thread Ben Nevile

Hi Ross,

 Well, Google App Engine would be a terrible place to serve most
 massively multiplayer games.  I hope that's not what got you
 interested in GAE.


Pretend I'm dumb... no, it's okay, I *am* dumb when it comes to MMOG.
Why would GAE be a terrible place to serve these? The type of
communication required?

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Observing GAE Scalability (or trying to ...)

2008-11-16 Thread Ben Nevile

Yes, it's clear that it is often convenient to complement GAE with a
server that's handling long-running processes, managing queues of
work, efficiently distributing messages, etc.  And your transactional
concerns seem valid, although it also seems like it should be possible
to create a new entity group/model type to represents the type of
contentious relationship that has you worried.  But I'm sure there are
lots of issues I haven't considered!

Thanks for the info.
Ben


On Nov 16, 12:10 pm, Josh Heitzman [EMAIL PROTECTED] wrote:
 The lack of any long running server side process also precludes doing
 any real time MMOGs.  With many turn based web based MMOGs there is no
 way to partition the game state into entities such that executing all
 player commands only requires updating entities in one entity group
 during the execution of the command, so its necessary to implement a
 cross-entity group transaction layer on top of the GAE datastore, but
 its currently unclear to me that this can actually be done while
 staying under GAEs quotas.

 On Nov 16, 11:52 am, Ben Nevile [EMAIL PROTECTED] wrote:

  HiRoss,

   Well, Google App Engine would be a terrible place to serve most
   massively multiplayer games.  I hope that's not what got you
   interested in GAE.

  Pretend I'm dumb... no, it's okay, I *am* dumb when it comes to MMOG.
  Why would GAE be a terrible place to serve these? The type of
  communication required?

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] memcache unavailable tomorrow 9-10am PST; serving unaffected

2008-11-13 Thread Ben Nevile

Yikes - serving unaffected for me is exactly wrong! The way I've
architected one of my apps, without memcache my requests will pretty
much all timeout.  I will need to put the app offline for the
duration of the maintenance.

We will be taking memcache offline tomorrow morning from 9-10am PST
(GMT-8) for routine maintenance.

Can I ask about the use of the phrase routine maintenance?  With all
due respect to your exciting and currently free product, can you
please confirm that as we move into a paid version of the service we
can expect this type of maintenance to be not routine, but the very
rare exception?

3,
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] property auto indexes

2008-11-11 Thread Ben Nevile

Howdy pardners,

I have a busy (40 req/sec) app that is occasionally throwing Timeout
errors on datastore operations.  I'm not sure what the cause of these
errors is - contention perhaps? - but in an effort to reduce the
amount of data being written, I want to confirm my understanding of
the put() operation.

Given the following model,

class Listening(db.Model):
  listener =
db.ReferenceProperty(User,collection_name=listening_to_set)
  listenee =
db.ReferenceProperty(User,collection_name=being_listened_to_set)
  created_at = db.DateTimeProperty(auto_now_add=True)


and assuming that no custom indexes have been defined, is it true that
a put() operation will write the entity to three different places?
(indexed on listener, indexed on listenee and indexed on created_at)

I am in the habit of sprinkling non-essential created_at properties
throughout my models, but I'm suspecting now that this carries
significantly more burden on the GAE datastore relative to MySQL.  Any
insight?

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: appcfg.py and 500 Server Error

2008-11-10 Thread Ben Nevile

Hi,

Yesterday I vaccumed an index.  It started off in the deleting
state, then moved to the error state.  I have tried to vacuum it
again and I get error 500s.

Any ideas?  Marzia?

Ben

ps. app is called n3xt.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] understanding the profiling output

2008-11-04 Thread Ben Nevile

Hi - I have a request that's sometimes taking more than 1000 mcycles,
and I'd like to use the profiler to figure out what's taking so long.
I implemented the profiling successfully, but the output doesn't help
me too much because I can't match a lot of the function names up with
parts of the API.  For instance, what is
google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait ?

Anyone got any insight they can lend into the below profile?

Thanks,
Ben



Profile data:
 75037 function calls (70686 primitive calls) in 0.341 CPU
seconds

   Ordered by: internal time
   List reduced from 699 to 80 due to restriction 80

   ncalls  tottime  percall  cumtime  percall
filename:lineno(function)
   150.1710.0110.1860.012
{google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait}
  2520.0220.0000.0220.000 {method '__reduce_ex__'
of 'object' objects}
   2575/30.0150.0000.0860.029 /base/python_dist/lib/
python2.5/pickle.py:263(save)
   439/580.0070.0000.0150.000
{google3.net.proto._net_proto___parse__python.MergeFromString}
 13050.0060.0000.0120.000 /base/python_dist/lib/
python2.5/pickle.py:221(memoize)
   247/420.0050.0000.0770.002 /base/python_dist/lib/
python2.5/pickle.py:649(_batch_setitems)
   252/470.0030.0000.0800.002 /base/python_dist/lib/
python2.5/pickle.py:339(save_reduce)
 51270.0030.0000.0030.000 {method 'get' of 'dict'
objects}
   110.0030.0000.0220.002 /base/python_dist/lib/
python2.5/pickle.py:839(load)
 45510.0030.0000.0030.000 {len}
   220.0030.0000.0030.000 /base/python_lib/
versions/1/google/appengine/datastore/entity_pb.py:3477(__init__)
   182/910.0020.0000.0030.000
{google3.net.proto._net_proto___parse__python.Encode}
 53570.0020.0000.0020.000 {id}
 49100.0020.0000.0020.000 {method 'read' of
'cStringIO.StringO' objects}
   290.0020.0000.0020.000 {__import__}
   210.0020.0000.0040.000 /base/python_lib/
versions/1/django/utils/html.py:53(urlize)
 45630.0020.0000.0020.000 {method 'write' of
'cStringIO.StringO' objects}
 13050.0020.0000.0030.000 /base/python_dist/lib/
python2.5/pickle.py:244(put)
  2460.0020.0000.0030.000 /base/python_dist/lib/
python2.5/pickle.py:1253(encode_long)
   150.0020.0000.0020.000
{google3.apphosting.runtime._apphosting_runtime___python__apiproxy.MakeCall}
   220.0020.0000.0120.001 /base/python_lib/
versions/1/google/appengine/api/datastore.py:502(_FromPb)
 11550.0020.0000.0020.000 /base/python_dist/lib/
python2.5/struct.py:54(pack)
  978/8610.0020.0000.0020.000 {getattr}
  2870.0010.0000.0060.000 /base/python_dist/lib/
python2.5/pickle.py:486(save_unicode)
   210.0010.0000.0050.000 /base/python_lib/
versions/1/django/template/defaultfilters.py:198(urlizetrunc)
  2520.0010.0000.0040.000 /base/python_lib/
versions/1/google/appengine/api/datastore_types.py:
1012(ValidateProperty)
  6020.0010.0000.0010.000 {method 'encode' of
'unicode' objects}
  3750.0010.0000.0050.000 /base/python_dist/lib/
python2.5/pickle.py:474(save_string)
   206/420.0010.0000.0780.002 /base/python_dist/lib/
python2.5/pickle.py:634(save_dict)
 17980.0010.0000.0010.000 {chr}
   210.0010.0000.0020.000 /base/python_lib/
versions/1/django/template/defaultfilters.py:438(timesince)
   439/580.0010.0000.0160.000 /base/python_lib/
versions/1/google/net/proto/ProtocolBuffer.py:61(MergeFromString)
  1480.0010.0000.0030.000 /base/python_lib/
versions/1/django/template/__init__.py:617(resolve_variable)
 33750.0010.0000.0010.000 {method 'append' of
'list' objects}
  6550.0010.0000.0010.000 {built-in method match}
   140.0010.0000.0010.000 {method 'clear' of
'dict' objects}
  5050.0010.0000.0030.000 /base/python_dist/lib/
python2.5/pickle.py:1166(load_long_binput)
 19480.0010.0000.0010.000 {isinstance}
  7370.0010.0000.0020.000 /base/python_dist/lib/
python2.5/pickle.py:254(get)
   820.0010.0000.0080.000 /base/python_dist/lib/
python2.5/pickle.py:684(save_inst)
 11540.0010.0000.0010.000 {repr}
  2520.0010.0000.0040.000 /base/python_dist/lib/
python2.5/pickle.py:526(save_tuple)
  3010.0010.0000.0020.000 /base/python_lib/
versions/1/google/appengine/api/datastore_types.py:77(ValidateString)
  2460.0010.0000.0040.000 /base/python_dist/lib/

[google-appengine] accessing a key without loading the model object

2008-11-02 Thread Ben Nevile

Hi,

Say model X has a ReferenceProperty to model Y.  I have an instance x
loaded into memory.  I want to know the key for model object y, but I
don't want to load y from the datastore.  From what I can tell in the
documentation, accessing the attribute y, which holds a key,
automatically loads the object y if it's not already in memory.

in code, I want

x.y.key()

but I don't want to load y from the datastore.  so I want something
like

x.y_key


Is this possible?  Thanks in advance,
Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] counting

2008-10-24 Thread Ben Nevile

Hi,

I have recently started using GAE for components of a large sports-
related Facebook app.  One of the contests has hundreds of thousands
of participants, and I need to be able to tell a user at any given
time what place they're in.  eg, you are 34,728th out of 234,829
participants.

After spending some time with the docs and browsing this group, it
appears that using the datastore there's no way to accomplish this
relatively mundane task.  So do I have to keep a mysql database active
on some other host just so that I can efficiently do this type of
analysis?

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] SMS code in Canada (Rogers)

2008-09-12 Thread Ben Nevile

Anyone using Rogers in Canada had trouble getting an SMS code sent?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Canada SMS Rogers

2008-09-12 Thread Ben Nevile

Anyone out there had problems receiving the SMS verification on Rogers
in Canada?  I have tried several times without success.

Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---



[google-appengine] Re: Canada SMS Rogers

2008-09-12 Thread Ben Nevile

My apols. The post I made yesterday on the topic didn't show up in my
search - thought it had not made it to the other side. Sorry 4 the
noise.

Ben



On Sep 12, 10:53 am, Wooble [EMAIL PROTECTED] wrote:
 You probably want to use the SMS problem form linked to in the FAQ
 (http://code.google.com/appengine/kb/sms.html) rather than posting the
 same question to the list repeatedly.

 On Sep 12, 1:38 pm, Ben Nevile [EMAIL PROTECTED] wrote:

  Anyone out there had problems receiving the SMS verification on Rogers
  in Canada?  I have tried several times without success.

  Ben
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---