[google-appengine] SSL for Naked Domain on Existing Project

[Devel63]

We have an existing app engine project started a number of years ago.  It 
serves managed security HTTPS on the www subdomain.

[image: Custom_Domain.png]
At the time of creation, naked domains were not supported.  The instructions 

 only 
cover how to do that for a new project, and don't work for an existing one.

Following the clues in the instructions, we added the 4  records to our 
DNS provider (a "new" requirement), and now the HTTP naked domain redirects 
to HTTPS for www, which is good.  But we'd like it to do the same thing for 
HTTPS on the naked domain, and that gets an ERR_CONNECTION_CLOSED.

 We tried adding the naked custom domain via the console, but that doesn't 
work because it's already at least partially mapped:

[image: Custom_Domain_Step2.png]
What can we do to get the naked domain working with HTTPS?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/688b9510-265f-4064-94f9-134cbb1f6c34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: App Engine Enable managed security not working

[Devel63]

I am having this same problem with subdomain.mydomain.com, which is 
currently using a self-signed SSL certificate.  The problem is that for 
some reason that certificate is no longer trusted by the browser, so I 
thought I'd switch over to Google's managed security.

But I can't do anything, and as the original poster says, I'm getting the 
message that I must be a domain owner.

However, I am also unable to reverify.  I have tried adding a TXT record, a 
CNAME record, and via Webmaster Central, the integrated GoDaddy check.  All 
those checks seem to be trying to verify the domain, not the subdomain, but 
the documentation seems to imply that's expected.

Not sure if it's related, but Webmaster Central says I am indeed the 
verified owner of the domain.  But it says this is the case via Delegation, 
from another verified owner that is my_email_addr...@gtempaccount.com, 
which is presumably the standard gmail login that got migrated when I 
created the Google Apps equivalent.  But that happened many years ago, and 
I added the subdomain.domain.com property to App Engine years long after 
that.  I suppose it's possible that with the new managed security, they are 
doing some other checks, and this has fouled things up.

Anyway, can someone help me?


On Tuesday, October 2, 2018 at 6:35:39 PM UTC-4, John Janzen wrote:
>
> Thanks Amit.
>
> Reverifying the domain did the trick. (Still not sure why I had to 
> reverify)
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f28d59dc-292c-4c79-89e6-11d50ee68dfd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Sporadically terrible performance

[Devel63]

Ed, was this something that Google fixed for you specifically, or part of a 
larger issue?  

I haven't seen the problem again today in my extremely limited testing, but 
frankly I have no idea what the real situation is.  I do see, though, that 
some interactions that always were very fast may need additional client 
logic to handle hugely slower interactions, and I'm wondering how often 
that occurs.  Occasionally on an ongoing basis, in batches on rare 
occasions, etc.  I was warned not to deploy on GAE in part for this reason, 
but I had hoped those issues were gone.


On Thursday, March 19, 2015 at 1:26:52 PM UTC-7, Ed wrote:

 Fyi -- this issue was resolved for us around 1:15 am pacific (8:15 am UTC) 
 this morning.

 Ed

 On Thursday, March 19, 2015 at 12:18:14 AM UTC-7, Ed wrote:

 This is unusual, but we are seeing the same behavior.

 Our static handlers and trivial health-check handlers are seeing 30s+ 
 latency. We've escalated with Google support, and we have an open ticket on 
 the issue.

 We'd been seeing strange behavior for a few days, including 30s+ on some 
 modules that do nontrivial work. But static files and trivial handlers 
 taking this long is new, and only started for us about a half hour ago.

 Ed

 On Wednesday, March 18, 2015 at 8:21:32 PM UTC-7, Devel63 wrote:

 I haven't been paying close attention, but suddenly(?) today I'm seeing 
 sporadically awful performance.  It took* 20 seconds just to serve a 
 static HTML page* with no one else using this instance, and no cold 
 start.  Normally this takes about 2ms, so 1 times worse. Let alone my 
 actual datastore interactions, which sometimes worked, sometimes took 
 orders of magnitude longer than usual, and sometimes issued a 500 with no 
 corresponding log entry.  What is going on with GAE?  Is anyone else seeing 
 problems?  

 Or is this just normal?



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5d15b401-f513-4e8c-9743-27ce822db4fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Sporadically terrible performance

[Devel63]

I haven't been paying close attention, but suddenly(?) today I'm seeing 
sporadically awful performance.  It took* 20 seconds just to serve a static 
HTML page* with no one else using this instance, and no cold start. 
 Normally this takes about 2ms, so 1 times worse. Let alone my actual 
datastore interactions, which sometimes worked, sometimes took orders of 
magnitude longer than usual, and sometimes issued a 500 with no 
corresponding log entry.  What is going on with GAE?  Is anyone else seeing 
problems?  

Or is this just normal?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f9d27f83-0fd3-48a1-8306-16134de1ddae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] How to upload file with new dev_appserver

[Devel63]

The documentation makes it appear very simple to get the contents of an 
uploaded file, but while I can do this with old_dev_appserver.py, I cannot 
get it with the current dev_appserver.py (v1.9.2).

Here's a simple example:

form name=myform action=http://localhost:8080/sync?cmd=tester; 
enctype=multipart/form-data method=post

Username: input type=file name=user_file /
input type=submit value=Submit /

/form


   - In old_dev_appserver.py, one could get the file in GAE via 
   self.request.get('user_file'), but not with dev_appserver.py


   - WebApp2 http://webapp-improved.appspot.com/guide/request.html#filessays 
   Uploaded files are available as cgi.FieldStorage (see the 
cgihttp://docs.python.org/2.7/library/cgi.html#cgi module) 
   instances directly in request.POST.  But, request.POST is empty, and 
   cgi.FieldStorage() does not contain 'user_file' either.

Strangely, if I print out self.request.params, I do see an element in the 
UnicodeMultiDict that is (u'user_file', FieldStorage(u'user_file', 
u'myfile.ico')).  But when I try to get that element, either via named 
access or just iterating over params, I cannot get it.  Even if I do a 
Len(self.request.params) I get one less than what I see, and the 
'user_file' element is missing.  If I do this with old_dev_appserver, the 
Len(self.request.params) is correct.

How do I get user_file?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] How to upload file with new dev_appserver

[Devel63]

Nope.  As I mentioned, self.request.POST is UnicodeMultiDict([]). 

On the other hand, self.request.params appears to have everything, but I 
cannot get at the dictionary entry, as described below.


On Tuesday, April 22, 2014 2:16:19 PM UTC-7, paddycarey wrote:

 I think this should work:

 file_content = self.request.POST.multi['user_file'].file.read()

 Paddy


 On Tue, Apr 22, 2014 at 9:40 PM, Devel63 danst...@gmail.com javascript:
  wrote:

 The documentation makes it appear very simple to get the contents of an 
 uploaded file, but while I can do this with old_dev_appserver.py, I cannot 
 get it with the current dev_appserver.py (v1.9.2).

 Here's a simple example:

 form name=myform action=http://localhost:8080/sync?cmd=tester; 
 enctype=multipart/form-data method=post

 Username: input type=file name=user_file /
  input type=submit value=Submit /

 /form


- In old_dev_appserver.py, one could get the file in GAE via 
self.request.get('user_file'), but not with dev_appserver.py 


- WebApp2 
http://webapp-improved.appspot.com/guide/request.html#filessays 
 Uploaded 
files are available as cgi.FieldStorage (see the 
 cgihttp://docs.python.org/2.7/library/cgi.html#cgi module) 
instances directly in request.POST.  But, request.POST is empty, and 
cgi.FieldStorage() does not contain 'user_file' either. 

 Strangely, if I print out self.request.params, I do see an element in the 
 UnicodeMultiDict that is (u'user_file', FieldStorage(u'user_file', 
 u'myfile.ico')).  But when I try to get that element, either via named 
 access or just iterating over params, I cannot get it.  Even if I do a 
 Len(self.request.params) I get one less than what I see, and the 
 'user_file' element is missing.  If I do this with old_dev_appserver, the 
 Len(self.request.params) is correct.

 How do I get user_file?

  -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] How to upload file with new dev_appserver

[Devel63]

Figured it out (finally!). I had been logging elements of the request 
(params, body, etc.) as I sorted through other issues. But you can only 
instantiate a cgi.FieldStorage element once, so by the time I got to the 
actual self.request.get('user_file'), it was already gone.

Classic case in which removing the debugging actually makes it work.


On Tuesday, April 22, 2014 2:34:44 PM UTC-7, Devel63 wrote:

 Nope.  As I mentioned, self.request.POST is UnicodeMultiDict([]). 

 On the other hand, self.request.params appears to have everything, but I 
 cannot get at the dictionary entry, as described below.


 On Tuesday, April 22, 2014 2:16:19 PM UTC-7, paddycarey wrote:

 I think this should work:

 file_content = self.request.POST.multi['user_file'].file.read()

 Paddy


 On Tue, Apr 22, 2014 at 9:40 PM, Devel63 danst...@gmail.com wrote:

 The documentation makes it appear very simple to get the contents of an 
 uploaded file, but while I can do this with old_dev_appserver.py, I cannot 
 get it with the current dev_appserver.py (v1.9.2).

 Here's a simple example:

 form name=myform action=http://localhost:8080/sync?cmd=tester; 
 enctype=multipart/form-data method=post

 Username: input type=file name=user_file /
  input type=submit value=Submit /

 /form


- In old_dev_appserver.py, one could get the file in GAE via 
self.request.get('user_file'), but not with dev_appserver.py 


- WebApp2 
http://webapp-improved.appspot.com/guide/request.html#filessays 
 Uploaded 
files are available as cgi.FieldStorage (see the 
 cgihttp://docs.python.org/2.7/library/cgi.html#cgi module) 
instances directly in request.POST.  But, request.POST is empty, 
and cgi.FieldStorage() does not contain 'user_file' either. 

 Strangely, if I print out self.request.params, I do see an element in 
 the UnicodeMultiDict that is (u'user_file', FieldStorage(u'user_file', 
 u'myfile.ico')).  But when I try to get that element, either via named 
 access or just iterating over params, I cannot get it.  Even if I do a 
 Len(self.request.params) I get one less than what I see, and the 
 'user_file' element is missing.  If I do this with old_dev_appserver, the 
 Len(self.request.params) is correct.

 How do I get user_file?

  -- 
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to google-appengi...@googlegroups.com.
 To post to this group, send email to google-a...@googlegroups.com.
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: GAE cold start time still too long

[Devel63]

I have complained about this many times in the past, so it's only fair
that I should write to congratulate Google.  For my app, anyway, start
times have improved amazingly over the past couple of weeks.  I still
wish there were no bootup times, but it's actually acceptable now.  A
big improvement!  I wonder if Google is now pre-compiling the code?

At the same time, I see that memcache is now expiring MUCH more
quickly.  That's too bad, but for me, for now, that's an acceptable
tradeoff, if indeed it is a tradeoff.

On Feb 21, 6:57 am, Takashi Matsuo matsuo.taka...@gmail.com wrote:
 Please make sure that your app doesn't load all the views on the
 memory when your index page is accessed.
 Loading all the views in your memory at startup will cause very long
 cold startup time.

 Perhaps you can use profiling for detecting what is the bottleneck.

 AFAIC, 10 secs cold-start is too long for any appengine/python application.

 --
 Takashi Matsuo
 Kay's daddy

 On Sun, Feb 21, 2010 at 10:30 PM, Anders i...@blabline.com wrote:
  Yes, it's the complete cold start I mean. Rarely nowadays the cold
  start time is as much as 10 seconds for me, but occasionally it's
  several seconds too long. Maybe it doesn't happen so frequently as to
  be a real problem, but it's still frustrating when suddenly the index
  page loads like some overbloated dot com site from the late 90s.

  On Feb 21, 11:03 am, Tim Hoffman zutes...@gmail.com wrote:
  HI

  If you don't need to start the django stack to serve a page from cache
  then you should be able to deliver a page from a warm instance in
  20-40ms
  and if you need to start an instance and only import memcache and
  retrieve the cached page you are looking at about 200ms.

  I am doing this on a number of sites (don't use django but use bfg,
  but a complete cold start with nothing in cache
  is typically around 8-10 seconds.  And even the main page is made up
  of many cachable bits) so its really rare
  we ever have to deal with a full stack startup with nothing in cache.

  T

  On Feb 21, 10:54 am, Anders i...@blabline.com wrote:

   I doubt that replacing calls to Django with calls to memcache would
   improve the cold start time significantly, although I could be wrong.
   What I guess happens in a cold start is that the application code has
   to be loaded through the network into a server and then instantiated
   before serving a webpage. Using the memcache would then not improve
   the initial loading of applications. Only if Django is taking a long
   time to load during a cold start would the use of memcache make much
   difference.

   On Feb 21, 3:37 am, Eli Jones eli.jo...@gmail.com wrote:

I have a testapp set up that I use to see if any goofy ideas I come up 
with
have any merit.

So you could try to create a simplified version of your page that used 
the
same imports and django templates as the live one.  Then create two
different test pages.. one where you experiment with caching different
things.. and then try to compare the cold start times.

On Sat, Feb 20, 2010 at 9:32 PM, Anders i...@blabline.com wrote:
 Strange, I don't see the preview release notice on
 appengine.google.com any longer. So I assumed that GAE was no longer
 in preview release version. And on the SDK download page it now says:
 Please note: The App Engine SDK is under active development, please
 keep this in mind as you explore its capabilities.

 I guess I could cache the html generated by Django, at least for the
 index page, if that library is more heavy to load than the Memcache
 API. I don't know exactly how Python works, but shouldn't it be
 possible for GAE to always have the standard frameworks always loaded
 into memory for all applications to share?

 On Feb 21, 3:17 am, Eli Jones eli.jo...@gmail.com wrote:
  First, when I log into appengine.google.com, it still says this 
  is a
  preview release with the preview release part being in bright 
  red
  letters... so.. take that to mean what you want.

  Second, you haven't mentioned what exactly the code for your index 
  page
  does? What is it loading?  How much caching are you doing?

  Caching isn't just for entities from the datastore.. you can and 
  should
 also
  cache html or page templates or whatever else you can..  Also, you
 shouldn't
  dump a bunch of imports at the top of your code.. only import 
  specific
  modules as needed within the code.

  Though, it's hard to make suggestions without knowing exactly how 
  your
 code
  works. (You may be doing all of the above things.. or feel like 
  you are
  doing them.)

  A lot of the annoying restrictions that people complain about, to 
  me, are
  inherent limitations to having a highly scalable infrastructure.. 
  the
  restrictions are there to force you to learn to code (from 

[google-appengine] Re: Running some code immediately before a Datastore put()?

[Devel63]

Nick,

I've been faced with your exact situation.  I have to say I don't
understand PreCallHook, and maybe that would be the best approach, but
here's what I do know.

Overriding put is a bad idea, and as you point out, the override
will not be called in a batch put.

I found that the most elegant solution was to derive a new type of
db.Property, and override its get_value_for_datastore.  Then GAE will
automatically do what you want regardless of how the entity is being
put.


On Jan 14, 1:51 am, Eric Ka Ka Ng ngk...@gmail.com wrote:
 would a PreCallHook works for your case?

 http://code.google.com/appengine/articles/hooks.html

 - eric

 2010/1/14 Nickolas Daskalou n...@daskalou.com:

  I have a property of a Model that is a hash of another property of the same
  Model, eg:

  class MyModel(db.Model):
    something = db.StringProperty()
    something_hash = db.StringProperty()
    def generate_hash(self):
      self.something_hash = sha1(self.something)

  I want generate_hash() to be automatically called just before the entity is
  put() into the Datastore, without the developer having to explicitly call
  entity.generate_hash().

  I know I can create a put() method that makes such a call and then calls
  put() on the superclass, but (a) is that the best way to do it, and (b) if
  the put() is part of a batch put (eg. via db.put(entities)), will the put()
  method of each model instance still be called?

  Nick

  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.
-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Performance Issues

[Devel63]

I've been very concerned about cold-start performance, since I have a
low traffic site (www.lifehints.com).

Over the past few days, though, performance has been great!!  My app
seems to often already be in memory, and even when it's apparently
not, it only takes 2-3 seconds to swap in and respond.

This afternoon, however, things deteriorated badly.  The app is
swapping out quickly, and startup times are taking 10+ seconds, and
unbelievably, I just monitored a startup that took almost 30
seconds.

GAE system status shows everything normal.

How can I get back to the great performance I've been enjoying (even
as late as this morning, when presumably everyone was back at work and
traffic levels were normal)?

Can someone explain these extreme fluctuations?

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Determine Location from IP Address via GAE

[Devel63]

A common task is to identify the country of origin for an incoming IP
address.  Seems silly for each of us to upload a country/IP database
to GAE.  Is there already a centralized way to handle this?

I saw this post (http://groups.google.com/group/google-appengine/
browse_thread/thread/50b7042f33166aba/c5e2dfcfd4d26af5?
lnk=gstq=country+ip+address#c5e2dfcfd4d26af5), but it's unclear to me
how reliable this is or whether it's being maintained.

BTW, if we can get country, can we get lower level divisions (e.g.
state/province) as well?

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Tragedy of the Commons, and Cold Starts

[Devel63]

Thanks for taking a look.  I agree, the site seems to be lasting
longer at the moment ... that's a good thing.  Startup time is worse
than usual (10+ seconds rather than 3 seconds), which isn't good.

I know you guys don't have many resources to actively improve GAE, and
I appreciate that you're doing what you can.  I look forward to
whatever solution you can come up with to improve the situation for
everyone: more hardware available, pre-compile Python, pay for warm
instance, somehow detect and disable pinging to free up resources for
all, whatever.


On Dec 14, 9:33 am, Toby Reyelts to...@google.com wrote:
 I took a few measurements of your app around 9AM PST today (Monday). Your
 VMs were timing out at between 90 seconds and two minutes. As you've
 noticed, the number of loading requests increases as the amount of traffic
 on App Engine increases. This is why we discourage active pinging.

 On Fri, Dec 11, 2009 at 9:16 PM, Devel63 danstic...@gmail.com wrote:
  Toby,

  Here's a site that gets almost no traffic (not yet publicized):
 www.lifehints.com

  Try it Monday morning at 9am PST and see how quickly it times out.
  I've been seeing it apparently time out in a few seconds during the
  workday.  I suppose it's possible that what I'm seeing is Google
  firing up another instance, but that sounds unlikely given that
  there's no one else on the site most of the time.

  On a Friday evening (now), it's staying warm for quite some time.

  On Dec 11, 8:57 am, Toby Reyelts to...@google.com wrote:
   On Thu, Dec 10, 2009 at 8:19 AM, bFlood bflood...@gmail.com wrote:
hi toby

you said: VM timeouts are measured on the order of minutes, not
seconds - I have not seen this in practice since over a year ago when
GAE will still young. currently, every site I've measured is collected
in seconds (10, maybe 20)

   Do you have the appids of specific sites that you believe are timing out
   every 10 seconds? This is not what we're seeing. Are you sure you aren't
   seeing loading requests to several different VMs in parallel?

   also: to write a Hello World python app that responds from a cold

start on the order of 100m - again, I have not seen this in practice
for quite some time. the simplest of python sites, with no imports and
very little code seem to start in the  500ms  to 1s range (and
sometimes, much longer). Please note Nick's post here, where he
changed his original cold-start metrics:http://bit.ly/6Fsoxv

   Nick's example is not Hello World. He's cold-starting his blogging app
  in
   less than 500ms. (I personally believe that is quite acceptably
  responsive).

   What I'm driving at is that there's a minimum startup time which you can
  not
   control as a developer, and that startup time is very low for python  (on
   the order of 100ms). Everything else is under your control: What
   dependencies you have, what web framework you use, whether you push
  static
   content and use ajax requests, etc...

I'm now under the impression that slow VM startups is a GAE issue and
while user imports are critical to keep them reasonable once they are
started, there is a lot of overhead that is completely out of our
control. The only way I've found to keep low traffic sites bearable is
to use polling from the task queue, so IMO, the title of this post is
quite appropriate

cheers
brian

On Dec 9, 12:26 pm, Toby Reyelts to...@google.com wrote:
 Responses inline.

 On Sun, Dec 6, 2009 at 7:49 PM, Devel63 danstic...@gmail.com
  wrote:
  Toby, you write that it doesn't usually pay to optimize loading
  requests.

  I agree with this whole-heartedly when you have your own server,
  and
  only load once per day or month.  It's probably true using GAE when
  you have 100K+ page views per day.

 I think there's a misunderstanding here. What I said was that it's
  not
worth
 optimizing loading requests in regards to quota. Latency is a
  separate
 concern.

  But for lower-volume web sites, GAE performance is atrocious.  In
  my
  personal case, we have optimized in all sorts of ways (js
  minification, liberal use of memcache, image sprites, sticking with
  Django 0.96, etc.) ... but the typical user experience is quite
  poor.
  It takes 3-10 seconds for the first page to load, and then often
  the
  instance is swapped out while the user reads the current page, so
  that
  the next request experiences the same thing.   If the app is warm,
  performance is fine.

 If your VM is timing out while a user is actively visiting the site,
  then
 your site is extremely low traffic. VM timeouts are measured on the
  order
of
 minutes, not seconds. So, for example, that means that you didn't
  receive
 any traffic to your VM at all for several minutes between the time
  the
user
 fetched the first and second pages.

  Maybe this gets appreciably

[google-appengine] Re: Tragedy of the Commons, and Cold Starts

[Devel63]

Toby,

Here's a site that gets almost no traffic (not yet publicized):
www.lifehints.com

Try it Monday morning at 9am PST and see how quickly it times out.
I've been seeing it apparently time out in a few seconds during the
workday.  I suppose it's possible that what I'm seeing is Google
firing up another instance, but that sounds unlikely given that
there's no one else on the site most of the time.

On a Friday evening (now), it's staying warm for quite some time.


On Dec 11, 8:57 am, Toby Reyelts to...@google.com wrote:
 On Thu, Dec 10, 2009 at 8:19 AM, bFlood bflood...@gmail.com wrote:
  hi toby

  you said: VM timeouts are measured on the order of minutes, not
  seconds - I have not seen this in practice since over a year ago when
  GAE will still young. currently, every site I've measured is collected
  in seconds (10, maybe 20)

 Do you have the appids of specific sites that you believe are timing out
 every 10 seconds? This is not what we're seeing. Are you sure you aren't
 seeing loading requests to several different VMs in parallel?

 also: to write a Hello World python app that responds from a cold

  start on the order of 100m - again, I have not seen this in practice
  for quite some time. the simplest of python sites, with no imports and
  very little code seem to start in the  500ms  to 1s range (and
  sometimes, much longer). Please note Nick's post here, where he
  changed his original cold-start metrics:http://bit.ly/6Fsoxv

 Nick's example is not Hello World. He's cold-starting his blogging app in
 less than 500ms. (I personally believe that is quite acceptably responsive).

 What I'm driving at is that there's a minimum startup time which you can not
 control as a developer, and that startup time is very low for python  (on
 the order of 100ms). Everything else is under your control: What
 dependencies you have, what web framework you use, whether you push static
 content and use ajax requests, etc...

  I'm now under the impression that slow VM startups is a GAE issue and
  while user imports are critical to keep them reasonable once they are
  started, there is a lot of overhead that is completely out of our
  control. The only way I've found to keep low traffic sites bearable is
  to use polling from the task queue, so IMO, the title of this post is
  quite appropriate

  cheers
  brian

  On Dec 9, 12:26 pm, Toby Reyelts to...@google.com wrote:
   Responses inline.

   On Sun, Dec 6, 2009 at 7:49 PM, Devel63 danstic...@gmail.com wrote:
Toby, you write that it doesn't usually pay to optimize loading
requests.

I agree with this whole-heartedly when you have your own server, and
only load once per day or month.  It's probably true using GAE when
you have 100K+ page views per day.

   I think there's a misunderstanding here. What I said was that it's not
  worth
   optimizing loading requests in regards to quota. Latency is a separate
   concern.

But for lower-volume web sites, GAE performance is atrocious.  In my
personal case, we have optimized in all sorts of ways (js
minification, liberal use of memcache, image sprites, sticking with
Django 0.96, etc.) ... but the typical user experience is quite poor.
It takes 3-10 seconds for the first page to load, and then often the
instance is swapped out while the user reads the current page, so that
the next request experiences the same thing.   If the app is warm,
performance is fine.

   If your VM is timing out while a user is actively visiting the site, then
   your site is extremely low traffic. VM timeouts are measured on the order
  of
   minutes, not seconds. So, for example, that means that you didn't receive
   any traffic to your VM at all for several minutes between the time the
  user
   fetched the first and second pages.

Maybe this gets appreciably better as traffic improves, but of course,
I can't see that at present.

   Yes, as stated above, VMs are not aggressively collected. In the normal
   case, if you have an active user of your website, you shouldn't see a
   cold-start per request. Maybe in your particular case you can
  asynchronously
   ping your backend (for example, with an AJAX request) a few seconds
  before
   they continue onto the next page?

   I love GAE in theory, but it's getting

harder to ignore the reality of low-volume performance.

   As stated above, I think you're falling into a particularly bad extreme
   (continuous cold requests for an active user). This might require some
   creativity (for example, as above) to work around.

   In terms of speeding up the loading request itself, the good news that
  the
   bulk of of that time is directly under your control. As an existence
  proof
   of this, you should be able to write a Hello World python app that
   responds from a cold start on the order of 100ms. This means you might
  try
   doing things like paring down the dependencies that you load on cold
   requests. You can also

[google-appengine] Re: Tragedy of the Commons, and Cold Starts

[Devel63]

Toby,

In the past, it was several minutes before an active instance would be
swapped out.  Of late, I have seen it repeatedly/regularly happening
within several seconds.

I've avoided (so far) the auto-ping approach; your idea to auto-ping
only when a user is on a page is intriguing.  Still wasteful and
wrong, but perhaps necessary.

Yes, I have voted for the paid warm instance :-)

Finally, I don't understand how I can significantly reduce my warm up
time. I suppose I could split each page into a separate app.yaml
handler (already done for admin versus user tasks), but then the user
would even more certainly run into startup issues when navigating
within the site.  Besides, most of the time is spent importing Django
and system stuff I can't control.

The only reason I'm using Dango (0.96) is for translations ... is
there a built-in way to handle translations via webapp?


On Dec 9, 9:26 am, Toby Reyelts to...@google.com wrote:
 Responses inline.

 On Sun, Dec 6, 2009 at 7:49 PM, Devel63 danstic...@gmail.com wrote:
  Toby, you write that it doesn't usually pay to optimize loading
  requests.

  I agree with this whole-heartedly when you have your own server, and
  only load once per day or month.  It's probably true using GAE when
  you have 100K+ page views per day.

 I think there's a misunderstanding here. What I said was that it's not worth
 optimizing loading requests in regards to quota. Latency is a separate
 concern.

  But for lower-volume web sites, GAE performance is atrocious.  In my
  personal case, we have optimized in all sorts of ways (js
  minification, liberal use of memcache, image sprites, sticking with
  Django 0.96, etc.) ... but the typical user experience is quite poor.
  It takes 3-10 seconds for the first page to load, and then often the
  instance is swapped out while the user reads the current page, so that
  the next request experiences the same thing.   If the app is warm,
  performance is fine.

 If your VM is timing out while a user is actively visiting the site, then
 your site is extremely low traffic. VM timeouts are measured on the order of
 minutes, not seconds. So, for example, that means that you didn't receive
 any traffic to your VM at all for several minutes between the time the user
 fetched the first and second pages.

  Maybe this gets appreciably better as traffic improves, but of course,
  I can't see that at present.

 Yes, as stated above, VMs are not aggressively collected. In the normal
 case, if you have an active user of your website, you shouldn't see a
 cold-start per request. Maybe in your particular case you can asynchronously
 ping your backend (for example, with an AJAX request) a few seconds before
 they continue onto the next page?

 I love GAE in theory, but it's getting

  harder to ignore the reality of low-volume performance.

 As stated above, I think you're falling into a particularly bad extreme
 (continuous cold requests for an active user). This might require some
 creativity (for example, as above) to work around.

 In terms of speeding up the loading request itself, the good news that the
 bulk of of that time is directly under your control. As an existence proof
 of this, you should be able to write a Hello World python app that
 responds from a cold start on the order of 100ms. This means you might try
 doing things like paring down the dependencies that you load on cold
 requests. You can also take advantage of the fact that requests for static
 content bypass your VM and are never cold. So, for example, you can serve
 a page that is comprised mostly of static content almost instantly, and let
 it make AJAX requests to asynchronously fill in its dynamic content as your
 VM warms up.

 If you'd rather just pay to have us maintain a warm VM for you, you can vote
 on that issue.

  On Dec 4, 3:38 pm, Toby Reyelts to...@google.com wrote:
   On Oct 23, 3:02 pm, bugaco ice...@gmail.com wrote:

I had a bit weird experience with this...

So I wrote app (http://analytics.bugaco.com) that runs on App Engine.
Than I looked at the request logs to see how it is running.
Request logs suggested that I'm using a lot of CPU time on hitting the
home page, but after that CPU time significantly decreases. It also
had annoying red flag suggesting that servlet is using excessive
resources and that I need to optimize it.
Testing a bit, I noticed that pinging lets app be warm, and I had cron
doing the pings for a few days; while also noticing that it does not
do anything useful

Conclusion:
1. If log files don't suggest that you are better off pinging people
would not ping

   I'm not sure what you mean here, but we have plans to change the admin
   console to explicitly call out loading requests, so you can take that
   into account when profiling your application. Until that becomes
   available, it's pretty easy for you to detect and log loading requests
   yourself.

2. It is stupid that google

[google-appengine] Re: Tragedy of the Commons, and Cold Starts

[Devel63]

Toby, you write that it doesn't usually pay to optimize loading
requests.

I agree with this whole-heartedly when you have your own server, and
only load once per day or month.  It's probably true using GAE when
you have 100K+ page views per day.

But for lower-volume web sites, GAE performance is atrocious.  In my
personal case, we have optimized in all sorts of ways (js
minification, liberal use of memcache, image sprites, sticking with
Django 0.96, etc.) ... but the typical user experience is quite poor.
It takes 3-10 seconds for the first page to load, and then often the
instance is swapped out while the user reads the current page, so that
the next request experiences the same thing.   If the app is warm,
performance is fine.

Maybe this gets appreciably better as traffic improves, but of course,
I can't see that at present.  I love GAE in theory, but it's getting
harder to ignore the reality of low-volume performance.


On Dec 4, 3:38 pm, Toby Reyelts to...@google.com wrote:
 On Oct 23, 3:02 pm, bugaco ice...@gmail.com wrote:

  I had a bit weird experience with this...

  So I wrote app (http://analytics.bugaco.com) that runs on App Engine.
  Than I looked at the request logs to see how it is running.
  Request logs suggested that I'm using a lot of CPU time on hitting the
  home page, but after that CPU time significantly decreases. It also
  had annoying red flag suggesting that servlet is using excessive
  resources and that I need to optimize it.
  Testing a bit, I noticed that pinging lets app be warm, and I had cron
  doing the pings for a few days; while also noticing that it does not
  do anything useful

  Conclusion:
  1. If log files don't suggest that you are better off pinging people
  would not ping

 I'm not sure what you mean here, but we have plans to change the admin
 console to explicitly call out loading requests, so you can take that
 into account when profiling your application. Until that becomes
 available, it's pretty easy for you to detect and log loading requests
 yourself.

  2. It is stupid that google counts warming up your app toward CPU time
  (leading to profiling, that leads to pinging)

 A couple of things:

 1) CPU time doesn't grow on trees, it comes out of your free or paid
 quota. Why should we hide this from you?

 2) The number of loading requests your application receives are
 inversely proportional to its traffic. If you get more traffic, you'll
 receive fewer loading requests. This means it usually doesn't pay to
 optimize loading requests, unless you're just trying to reduce user
 latency.

  3. It is very stupid that applications can not denote 'keep this code
  path warm/cache it/or something' that will allow new users not to give
  up on the up until they get first response.

 Unfortunately, it takes an inordinate amount of physical hardware to
 keep on the order of millions of applications in memory, which is
 somewhat counter to free. If our startup optimizations plus your own
 optimizations don't satisfy you, then maybe you can voice your opinion
 on paying for a warm VM (http://code.google.com/p/googleappengine/
 issues/detail?id=2456)?



  So, as a conclusion, I think AppEngine is AWESOME. And I also think it
  SUCKS.
  I love SDK, ability to deploy and test and use all the cool things.
  I don't like the idea that it can not serve a (entry)page in 3-5
  seconds as I think that it leaves bad taste in users mouth, and
  consequently bad taste in developers mouth.

  Finally, I am not sure I'll use AppEngine for developing other
  applications as I'd rather go with paid hosting that provides some
  level of performance on serving pages. I think Google would win a lot
  of good will if they at least provide quick serving of static
  resources.

 Google App Engine already serves static resources without intervening
 requests to application VMs. This means that, for example, you could
 serve a page that was entirely static content, with a small amount of
 JS to ping your VM with an asynchronous dynamic request to wake it up.
 That page would be served instantly to the user. You need to ensure
 though, that the resources are indeed specified as static content in
 your app.yaml or appengine-web.xml.



  One may wonder how to do that, and given that they have all those yaml
  files there may be yaml file that specifies a warm static resource.
  This would decrease a need for pinging your app as it would allow user
  to hit entry page, and google to pre-cache app much easier.

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What IDE the community recommend to developers of Python+GAE

[Devel63]

We use Komodo IDE.  It's not free, but it's FANTASTIC.  It will more
than pay for itself in increased productivity.

On Nov 27, 1:07 pm, Olemis Lang ole...@gmail.com wrote:
 On Tue, Nov 24, 2009 at 6:16 PM, frankabel frank.abel...@gmail.com wrote:
  Hi all,

  I know that Java+GAE havehttp://code.google.com/eclipse/as google
  recommendation, but what about Python+GAE developers?

 You can try Eclipse + PyDev too ;o)

  Cheers

 Oh ! yes ! Cheers !

 --
 Regards,

 Olemis.

 Blog ES:http://simelo-es.blogspot.com/
 Blog EN:http://simelo-en.blogspot.com/

 Featured article:
 Looking for a technique to create flexible, graphical dashboards ...
 -http://feedproxy.google.com/~r/TracGViz-full/~3/YfBiyx_E6jM/0298ec51c...

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Low traffic site sleeps?

[Devel63]

This is a real problem for GAE.

Low traffic sites are getting cycled out very aggressively, sometimes
in 1-3 seconds.  This creates an unacceptable user experience for low
traffic sites, because almost every user sees a 3-10 second delay, and
if they take a couple of seconds to read the page, they see the delay
for every page.  GAE is great in theory, but in practice this one
aspect is horrible.

What to do:
  - As you describe, poll your app every 2 seconds
  - Google is considering allowing you to pay to keep your app warm
  - Switch to something else, like LAMP or EC2

If everyone polls their app, Google gets more aggressive about cycling
out apps; see the Tragedy of the Commons:
http://groups.google.com/group/google-appengine/browse_thread/thread/22692895421825cb/

I don't know what the answer is, but as my company gets closer to
mainstream launch, this is very worrisome: how will we ever attract a
user base big enough to keep our app warm if the initial users all
experience a horribly slow site?

Note that as I write this on Saturday, things actually seem pretty
good.  Our app isn't getting cycled out every couple of seconds, and
startup time is 3 seconds.  But this past week we would disappear
every 2 seconds, and startup time was frequently 6-10 seconds.  I hope
Google has done something, but I fear it's just light weekend load.

On Nov 13, 1:59 pm, JohnJ ufm...@gmail.com wrote:
 It seems that since my site receives a low number of requests the site
 can go to sleep (GAE reclaims the resources). As a consequence if
 you're the unlucky person who makes the next request you have to wait
 a LONG time to get a response.

 How do folks deal with this?

 A couple ideas come to mind 1) create a task that runs every so often
 to keep my app alive 2) enable an external monitor to ping/heartbeat
 my app every so often.  Both ideas seem to artificially eat up my
 quote so i'm not too exited about these solutions. Also what is the
 proper amount of time to fire one of these tasks/heartbeats?  After
 how much in-activity does GAE reclaim the JVM?

 Thanks for the help up front

 JohnJ

--

You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=.

[google-appengine] Re: frame/object control with login

[Devel63]

+ another 1

User testing showed the Google login experience to be extremely
confusing for users (so we switched to Facebook Connect ... don't get
me started on the problems with Facebook, but at least their Connect
Experience doesn't confuse users).

Would be wonderful if we could use our own custom look on the front-
end for a Google account, or if Google developed a similar awareness
as Facebook has done with Connect so that people understand what's
going on.


On Nov 4, 11:06 am, Joshua Smith joshuaesm...@charter.net wrote:
 +1.

 We've abandoned using google's user authentication for exactly this  
 reason.  It confused the hell out of all our users.

 When we started using google apps for our email, it started confusing  
 the hell out of us.

 -Joshua

 On Nov 4, 2009, at 1:44 PM, ryan baldwin wrote:

  Nick,

  I think you're overestimating the proverbial average user. In  
  fact, in user testing our own application, users are almost  
  unanimously tripped up when they are redirected to Google to login.  
  We frequently heard the user say this:

  Okay, now I click here to login and... oh... why am I asked to  
  login to Google?

  I'm just pointing this out so that we don't all get stuck in a  
  belief set that may not be accurate. Personally, I think Google  
  needs to come up with an API (whether it be javascript or otherwise)  
  for AppEngine apps to authenticate users more naturally. It's very  
  jarring and, dare I say, short sited to force all users of all apps  
  away from an application to an unbranded login screen.

  - ryan.

  On Wed, Nov 4, 2009 at 2:40 AM, Nick Johnson (Google) 
  nick.john...@google.com
   wrote:
  Hi,

  Framing the login page is strongly discouraged, and may in fact be  
  contrary to the TOS. The only way for the user to determine that a  
  login page for a Google account is legitimate is to check if it's  
  being served off the google.com domain, and this is not possible  
  inside a frame.

  -Nick Johnson

  On Mon, Nov 2, 2009 at 10:36 PM, reyelts reye...@gmail.com wrote:

  I apologize if I'm posting to the wrong place, I don't know if this is
  a basic HTML question, a Python question, a GAE question, or some
  combination of the set...

  Since we can't have much impact on the use login window for Google App
  Engine (ie to change the language or add an image), I'm looking at
  embedding the login page, like so:

  from google.appengine.ext        import webapp
  from google.appengine.api        import users

  class main(webapp.RequestHandler):
    def get(self):
       user = users.get_current_user()

       if user == None:
          self.response.out.write('html\nbody\n')
          self.response.out.write('pMy intro text/p\n')
          self.response.out.write('hr\n')
          self.response.out.write('object\n')
          self.response.out.write('  data=' + users.create_login_url
  (self.request.uri) + '\n')
          self.response.out.write('  type=text/html\n')
          self.response.out.write('  width=100% height=50%\n')
          self.response.out.write('/object')
          self.response.out.write('hr\n')
          self.response.out.write('/body\n/html\n')
       else:
          self.response.out.write('html\nbody\n')
          self.response.out.write('p' + user.nickname() + 'is logged
  in!/p\n')
          self.response.out.write('/body\n/html\n')

  This works fine: I get My intro text at the top of the window
  followed by a pane with the login. However, when I login, the result
  (user is logged in) is written to the pane where the login occurred,
  vs. replacing the whole window. I tried this with the older iframe,
  and get the same result.

  Is there a way to re-take over the complete window (ie make the frame
  I created go away) on the redirect?

  --
  Nick Johnson, Developer Programs Engineer, App Engine
  Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration  
  Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Logging in with Google Apps account

[Devel63]

Matt,

When you create your app, you make an irrevocable decision in the
creation forms as to which type of logins it will support: Google
Accounts, or Google App Accounts.  There's no way to change it.  But
you can simply create another app and check the box you want.

On Oct 22, 1:09 pm, Matt m...@gwalski.com wrote:
 From the end user perspective, is there any way to login to a appspot
 app using a Google Apps (hosted domain) account?
 I can add an app to my domain using the control panel, but I can't
 actually login when I visit the .appspot.com URL.
 It is a real PITA how Google decided that Google Apps accounts weren't
 Google accounts. And they won't listen to their users and fix that.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Tragedy of the Commons, and Cold Starts

[Devel63]

In a separate thread, someone from Google confirmed that they compile
all the files from scratch with each load.

Speeding up cold starts is clearly the best solution, but I don't know
how much time pre-compiling would save.  Paying for warm instances may
help, but because anyone can auto-ping every second, the tragedy of
the commons will still proceed to its inevitable conclusion, causing
tremendous thrashing.

On Oct 22, 6:05 am, bFlood bflood...@gmail.com wrote:
 agreed, this a catch22 for low request sites. pinging only goes so far
 and seems completely wasteful (for everyone)

 I'd love to know if they precompile python modules (and if not, there
 must be a good reason why)
 paid for warm instances - yes!

 cheers
 brian

 On Oct 22, 7:25 am, ted stockwell emorn...@gmail.com wrote:

  On Oct 21, 10:31 pm, Devel63 danstic...@gmail.com wrote:

   I see only 2 ways out:

  Another way would be for Google to charge to keep applications warm.
  Amazon has a similar feature where you can pay extra to reserve EC2
  instances to make sure that the instance are always available.
  Keeping apps warm is quite resource intensive, so I don't see how
  Google could not charge for it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Tragedy of the Commons, and Cold Starts

[Devel63]

I fear we are in a destructive cycle:
  - Cold starts take a long time (many seconds)
  - So app developers set up auto-pings to keep their app warm
  - So Google gets more aggressive in cycling out instances
  - So app developers increase their ping frequency
  - So even popular apps constantly hit cold starts
  - So GAE becomes unusable for all

I have personally held off doing any auto-pings because I felt it was
wrong, and bad for the common good.  But cold starts have gotten
slower, and my app seems to get cycled out even more aggressively of
late (just a few seconds).

It would be nice to ask everyone to stop auto-pinging, but there's no
way to enforce it, and history has shown that the tragedy of the
commons is hard to avoid.

I see only 2 ways out:
  -- Make cold starts much faster (perhaps by pre-compiling the code?)
  -- Somehow associate enough cost that auto-pingers will stop
 I have no idea how to do this.
 Even if you ping every 3 seconds, that's still only 30K page
views per day
 People with no traffic could ping every single second and
never pay anything

So, at a minimum, perhaps pre-compiling the code is a good first
start?

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Add Domain does not work

[Devel63]

You do not need to sign up for Google Apps.

The link Gaurav provided explains how to link a domain in either case
(with Google Apps or without).

On Oct 18, 10:36 am, Gaurav ano...@gmail.com wrote:
 Signing up with google apps is 
 required.http://code.google.com/appengine/articles/domains.html

 On Oct 18, 8:15 pm, JonRWarren jonrwar...@gmail.com wrote:

  Are you saying that only by using google apps can we map a domain name
  to an app_engine app? That does not sound correct to me, from all I
  have read in the documentation we are able to map a domain name to an
  app_engine app without requiring use of google app. No? Am I mistaken?

  On Oct 18, 12:35 am, Gaurav ano...@gmail.com wrote:

   Are you using google apps?http://www.google.com/apps/

   if not, then you cannot use your domain for app engine.
   It's free and easy to sign up for google apps.
   go ahead do it.

   On Oct 18, 9:18 am, lordjoe smle...@lordjoe.com wrote:

I had a google app lordjoe2...@appspot.com - it works well
I want to map it towww.lordjoe.com
I went through all the steps - added a CName, added the domail and in
the
dashboard I see both
lordjoe2...@appspot.com
 www.lordjoe.com

However  www.lordjoe.com
leads to a page saying
Google           Error
Not Found

Help - what am I doing wrong

PS - the same procedure works well for a second domain pointed at a
different one of my apps
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Add Domain does not work

[Devel63]

Yep, my mistake.  I thought I had done this, but it had been so long I
had forgotten that being an Apps domain was required.

On Oct 19, 10:16 am, Wooble geoffsp...@gmail.com wrote:
 On Oct 19, 12:18 pm,Devel63danstic...@gmail.com wrote:

  You do not need to sign up for Google Apps.

  The link Gaurav provided explains how to link a domain in either case
  (with Google Apps or without).

 No it doesn't, because that's impossible.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Deferred Tasks

[Devel63]

I've been deferring tasks by using a callback from the user's
browser that follows up an immediate request with other things I want
to happen in the background as a result of that request (e.g.,
preparing for the next probable request, putting memcache items into
DB, etc.).

Is the new deferred capability suitable for this?  Is it better or
worse than my approach?  What are the downsides?

Is it subject to the 10K task queue insertions quota (making it
completely unsuitable for this)?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: What are your top three issues in appengine?

[Devel63]

1. Full text search

2. Memcache objects that get a chance to write themselves to disk if
being evicted (yes, this would require some changes to the memcache
allocation approach)

3. Faster cold start with Django (I think there are many apps pinging
themselves constantly to try to overcome this issue, defeating
Google's ability to manage the resources effectively ... with a faster
cold start, this would become less problematic.  My understanding is
that Google compiles the Python files every cold start: perhaps a
simple case of precompilation would help and be easy to implement).


On Oct 8, 10:37 am, Henrik Lindqvist henrik.lindqv...@gmail.com
wrote:
 1. CPU cost of datastore writes. This is by far the biggest show-
 stopper for any data centric application. GAE is just to expensive!

 2. The missing java.awt.* classes that is needed to get FOP/iText PDF
 generation. Another show-stopper for most enterprise applications.

 3. HTTPS. Any serious business won't be content using a appspot.com
 sub-domain to get HTTPS.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Requesting Same Entity Multiple Times in 1 Model.get_by_key_name Call

[Devel63]

Thanks for the response; already de-duped just to be safe.

On Oct 5, 2:07 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Devel63,
 The SDK will pass the list of keys through to the backend unmodified.
 Whether the backend will de-duplicate before fetching is undefined, but
 probable. In any case, your code will still have the overhead of decoding
 the same entity several times - so I would suggest deduplicating yourself
 before sending it to the API.

 -Nick Johnson

 On Fri, Oct 2, 2009 at 5:34 PM, Devel63 danstic...@gmail.com wrote:

  If I pass in a list of key_names to Model.get_by_key_name(), and the
  list contains duplicates, everything works and I get the complete,
  duplicative list of entities back.

  But behind the scenes, is GAE making only one query for all the
  duplicates, or is it being smart and just duplicating the results?

 --
 Nick Johnson, Developer Programs Engineer, App Engine
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Warnings about db.model_from_protobuf

[Devel63]

I was excited to learn about the increased memcache efficiency
available using protocol buffers, as described in Nick's excellent
blog post: http://blog.notdot.net/2009/9/Efficient-model-memcaching

I have learned the hard way, however, that there are a couple of
issues:

 - They don't work with search.Searchable
 - They delete/drop any in-memory data you have attached to an entity

Word to the wise...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Requesting Same Entity Multiple Times in 1 Model.get_by_key_name Call

[Devel63]

If I pass in a list of key_names to Model.get_by_key_name(), and the
list contains duplicates, everything works and I get the complete,
duplicative list of entities back.

But behind the scenes, is GAE making only one query for all the
duplicates, or is it being smart and just duplicating the results?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Facebook Integration?

[Devel63]

Unfortunately, this is where I got my copy of pyfacebook, and as I
mentioned, it relies on receiving an 'added' parameter which Facebook
no longer apparently provides.  I don't know what else is wrong with
it, if anything.

On Sep 8, 3:14 pm, ramu rslet...@gmail.com wrote:
 http://github.com/sciyoshi/pyfacebook/tree/master

 On Sep 9, 2:09 am, Devel63 danstic...@gmail.com wrote:

  Can anyone point me in the direction of a good resource for Facebook
  Connect/GAE integration?

  Neither the ShelfTalkers article (http://code.google.com/appengine/
  articles/shelftalkers.html), nor PyFacebook (upon which ShelfTalkers
  relies) appear to work anymore.  For example, PyFacebook relies on an
  'add' parameter in the session cookie which is not (any longer?)
  there.

  This post (http://forum.developers.facebook.com/viewtopic.php?
  id=23276) enabled me to move forward (just look for the 'user' param
  and set 'added' true if present), but it's scary to be relying on
  something that's possibly no longer maintained.

  Is PyFacebook still maintained and working?

  And how do people test this FB stuff on a local server (I'm having to
  upload to Google every time)?  The ShelfTalkers article implied I
  could just set the FB callback URL to 127.0.0.1:8080, but either that
  or something else is causing weird failures.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Facebook Integration?

[Devel63]

Thanks for this example.

It looks to me like you started with pyfacebook, and then included a
simplified check_session/check_connect_session function (that ignores
this 'added' parameter I was complaining about).  Is that correct?
Were there any other 'gotchas'?

Or is there some official or centralized location to get the latest
version of the facebook set of libraries?


On Sep 8, 3:17 pm, P@ chane...@gmail.com wrote:
 stripedbananas is an excellent example of FB Connect integration on
 GAE Python
 Code 
 at:http://code.google.com/p/cassiesplayground/source/browse/#svn/trunk/s...
 Demo at:http://stripedbananas.appspot.com/

 And you get Google Friend Connect for the same price:-)

 P@

 On Sep 8, 2:09 pm, Devel63 danstic...@gmail.com wrote:

  Can anyone point me in the direction of a good resource for Facebook
  Connect/GAE integration?

  Neither the ShelfTalkers article (http://code.google.com/appengine/
  articles/shelftalkers.html), nor PyFacebook (upon which ShelfTalkers
  relies) appear to work anymore.  For example, PyFacebook relies on an
  'add' parameter in the session cookie which is not (any longer?)
  there.

  This post (http://forum.developers.facebook.com/viewtopic.php?
  id=23276) enabled me to move forward (just look for the 'user' param
  and set 'added' true if present), but it's scary to be relying on
  something that's possibly no longer maintained.

  Is PyFacebook still maintained and working?

  And how do people test this FB stuff on a local server (I'm having to
  upload to Google every time)?  The ShelfTalkers article implied I
  could just set the FB callback URL to 127.0.0.1:8080, but either that
  or something else is causing weird failures.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Slow response static javascript files on development server

[Devel63]

If you don't want to turn off IPv6, referencing your local server via
http://127.0.0.1:8080 accomplishes the same thing.

On Sep 8, 8:57 am, Nick Winter livel...@gmail.com wrote:
 For Firefox, go to about:config and set network.dns.disableIPv6 to
 true. For some reason, localhost requests are mega slow with the IPv6
 on (not an App Engine thing).
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Local datastore speed

[Devel63]

My local app was extremely slow when accessing via Firefox, fast when
accessing via IE.  I have no idea what caused the difference.

I just recently discovered that accessing the app via http://127.0.0.1:8080
(rather than http://localhost:8080) made all the difference, and now
FF is fast, too.  Would be nice if someone could explain why, but I'm
just happy to have things fast now.

Of course, all this is on Vista, and may not apply to you.

On Sep 8, 10:19 am, GAEfan ken...@gmail.com wrote:
 Are there any tricks to speeding up reads/writes to the local
 datastore?

 Are there any throttles installed in the SDK that cap the access rate?

 I have a quad core Mac Pro, and this seems rather slow accessing
 locally.

 Thanks.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Facebook Integration?

[Devel63]

Can anyone point me in the direction of a good resource for Facebook
Connect/GAE integration?

Neither the ShelfTalkers article (http://code.google.com/appengine/
articles/shelftalkers.html), nor PyFacebook (upon which ShelfTalkers
relies) appear to work anymore.  For example, PyFacebook relies on an
'add' parameter in the session cookie which is not (any longer?)
there.

This post (http://forum.developers.facebook.com/viewtopic.php?
id=23276) enabled me to move forward (just look for the 'user' param
and set 'added' true if present), but it's scary to be relying on
something that's possibly no longer maintained.

Is PyFacebook still maintained and working?

And how do people test this FB stuff on a local server (I'm having to
upload to Google every time)?  The ShelfTalkers article implied I
could just set the FB callback URL to 127.0.0.1:8080, but either that
or something else is causing weird failures.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Merge-Join Performance?

[Devel63]

I've watched this video, so I watched again.  At first he says he
thinks filter order might matter, but then corrects himself based on
input from the author of the code to say that filter order has a very
small effect.

On Sep 2, 5:52 am, Paul Kinlan paul.kin...@gmail.com wrote:
 There is a video about building scalable apps by Brett Slatkin 
 onhttp://code.google.com/io
 At the end of the video it is hinted (I believe - I could be wrong) that the
 order of your filters matters, so that means that you should choose key with
 the highest entropy first (my terminology might be incorrect) but if your
 data is not very unique it should be used later in the merge join query.

 Paul

 2009/9/2 Jai sharma...@gmail.com



  Also does it use the keyword histogram to optimize the set-
  intersection performance(AND queries) in merge-join?

  Regards,
  Jai

  On Sep 1, 12:07 pm, Devel63 danstic...@gmail.com wrote:
   I'm seeing very slow times on some merge-join (zig zag) queries, and
   just want to confirm that App Engine is smart about these things.

   In particular, if I have a poperty for which most records have an
   identical value, does GAE does have to scan the related entire index
   linearly, or, if I'm in the middle of a merge join, does it
   intelligently find a good starting point based on the desired __key__
   value?

   Imagine 1M records with a db.BooleanProperty set to False...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Merge-Join Performance?

[Devel63]

I'm seeing very slow times on some merge-join (zig zag) queries, and
just want to confirm that App Engine is smart about these things.

In particular, if I have a poperty for which most records have an
identical value, does GAE does have to scan the related entire index
linearly, or, if I'm in the middle of a merge join, does it
intelligently find a good starting point based on the desired __key__
value?

Imagine 1M records with a db.BooleanProperty set to False...


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to Delete Property from Model?

[Devel63]

Perfect, thanks.  I was just missing the step about temporarily
converting to db.Expando.

On Aug 26, 12:15 pm, vivpuri v...@vivekpuri.com wrote:
 Read this article:

 http://code.google.com/appengine/articles/update_schema.html

 -V
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Clean way to code around timeouts?

[Devel63]

Got it, thanks.

On Aug 25, 11:50 am, Alkis Evlogimenos ('Αλκης Ευλογημένος)
evlogime...@gmail.com wrote:
 @retry_on_timeout(retries=10, secs=0.2)
 def some_idempotent_function():  # do stuff

 If you do not give retries or secs it defaults to 3 retries with 1 sec
 starting delay. The delay is exponential, it doubles after each retry.

 2009/8/25 Devel63 danstic...@gmail.com





  Can you give an example as to how this is used?  I understand the
  purpose, I'm just a little hazy on the calling syntax.

  On Aug 24, 5:25 am, Alkis Evlogimenos ('Αλκης Ευλογημένος)
  evlogime...@gmail.com wrote:
   You can make it into a decorator which will make it easier for your
   functions to code. I use this:
   def retry_on_timeout(retries=3, secs=1):
     A decorator to retry a given function performing db operations.
     def _decorator(func):
       def _wrapper(*args, **kwds):
         tries = 0
         while True:
           try:
             tries += 1
             return func(*args, **kwds)
           except db.Timeout, e:
             logging.debug(e)
             if tries  retries:
               raise e
             else:
               wait_secs = secs * tries ** 2
               logging.warning(Retrying function %r in %d secs % (func,
   wait_secs))
               time.sleep(wait_secs)
       return _wrapper
     return _decorator

   On Mon, Aug 24, 2009 at 1:53 PM, Bemmu bemmu@gmail.com wrote:

I decided to finally do something about the Timeout exceptions
littering my log.

I read somewhere on this forum that I am supposed to code around data
store accesses to try things out several times in case of timeouts. Is
this still necessary? Why won't the methods just do that internally?

This is my first attempt to handle a timeout situation, is there any
nicer way to code this?

       retries = 3
       retry = True
       while retry:
               try:
                       retry = False
                       recipient.put()
               except:
                       retries -= 1
                       if retries  0:
                               retry = True
                               logging.info(recipient.put() failed,
retrying)
                       else:
                               logging.error(failed even after trying
recipient.put() three
times)

   --

   Alkis

 --

 Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: keys_only does not work with Searchable

[Devel63]

Thanks for the workaround!

On Aug 25, 5:19 pm, Jeff S (Google) j...@google.com wrote:
 Hi Devel63,

 Yes I see what you mean. The keys_only argument should be added to the 'all'
 class method. As a temporary workaround, you can do

 query = MySearchableModel.all()
 query._keys_only = True

 Thanks for pointing this out,

 Jeff

 On Thu, Aug 20, 2009 at 5:36 PM, Devel63 danstic...@gmail.com wrote:

  It seems that the keys_only flag in the Kind.all(keys_only=True) does
  not work/compile for the search.Searchable model.

  I get an 'unexpected keyword' error msg.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Memcache - how much space?

[Devel63]

Can someone from Google comment on this?  This seems pretty
important...

You don't have to give a hard number, but 8K items isn't much, and
could cause some re-architecting to be needed.

On Aug 25, 6:13 am, Colin aliq...@gmail.com wrote:
 My memcache stats, expire time is 48h, non of the itemd are older than
 5h, but often items that should be in the cache aren't there:
 - items: 8423
 - bytes: 10655912

 I (wrongly) assumed that i hit the size limit once i had 25mb of data
 in memcache, i optimized (compressed and pickled data before putting
 it in the cache) and now i hit the limit  at roughly 10mb :/
 Thus my conclusion: there is an item limit at about 8k.

 On 13 aug, 17:11, ted stockwell emorn...@gmail.com wrote:

  Hi Jeff,

  A question
  Can I assume that thememcachesize limit will increase as the number
  of servers running my application increases?

  On Aug 10, 5:53 pm, Jeff S (Google) j...@google.com wrote:

   The overallmemcachesize limit is not set in stone so we usually don't give
   a hard number as whatever I say might soon be out of date. I recommend 
   using
   as you need it because if your app goes over it's allotment, then the
   less-used data is evicted.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] How to Delete Property from Model?

[Devel63]

I would like to change the definition of a db.Model (and a
search.Searchable), removing some properties that are no longer
needed.

What's the best way to clean up existing entities in the DB?

I tried setting all the unwanted property values to None for every
entity, then uploading the new definition.  But the DataViewer still
shows those old property slots with null values, whereas new
entities show the property slots with missing.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Clean way to code around timeouts?

[Devel63]

Can you give an example as to how this is used?  I understand the
purpose, I'm just a little hazy on the calling syntax.

On Aug 24, 5:25 am, Alkis Evlogimenos ('Αλκης Ευλογημένος)
evlogime...@gmail.com wrote:
 You can make it into a decorator which will make it easier for your
 functions to code. I use this:
 def retry_on_timeout(retries=3, secs=1):
   A decorator to retry a given function performing db operations.
   def _decorator(func):
     def _wrapper(*args, **kwds):
       tries = 0
       while True:
         try:
           tries += 1
           return func(*args, **kwds)
         except db.Timeout, e:
           logging.debug(e)
           if tries  retries:
             raise e
           else:
             wait_secs = secs * tries ** 2
             logging.warning(Retrying function %r in %d secs % (func,
 wait_secs))
             time.sleep(wait_secs)
     return _wrapper
   return _decorator



 On Mon, Aug 24, 2009 at 1:53 PM, Bemmu bemmu@gmail.com wrote:

  I decided to finally do something about the Timeout exceptions
  littering my log.

  I read somewhere on this forum that I am supposed to code around data
  store accesses to try things out several times in case of timeouts. Is
  this still necessary? Why won't the methods just do that internally?

  This is my first attempt to handle a timeout situation, is there any
  nicer way to code this?

         retries = 3
         retry = True
         while retry:
                 try:
                         retry = False
                         recipient.put()
                 except:
                         retries -= 1
                         if retries  0:
                                 retry = True
                                 logging.info(recipient.put() failed,
  retrying)
                         else:
                                 logging.error(failed even after trying
  recipient.put() three
  times)

 --

 Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Question on increasing performance..

[Devel63]

Makes sense, but in this case I may need to wrap past the end of the
collection, so I need to know how many entities satisfied the request
before the end of the collection, so I know how many to skip over from
the beginning.

So, I still have the original question...

On Aug 25, 5:32 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 On Mon, Aug 24, 2009 at 6:28 PM, Devel63 danstic...@gmail.com wrote:

  If you only want the 3rd object, do you think it's faster

   -- Just do a normal fetch(3)
   -- Do a keys_only fetch(3), then a get of the 3rd key?

 Neither - use fetch(1, offset=2).

 -Nick Johnson





  On Aug 23, 4:41 pm, djidjadji djidja...@gmail.com wrote:
   Your query will probably fetch more then 25 Action objects. Fetching
   all those Keys is fast. Fetching all the complete objects is slow, you
   fetch a lot more then you need.

   If you iterate over the GQL query, no fetch, that requests full Action
   objects (SELECT * FROM ACTION...), the objects will be fetched in
   batches of 20. You then only fetch a few more objects then needed.

   Performing XX_multi operations on the memcache will also help reduce
   the response time.

   2009/8/23 bvelasquez bvelasq...@gmail.com:

   http://pastie.org/592489

If you can help me answer this question, I would appreciate it.

The above code takes the following ~ time: 1370ms 1725cpu_ms
670api_cpu_ms.

Changing the query to return the Actions bumps it up to : 2311ms
3050cpu_ms 1018api_cpu_ms

Changing the Action.get() to use the list of Keys from the SELECT
takes ~ 2000cpu_ms.

I was surprised that doing an individual get on each key was faster
than passing the array into get or returning the actions with the
query.

What are your suggestions for optimizing this, if anything is
obviously wrong or maybe there is another way completely for achieving
the same results, which my limited experience is not telling me.

A couple notes.  Project is a ReferenceProperty in Action.  I'm
getting the key and pulling that from memcache if it is available.

I'm breaking on count = 25 because I only want 25 results where the
Actions project is active and not deleted.  I cannot limit the fetch
on 25 because those 25 might be in an inactive Project.

Thanks!

 --
 Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Question on increasing performance..

[Devel63]

If you only want the 3rd object, do you think it's faster

 -- Just do a normal fetch(3)
 -- Do a keys_only fetch(3), then a get of the 3rd key?

On Aug 23, 4:41 pm, djidjadji djidja...@gmail.com wrote:
 Your query will probably fetch more then 25 Action objects. Fetching
 all those Keys is fast. Fetching all the complete objects is slow, you
 fetch a lot more then you need.

 If you iterate over the GQL query, no fetch, that requests full Action
 objects (SELECT * FROM ACTION...), the objects will be fetched in
 batches of 20. You then only fetch a few more objects then needed.

 Performing XX_multi operations on the memcache will also help reduce
 the response time.

 2009/8/23 bvelasquez bvelasq...@gmail.com:



 http://pastie.org/592489

  If you can help me answer this question, I would appreciate it.

  The above code takes the following ~ time: 1370ms 1725cpu_ms
  670api_cpu_ms.

  Changing the query to return the Actions bumps it up to : 2311ms
  3050cpu_ms 1018api_cpu_ms

  Changing the Action.get() to use the list of Keys from the SELECT
  takes ~ 2000cpu_ms.

  I was surprised that doing an individual get on each key was faster
  than passing the array into get or returning the actions with the
  query.

  What are your suggestions for optimizing this, if anything is
  obviously wrong or maybe there is another way completely for achieving
  the same results, which my limited experience is not telling me.

  A couple notes.  Project is a ReferenceProperty in Action.  I'm
  getting the key and pulling that from memcache if it is available.

  I'm breaking on count = 25 because I only want 25 results where the
  Actions project is active and not deleted.  I cannot limit the fetch
  on 25 because those 25 might be in an inactive Project.

  Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] First __key__?

[Devel63]

It would make my paging code a lot cleaner if there were a way to
indicate 'the first key'.  Is there?  Like db.Key.from_path
(MyModel.kind(), ) when using key_names?

That way, I could compile my GQL and just keep plugging in a new :key
value each iteration, including the first.

I know, the example cases always look clean with their starting query,
and then the follow-ups, but in real life I sometimes find myself
pretty nested with various logic clauses such that it would be nice to
just be able to do this.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] How Many References Allowed?

[Devel63]

Is there a limit to the number of entities that can reference a given
entity?

For example,

  class Class1(db.Model):
prop1 : db.IntegerProperty()

  class Class2(db.Model):
ref_prop: db.ReferenceProperty(Class1, collection=my_references)

Is there a limit to the number of class2 entities that can refer to
a given class1 entity?

[Sorry for the repost: I can't seem to get this findable]
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] How Many References Allowed?

[Devel63]

Is there a limit to the number of entities that can reference a given
entity?

For example,

  class Class1(db.Model):
prop1 : db.IntegerProperty()

  class Class2(db.Model):
ref_prop: db.ReferenceProperty(Class1, collection=my_references)

Is there a limit to the number of class2 entities that can refer to a
given class1 entity?

[Sorry for the repost: my initial post is not findable]
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Simple App Engine Recipe?

[Devel63]

I'd like to be able to find all Articles that contain certain words
and have been tagged by a given user.

Is there a good App Engine way to do this?

I started down this path:
- Articles are db.Searchable entities
- Each user tag creates a new UserTag entity
 userID : db.UserProperty
 article : db.ReferenceProperty(Article,
 collection_name='user_tags')
 other props about the given user tag (date, stars, etc.)
- At query time, find all articles that contain the keywords and
  are referenced by a UserTag for userID

First, is there a better way to do this?

If not, my problem is enforcing the query constraint.  For this to
work, I think UserTag must have a key_name that includes the userID,
say, userID_articleID.  But how do I actually phrase the query?
- Article.all().search(words).filter('user_tags ',
   userID+'_').filter('user_tags ', userID+'z').fetch(100)

But this won't work because those are key_names, not
keys.  Any ideas?

[Sorry for the repost: my first version is not findable]
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] keys_only does not work with Searchable

[Devel63]

It seems that the keys_only flag in the Kind.all(keys_only=True) does
not work/compile for the search.Searchable model.

I get an 'unexpected keyword' error msg.

[Tried to post this a few days ago, but it's not findable]
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Limit to Reference Collection Size?

[Devel63]

Is there a limit to the number of entities that can reference a given
entity?

For example,

  class Class1(db.Model):
prop1 : db.IntegerProperty()

  class Class2(db.Model):
ref_prop: db.ReferenceProperty(Class1, collection=my_references)

  Is there a limit to the number of class2 entities that can refer to
a given class1 entity?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Simple Common Query is Hard?

[Devel63]

I'd like to be able to find all Articles that contain certain words
and have been tagged by a given user.

Is there a good App Engine way to do this?

I started down this path:
  - Articles are their own db.Searchable entities
  - Each user tag creates a new UserTag
 userID : db.UserProperty
 article : db.ReferenceProperty(Article,
collection='user_tags')
 other properties about the given user tag (date, stars, etc.)
  - At query time, find all articles that contain the keywords and are
referenced by a UserTag for userID

First, is there a better way to do this?

If not, my problem is enforcing the query constraint.  For this to
work, I think UserTag must have a key_name that includes the userID,
say, userID_articleID.  But how do I actually phrase the query?
  - Article.all().search(words).filter('user_tags ', userID
+'_').filter('user_tags ', userID+'z').fetch(100)

But I don't think this will work because those are key_names, not
keys.  Any ideas?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] keys_only does not work with Searchable

[Devel63]

It seems that the keys_only flag in the Kind.all(keys_only=True) does
not work/compile for the search.Searchable model.

I get an 'unexpected keyword' error msg.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: google-app-engine-django

[Devel63]

I am very interested in this also.  There are features of the Patch
I'd really like to use, but I keep hearing that startup times (for new
machine instantiations, which we all know happens quite frequently)
are significantly higher than bundled Django, which in turn are
significantly higher than using just web_app.

Can anyone quantify this?

On Aug 15, 1:53 am, Waldemar Kornewald wkornew...@gmail.com wrote:
 Hi Andy,

 On Aug 14, 7:05 pm, Andy a...@robotlovesyou.com wrote:

  I've not compared appengine-patch and google-app-engine-django for
  performance but when using google-app-engine-django r86 (which used
  django 1.0 via zipimport) I was getting a high-cpu warning for every
  second or third page load and for each of these pages the logs showed
  that zipimport wa re-loading django. When I moved across to google-app-
  engine-django r90, which uses the built in django 1.0 libraries this
  issue went away and my cpu usage and page load times dropped
  significantly.

 How did you check this? Did you go to the logs and then select
 Minimum severity: Requests only? The zipimporter always adds INFO
 logging messages, so you'll notice Django loads. Without zipimport you
 have to go to the request level to see Django loads. I'm pretty sure
 you'll find some requests with unusually high CPU and long response
 times.

 I'm really interested in some real-world data on whether there is a
 difference between integrated and external libraries - this is not an
 aep vs helper battle.

 Bye,
 Waldemar Kornewald
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Need Index Quota Reset

[Devel63]

Need Index Quota rest on lifehints-beta.  Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to translate an application running on GAE?

[Devel63]

In the templates, use {% trans 'My string' %}.

Put {% load i18n %} at the top of the file (must be one of first
couple lines).

On Aug 1, 11:22 am, Emilien Klein emilien.kl...@gmail.com wrote:
 Hi Devel63,
 Thanks for your answer. But this is just for inside your Python files,
 how do you manage text in the templates? Or don't you use templates? I
 don't really see how you're managing this...
 Is your app open source (i.e. Is there a place where I can see your
 code)?

 On 1 août, 10:31, Devel63 danstic...@gmail.com wrote:

  We use the Django 0.96 that is bundled with GAE.

      from django.utils.translation import gettext as _

  Then just use _('My String) in the code, combined with the .po/.mo
  files.

  On Aug 1, 8:16 am, Rodrigo Moraes rodrigo.mor...@gmail.com wrote:

   On Sat, Aug 1, 2009 at 11:51 AM, Emilien Klein wrote:
The problem I see with using Babel is that it IS an external library,
which means that I need to integrate it with my project. What I'm
looking for is a way to have a translated application WITHOUT having
to install any external library...

   Without any other external library, you have 2 options:

   1. use gettext -http://docs.python.org/library/gettext.html
   2. hack and patch django's i18n

   The problem with the 2nd option is that your app stays tied to
   django's ecosystem - later you need i18n in a small project and you'll
   have to add django just for it. Ok, it is a valid solution, I just
   personally prefer to use a external library that is not coupled with a
   bunch of unrelated things, and avoid all the monkeypatching mess.

   -- rodrigo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to translate an application running on GAE?

[Devel63]

We use the Django 0.96 that is bundled with GAE.

from django.utils.translation import gettext as _

Then just use _('My String) in the code, combined with the .po/.mo
files.


On Aug 1, 8:16 am, Rodrigo Moraes rodrigo.mor...@gmail.com wrote:
 On Sat, Aug 1, 2009 at 11:51 AM, Emilien Klein wrote:
  The problem I see with using Babel is that it IS an external library,
  which means that I need to integrate it with my project. What I'm
  looking for is a way to have a translated application WITHOUT having
  to install any external library...

 Without any other external library, you have 2 options:

 1. use gettext -http://docs.python.org/library/gettext.html
 2. hack and patch django's i18n

 The problem with the 2nd option is that your app stays tied to
 django's ecosystem - later you need i18n in a small project and you'll
 have to add django just for it. Ok, it is a valid solution, I just
 personally prefer to use a external library that is not coupled with a
 bunch of unrelated things, and avoid all the monkeypatching mess.

 -- rodrigo
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Getting around joins/in queries

[Devel63]

I haven't given your situation much thought, but you could write the
list to memcache and have a cron job that reads memcache and does the
writing.  You don't get instant results, but the user doesn't have to
wait.

Caution: you aren't going to be able to do queries that return 500 or
1000 results anyway ... it will probably take too long to return.  As
I said, I haven't taken the time to really understand what you are
doing, but perhaps you will need to rethink the architecture.

On Jul 27, 12:10 pm, Taylor tgaut...@gmail.com wrote:
 Hi,

 I have a facebook app that naturally would like to perform set
 operations on a logged in user's friends against the data in the
 application (stored in app engine).  This presents a particular
 challenge, suppose for example that in my application a logged in user
 can subscribe to a particular event, we then have some classes like
 this:

 User {
    Long key;
    Long facebookId;
    // etc...

 }

 Event {
   Long key;
   String name;
   // etc...

 }

 Subscribed {
   Long userKey;
   Long eventKey;
   Long start;
   Long end;
   //etc...

 }

 Now, if I have a particular user logged in, who has a set of friends,
 I want to know which events that set of friends have subscribed to.
 A normal query would be SELECT * from SUBSCRIBED where userkey in
 ( the set of friends )

 By now, I know that there are no conditional OR operations, joins, or
 in queries allowed in app engine.

 My initial attempt to solve this problem has resulted in writing out
 the intersection at subscription time using an additional table,
 something like:

 FriendSubscribed extends Subscribed {
   Long friendKey;
   // etc...

 }

 // for every friend of the logged in user, write a record so the query
 can simply pick up
 // all records for that friend
 void persistForFriends(CollectionLong friends, Subscribed
 subscribed) {
   for (Long id : friends) {
     pm.persist(new FriendSubscribed(id, subscribed));
   }

 }

 Now I have shifted the query burden to the write side and I can do a
 simple query on the FriendSubscribed table for a particular user which
 is nice and fast: SELECT * from FRIENDSUBSCRIBED where friendkey =
 myuserid.  This returns all the subscriptions that the logged in
 user's friends have made (with some slight gotchas, e.g. if the user
 adds a new friend, then any subscriptions that the new friend already
 has will not be seen - this is tolerable)

 So, what's the problem?  On facebook, a user typically has order 100
 friends.  And I suspect many will have 500 and a few will have 1000+.

 Even for order 100 friends, at the current write speed (5/s) I can
 expect a write operation to take approx. 20s!!  This is rather long
 for a web operation, and gets worse as you increase the number of
 friends.

 I am curious if anyone has any ideas for solutions?  I have a few
 thoughts, but wanted to see what people thought before moving on to
 the next optimization.  Here are my thoughts:

 1) Queue the friend write operation - it's not critical to the core
 write operation that all the join data be written out, it can always
 be re-computed at any point in time by analyzing the Subscribed
 records and current friend list for each friend.  Only problem is I am
 using Java and the scheduled tasks API is not yet supported.
 Furthermore, it still seems like a lot of work to go through if we
 imagine thousands or hundreds of thousands of users making
 subscriptions as each write operation they do gets multiplied by a
 factor of 100-1000.

 2) Make an owned relationship from Events to Subscribed. Something
 like:

 Event {
   Long key;
   String name;

   SetSubscribed subscribed;
 // etc...

 }

 Now reading an event records gives precisely the information I want
 and simply has to be culled down (the intersection of the logged in
 user's friends and the Event subscribed list is the list of friends
 that are subscribed to that event).  The problem I see with this
 implementation is high contention.  For every subscription event I
 have to lock out updates and make sure everyone is writing a coherent
 view of the set (don't we have to update the value into the Event
 record?)  Or I may be wrong and it may be that the ownership
 relationship is maintained automatically by the datastore and this
 isn't as much of a concern as I think it is as the owned relationship
 doesn't really write anything into the owner record, it just does a
 join-like query to populate the set on read...that might be ideal??

 3) ??  Something else?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Contact Application Owner?

[Devel63]

Is there a way to contact the owner of a given application ID?

I ask because someone has registered an app using my company's name,
and we've had the domain.com for years.  I don't know if it was
someone from my company (past or present) doing research, or a third
party who doesn't realize that the name is taken.

Normally these things don't matter, but since GAE doesn't support
HTTPS on an external domain, occasionally pages do need to be served
from appname.appspot.com.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Contact Application Owner?

[Devel63]

No such gmail address.  Also, it's someone who registered the name,
but hasn't deployed anything to it.

On Jul 24, 8:45 am, Barry Hunter barrybhun...@googlemail.com wrote:
 Have you tried contacting appn...@gmail.com ?

 You can't register apps for already existing gmail addresses (except your 
 own),

  if a valid gmail address you could ask them to signup for appengine,
 and then transfer ownership to you.

 2009/7/24 Devel63 danstic...@gmail.com:



  Is there a way to contact the owner of a given application ID?

  I ask because someone has registered an app using my company's name,
  and we've had the domain.com for years.  I don't know if it was
  someone from my company (past or present) doing research, or a third
  party who doesn't realize that the name is taken.

  Normally these things don't matter, but since GAE doesn't support
  HTTPS on an external domain, occasionally pages do need to be served
  from appname.appspot.com.

 --
 Barry

 -www.nearby.org.uk-www.geograph.org.uk-
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Need Datastore Indices Count Reset

[Devel63]

The app name is judysapps-qa

Strange: this was reset a couple of weeks ago, we haven't changed any
index definitions since then, and are well below the index quota.

Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Contact Application Owner?

[Devel63]

Google looked into it and told me it had been registered.

On Jul 24, 1:21 pm, Barry Hunter barrybhun...@googlemail.com wrote:
 2009/7/24 Devel63 danstic...@gmail.com:



  No such gmail address.  Also, it's someone who registered the name,
  but hasn't deployed anything to it.

 How can you tell? I get the same 404 for a appid registered but not
 deployed (one of mine) and a bogus one.



  On Jul 24, 8:45 am, Barry Hunter barrybhun...@googlemail.com wrote:
  Have you tried contacting appn...@gmail.com ?

  You can't register apps for already existing gmail addresses (except your 
  own),

   if a valid gmail address you could ask them to signup for appengine,
  and then transfer ownership to you.

  2009/7/24 Devel63 danstic...@gmail.com:

   Is there a way to contact the owner of a given application ID?

   I ask because someone has registered an app using my company's name,
   and we've had the domain.com for years.  I don't know if it was
   someone from my company (past or present) doing research, or a third
   party who doesn't realize that the name is taken.

   Normally these things don't matter, but since GAE doesn't support
   HTTPS on an external domain, occasionally pages do need to be served
   from appname.appspot.com.

  --
  Barry
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Outdated Files Being Served?

[Devel63]

When I upload a new site.css file, my browser sometimes seems to get
the old version, or the new version, randomly with each call.

I have cleared my browser cache, tried it on different computers,
etc.

Even with cleared browser caches, when I happen to get the old version
I don't see a record of the access in the server logs.

Does the Internet cache old versions of my file, and not go back to
the server?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Strange App Name Conflict - Google?

[Devel63]

I tried to register an app today, and got an error message that An
App or Google Account already exists with that name.

But, there is no email address appn...@gmail.com, there is no app
active at the requested appname, it's my company's name, and FWIW,
we've had the domain.com for years.

We do use Google Apps for our internal email.   Does that constitute
an illegal conflict?  If we have everything on appname.com, can we not
register an app named appname?

Normally, this wouldn't matter, but since you actually need to use
appname.appspot.com when doing anything over https, I guess the name
does sometimes matter.

Is there any way to track this down and see what's going on?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Application Updates but Does Not Serve New Files?

[Devel63]

Nick and I emailed privately, but for anyone else who is interested,
CSS files were not being updated.

This occurred in both FF and IE with flushed browser caches.  I
suppose it's possible these were being cached somewhere in the
Internet in between me and Google, but it eventually resolved itself
whatever the cause.

On Jul 15, 1:32 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Devel63,

 We need to know what files you're serving that appear to be old, and
 what they should be serving. My money is on a client-side caching
 issue, but it's impossible to tell without more details.

 -Nick Johnson

 On Tue, Jul 14, 2009 at 11:29 PM, Devel63danstic...@gmail.com wrote:

  Today I am suddenly unable to effectively update my app.

  appcfg.py appears to update the app, the serving version is set
  correctly, but the old files are being served.

  The app name is judysapps-qa

  Any ideas?

 --
 Nick Johnson, App Engine Developer Programs Engineer
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
 Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Need Index Count Reset by Google

[Devel63]

Can someone at Google please reset the index count for judysapps-qa?

Separately, I no longer appear able to successfully upload new
versions.  appcfg.py reports success, but the files being served are
the old ones.

Yes, I have checked the default version is correct and my browser
cache has been emptied. Is this related to the index count error, or
something else?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Application Updates but Does Not Serve New Files?

[Devel63]

Today I am suddenly unable to effectively update my app.

appcfg.py appears to update the app, the serving version is set
correctly, but the old files are being served.

The app name is judysapps-qa

Any ideas?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: incorrect files served

[Devel63]

Yes, I am seeing this (for the first time) today with my app as well.
I hope it resolves itself!

Nick, I will send my app id.

On Jul 13, 9:27 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Jay,

 Can you post the app's ID and the URLs in question? In a private
 reply, if you're not comfortable linking to them here.

 -Nick Johnson



 On Mon, Jul 13, 2009 at 5:14 PM, Jayjbaker.w...@gmail.com wrote:

  Has anyone had issues with incorrect files being served? I have an app
  running on the python runtime. Part of it serves up static images from
  files - these are not in the datastore, but files.

  A couple of weeks ago when Google had their file system problem, my
  app started serving incorrect files. For example, instead of picA.jpg,
  it was serving picB.jpg. This was not a function of the app itself, I
  confirmed this by hitting the images with their URI directly.

  Today this started happening again. It is not every image, just some.
  And it consistently does it, i.e. the files that are wrong are random,
  but they stay that way.

  The last time this happened, it just fixed itself, but it took hours.

  Again, I'm wondering if other people have seen this.

  Thanks.

 --
 Nick Johnson, App Engine Developer Programs Engineer
 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
 Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: incorrect files served

[Devel63]

This started working correctly for me, and then reverted again to
serving the old files.

FWIW, it also seems to be taking twice as long to spool up a new
instance.

On Jul 14, 3:33 pm, Devel63 danstic...@gmail.com wrote:
 Yes, I am seeing this (for the first time) today with my app as well.
 I hope it resolves itself!

 Nick, I will send my app id.

 On Jul 13, 9:27 am, Nick Johnson (Google) nick.john...@google.com
 wrote:

  Hi Jay,

  Can you post the app's ID and the URLs in question? In a private
  reply, if you're not comfortable linking to them here.

  -Nick Johnson

  On Mon, Jul 13, 2009 at 5:14 PM, Jayjbaker.w...@gmail.com wrote:

   Has anyone had issues with incorrect files being served? I have an app
   running on the python runtime. Part of it serves up static images from
   files - these are not in the datastore, but files.

   A couple of weeks ago when Google had their file system problem, my
   app started serving incorrect files. For example, instead of picA.jpg,
   it was serving picB.jpg. This was not a function of the app itself, I
   confirmed this by hitting the images with their URI directly.

   Today this started happening again. It is not every image, just some.
   And it consistently does it, i.e. the files that are wrong are random,
   but they stay that way.

   The last time this happened, it just fixed itself, but it took hours.

   Again, I'm wondering if other people have seen this.

   Thanks.

  --
  Nick Johnson, App Engine Developer Programs Engineer
  Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
  Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Querying for N random records on Appengine datastore

[Devel63]

Barry, I understand your objections below, but do you have a better
approach?

Assigning random numbers to entities is guaranteed to be worse.  If
you are worried about an entity being deleted and opening a gap in the
sequence, imagine the thousand-fold gaps you will see with random ID
generation (e.g. 1, 10001, 10002, 2, ...).

See below.

On Jul 10, 3:03 pm, Barry Hunter barrybhun...@googlemail.com wrote:
 On 10/07/2009, Devel63 danstic...@gmail.com wrote:



   The best way is to assign a one-up counter to each record as you
   create it, then call random.randint(1,max_counter) to determine the
   desired record.

   To retrieve multiple random entities in a query, do a filter('IN ',
   [my random nums]).

 doent work that well once records start getting deleted (get the same
 issue, non uniform distribution)

If an app needs to support entity deletion, you can still ensure
uniformity by running a periodic cron job to compress the counter
sequence.


 nor does it work if you filtering at the same time :(


Correct, in that the distribution is no longer uniform.  But this is
also true of the random ID approach. I admit that the random ID
approach seems appealing at first, but when you actually look into it,
you'll find that you are guaranteed that many results will be 3X more
likely than others, or worse.  It IS better in the case that you want
to randomize based on time of entity creation, but there are other
ways to deal with this.

I would love it if someone could come up with a good way to do true
random results of an arbitrary query set!!



   Note that behind the scenes this generates multiple queries, so you're
   not saving much time.

   On Jul 10, 7:34 am, Wooble geoffsp...@gmail.com wrote:
    Highly non-optimal solution: have a cron job assign new random numbers
    to your entities often enough to simulate randomness.  Even just re-
    assigning numbers to entities that have been previously selected might
    work.  This involves a lot more CPU as you'd be doing writes, but
    shifts the work from request time to a background process so your
    users don't see the added latency for doing N queries.

    Another possible solution would be to fetch keys only for X*N entities
    (where greater X's produce more apparent randomness) then choose N of
    those keys to actually fetch entities.

    On Jul 9, 12:33 pm, aloo aleem.maw...@gmail.com wrote:

     Hi all,

     I'm trying to write a GQL query that returns N random records of a
     specific kind. My current implementation works but requires N calls to
     the datastore. I'd like to make it 1 call to the datastore if
     possible.

     I currently assign a random number to every kind that I put into the
     datastore. When I query for a random record I generate another random
     number and query for records  rand ORDER BY asc LIMIT 1.

     This works, however, it only returns 1 record so I need to do N
     queries. Any ideas on how to make this one query? Thanks.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Querying for N random records on Appengine datastore

[Devel63]

The best way is to assign a one-up counter to each record as you
create it, then call random.randint(1,max_counter) to determine the
desired record.

To retrieve multiple random entities in a query, do a filter('IN ',
[my random nums]).

Note that behind the scenes this generates multiple queries, so you're
not saving much time.

On Jul 10, 7:34 am, Wooble geoffsp...@gmail.com wrote:
 Highly non-optimal solution: have a cron job assign new random numbers
 to your entities often enough to simulate randomness.  Even just re-
 assigning numbers to entities that have been previously selected might
 work.  This involves a lot more CPU as you'd be doing writes, but
 shifts the work from request time to a background process so your
 users don't see the added latency for doing N queries.

 Another possible solution would be to fetch keys only for X*N entities
 (where greater X's produce more apparent randomness) then choose N of
 those keys to actually fetch entities.

 On Jul 9, 12:33 pm, aloo aleem.maw...@gmail.com wrote:

  Hi all,

  I'm trying to write a GQL query that returns N random records of a
  specific kind. My current implementation works but requires N calls to
  the datastore. I'd like to make it 1 call to the datastore if
  possible.

  I currently assign a random number to every kind that I put into the
  datastore. When I query for a random record I generate another random
  number and query for records  rand ORDER BY asc LIMIT 1.

  This works, however, it only returns 1 record so I need to do N
  queries. Any ideas on how to make this one query? Thanks.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Gae is down ?

[Devel63]

Still problematic here; haven't been able to upload new version for a
couple of hours.

On Jun 26, 9:58 pm, cz czer...@gmail.com wrote:
 Can't upload a new version either: 500 internal server error
 The dashboard is inaccessible as well.
 App is slow but works.

 On Jun 26, 9:55 pm, gg bradjyo...@gmail.com wrote:

  Seems to be just the admin...

  On Jun 26, 9:52 pm, Tom Wu service.g2...@gmail.com wrote:

    Server Error

   A server error has occurred.

   Return to Applications screen » http://appengine.google.com/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] 403 Application Over Quota Problem - Not True!

[Devel63]

All of a sudden, my app is returning 403 application over quota
whenever I do anything a bit strenuous.

All of the quotas are WAY under, but things that used to work fine are
now triggering this message.

A guess is that the budgeting process has become much more fine-
grained, and is mistakenly extrapolating from one request that may do
a number of DB writes and take 10 seconds.  But these are extremely
rare.

The app name is judysapps-qa.


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Shorten keys on URLs

[Devel63]

Use the key_name or key_id in the URL, then use Model.get_by_key_name
or db.Key.from_path to get the entity when asked.

On Jun 13, 4:05 pm, david da...@altasoftware.com wrote:
 Hello everyone,

 I've been looking for a way to shorten keys on URLs, for example: I
 learned that an entity key is built from the app id and a path to that
 entity using base64, so basically all of the keys for my application
 have the same prefix which I could remove from key strings when
 putting them in URLs and preppend the common prefix every time I'm
 going to process one of those. Have anybody done something similar or
 does anybody have and advice on that matter?

 Thanks.

 David
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Bulkloader and Preserving Key_Names

[Devel63]

Thanks very much for the helpful info.  Generate_key is a great find,
and it's useful to know that we should waste out time trying to export
key_names at this time.

As for preserving reference links, do we need to create a translation
table [entity type, old_key, new_key] and do writes to the table as we
upload each entity, and reads to determine new reference value as we
upload references?

Or is there some, more built-in way to handle this?

On May 18, 5:10 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Dan,

 On Sun, May 17, 2009 at 1:24 AM, Devel63 danstic...@gmail.com wrote:

  Is there some way to export key_names and then later import them using
  the bulkloader?

 Unfortunately, the exporter code doesn't provide any easy way to
 export key names. Your best bet is probably to extend output_entities,
 which I agree is not ideal.



  I've seen the examples from folks messing around with Handle_Entity,
  but that seems pretty kludgy (and dangerous, given that this is inside
  the source, and changed last release).

 If you just want to determine the key name, you can override
 generate_key:http://code.google.com/p/googleappengine/source/browse/trunk/python/g...
 . Overriding handle_entity is explicitly encouraged, though - as the
 docstring states: Subclasses can override this to add custom entity
 conversion code..



  And key_name aside, what about preserving Reference links in general.
  How is that done?

 You can write a function to modify it on download, or on upload -
 whatever works best for your particular situation.

 -Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Bulk upload specifying a key name

[Devel63]

Fantastic, Nick, thanks for the info.

Is there an equally simple way to output the key_name on bulk
download?

On May 18, 5:00 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi ecognium,

 You can do this by implementing the generate_key method in your
 bulkloader.Loader class. See the code for 
 details:http://code.google.com/p/googleappengine/source/browse/trunk/python/g...

 -Nick Johnson

 On Mon, May 18, 2009 at 4:54 AM, ecognium ecogn...@gmail.com wrote:

  Hi All,
     How do i specify a key name when using the bulk upload tool? I am
  currently using WWW:: Mechanize to upload my data one row at a time
  but would be nice if i can use the bulk loader as i don't want to
  reinvent the many options available in the tool.

  Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Bulkloader and Preserving Key_Names

[Devel63]

Is there some way to export key_names and then later import them using
the bulkloader?

I've seen the examples from folks messing around with Handle_Entity,
but that seems pretty kludgy (and dangerous, given that this is inside
the source, and changed last release).

And key_name aside, what about preserving Reference links in general.
How is that done?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Bulkloader and Preserving Key_Names

[Devel63]

What is best practice (or any way) to download and later upload
key_names when using BulkLoader?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Too many indexed properties for entity

[Devel63]

What you are doing SHOULD work.  I use SearchableModel with many lines
in a TextProperty, plus StringProperties, ListProperties, and others.
I have 20+ indexes explicitly defined for this core class, and it all
works quite well.

I don't see anything wrong in your class declaration below (other
than, perhaps, the setting indexed=false on the TextProperty ... I
haven't tried that).

Two possible ways to explore:

- Pardon for asking, but are you positive the error message is about
the Item class you defined?
- What indexes do you have defined in index.yaml for Item?


On May 12, 9:58 am, Ben bhym...@gmail.com wrote:
 Ok, thank you for the response.  I was just worried i was missing
 something obvious.  My method of keeping only the title property as
 searchable model, and the rest of the fields as normal DB models does
 work fine, just didn't seem the most efficient way to do it.  but it
 looks like that is the way to go.  thank you for the response!

 On May 12, 3:35 am, baytiger troels...@gmail.com wrote:

  TextProperty and BlobProperty are ALREADY not indexed. So it doesn't
  make any difference.

  The problem with the searchable model is that it makes a few extra
  rows in your database that contains among other things a list of all
  the words that appear in your TextProperty. This means that it runs
  out of indexes and explodes and gives you the too many indexes error.

  There is no way around this. Its, unfortunately, a pitfall of the
  system. I think we can assume that Google is working on it. But nobody
  knows when it will be ready. It is likely that their database
  structure was not built for such large indexes.

  On May 12, 1:57 am, Ben bhym...@gmail.com wrote:

   I thought maybe the 1.2.2 update with the addition of indexed=false
   would solve this problem for me, but it doesn't seem to have any
   effect.  I can move the title into a separate property is i mentioned
   above and that works ok, but i feel like i must be missing some other
   obvious solution here.  my property is set like so:

   -
   class Item(search.SearchableModel):
 title = db.StringProperty(required=True)
 category = db.IntegerProperty()
 product =  db.IntegerProperty()
 priority = db.IntegerProperty()
 description = db.TextProperty(indexed=False)
 contributors = db.StringProperty()
   -

   any ideas?

   On May 9, 12:20 am, 风笑雪 kea...@gmail.com wrote:

SDK 1.2.2 can use indexed=False parameter on property constructor now.

2009/5/9 Ben bhym...@gmail.com

 I didn't figure out what was causing the problem, but i solved it by
 moving my Title  string prop to a separate searchable class with a
 reference property, and leaving the description in a standard db
 model.  might not work for everyone but solved my problem. thanks for
 the help.

 On May 8, 10:11 am, Devel63 danstic...@gmail.com wrote:
  You should be able to have a searchable TextProperty and a
  StringProperty.  I do.  If the advice below doesn't help, try 
  posting
  your class definition here.

  On May 7, 10:27 am, ryan 
  ryanb+appeng...@google.comryanb%2bappeng...@google.com
 wrote:

  http://code.google.com/appengine/docs/python/datastore/queriesandinde.
 ..
   describes this error.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Too many indexed properties for entity

[Devel63]

You should be able to have a searchable TextProperty and a
StringProperty.  I do.  If the advice below doesn't help, try posting
your class definition here.

On May 7, 10:27 am, ryan ryanb+appeng...@google.com wrote:
 http://code.google.com/appengine/docs/python/datastore/queriesandinde...
 describes this error.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Using PyCrypto

[Devel63]

Thanks, Nick!

On May 6, 3:29 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 On Tue, May 5, 2009 at 11:25 PM, Devel63 danstic...@gmail.com wrote:

  Has anyone used PyCrypto successfully with App Engine (http://
  code.google.com/appengine/docs/python/tools/libraries.html#PyCrypto)?
  I've run into 2 obstacles -

  1) Since there's no distribution with App Engine SDK, does anyone know
  a safe source of the library for use on a Windows development machine?

 Install easy_install:http://peak.telecommunity.com/DevCenter/EasyInstall

 Then issue the command easy_install pycrypto



  2) Trying to use it onhttp://shell.appspot.com, I get errors

     from Crypto.Cipher import DES
     obj=DES.new('abcdefgh', DES.MODE_ECB)

      --all sorts of traceback errors mentioning pickling

 As I said in your other thread, shell.appspot.com uses pickle to
 persist data between requests. This is not an issue with usage inside
 your app.

 -Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: PyCrypto Access

[Devel63]

The problem is that I don't see a file or a directory named 'Crypto'
anywhere on my system (it's certainly not in google_appengine/lib with
antlr3 and django) .  Where is it supposed to be?

On May 5, 5:04 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 Hi Dan,

 Just the same as you would in any other app using PyCrypto: Consult
 the relevant documentation. PyCrypto puts all the relevant modules
 under the 'Crypto' module.

 -Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: PyCrypto Access

[Devel63]

What??  How does this reconcile with the 1.2.1 announcement:

App Engine includes a custom version of the Python Cryptography
Toolkit, also known as PyCrypto.
The version included with App Engine is based on pycrypto 2.0.1.
This is not the latest version, but
should be largely compatible with more recent versions.

http://code.google.com/appengine/docs/python/tools/libraries.html


On May 5, 7:54 am, Nick Johnson (Google) nick.john...@google.com
wrote:
 PyCrypto is a third-party library. If it is installed, it will be in
 your Python install's site-packages directory, not under the App
 Engine SDK.

 -Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: PyCrypto Access

[Devel63]

OK, I hear what you all are saying: I need to install the libraries on
my dev server, but not upload them to my workspace because they will
already be accessible there.

However, I don't know why people on this thread keep saying that this
is the way it works with all the other 3rd party libraries.  To the
contrary, antlr3, django v0.96, webob, and yaml ... everything else
mentioned on http://code.google.com/appengine/docs/python/tools/libraries.html
... gets installed with the SDK under the lib directory.

I guess I can give it a go and see what happens, but it seems weird to
me that Google would make a custom version, tell us it is not based on
the latest release of the public version, and then not distribute it
with the SDK as they have with every other incorporated library
(unless there's some security reason not to do so).  So I guess we
have to get it working locally, then upload and hope that it works
with their modified and reduced functionality module, which we have no
way of examining beforehand.


On May 5, 12:57 pm, Wooble geoffsp...@gmail.com wrote:
 The libraries are installed on the App Engine production servers.
 They are not part of the SDK, so they're not on your machine unless
 you installed them, the same as the other third party libraries usable
 with App Engine.

 On May 5, 1:38 pm, Devel63 danstic...@gmail.com wrote:

  What??  How does this reconcile with the 1.2.1 announcement:

      App Engine includes a custom version of the Python Cryptography
  Toolkit, also known as PyCrypto.
      The version included with App Engine is based on pycrypto 2.0.1.
  This is not the latest version, but
      should be largely compatible with more recent versions.

     http://code.google.com/appengine/docs/python/tools/libraries.html

  On May 5, 7:54 am, Nick Johnson (Google) nick.john...@google.com
  wrote:

   PyCrypto is a third-party library. If it is installed, it will be in
   your Python install's site-packages directory, not under the App
   Engine SDK.

   -Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: PyCrypto Access

[Devel63]

Doesn't seem to work on http://shell.appspot.com

I can import a module (e.g., from Crypto.Cipher import DES), but when
I try to use it I get all sorts of errors about not being able to
pickle it (e.g., obj=DES.new('abcdefgh', DES.MODE_ECB).  I picked
those 2 lines (and others) because they are straight out of the
pycrypto examples.


On May 5, 1:24 pm, Devel63 danstic...@gmail.com wrote:
 OK, I hear what you all are saying: I need to install the libraries on
 my dev server, but not upload them to my workspace because they will
 already be accessible there.

 However, I don't know why people on this thread keep saying that this
 is the way it works with all the other 3rd party libraries.  To the
 contrary, antlr3, django v0.96, webob, and yaml ... everything else
 mentioned onhttp://code.google.com/appengine/docs/python/tools/libraries.html
 ... gets installed with the SDK under the lib directory.

 I guess I can give it a go and see what happens, but it seems weird to
 me that Google would make a custom version, tell us it is not based on
 the latest release of the public version, and then not distribute it
 with the SDK as they have with every other incorporated library
 (unless there's some security reason not to do so).  So I guess we
 have to get it working locally, then upload and hope that it works
 with their modified and reduced functionality module, which we have no
 way of examining beforehand.

 On May 5, 12:57 pm, Wooble geoffsp...@gmail.com wrote:

  The libraries are installed on the App Engine production servers.
  They are not part of the SDK, so they're not on your machine unless
  you installed them, the same as the other third party libraries usable
  with App Engine.

  On May 5, 1:38 pm, Devel63 danstic...@gmail.com wrote:

   What??  How does this reconcile with the 1.2.1 announcement:

       App Engine includes a custom version of the Python Cryptography
   Toolkit, also known as PyCrypto.
       The version included with App Engine is based on pycrypto 2.0.1.
   This is not the latest version, but
       should be largely compatible with more recent versions.

      http://code.google.com/appengine/docs/python/tools/libraries.html

   On May 5, 7:54 am, Nick Johnson (Google) nick.john...@google.com
   wrote:

PyCrypto is a third-party library. If it is installed, it will be in
your Python install's site-packages directory, not under the App
Engine SDK.

-Nick Johnson
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Using PyCrypto

[Devel63]

Has anyone used PyCrypto successfully with App Engine (http://
code.google.com/appengine/docs/python/tools/libraries.html#PyCrypto)?
I've run into 2 obstacles -

1) Since there's no distribution with App Engine SDK, does anyone know
a safe source of the library for use on a Windows development machine?

2) Trying to use it on http://shell.appspot.com, I get errors

from Crypto.Cipher import DES
obj=DES.new('abcdefgh', DES.MODE_ECB)

 --all sorts of traceback errors mentioning pickling


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] PyCrypto Working?

[Devel63]

Has anyone gotten the new support for PyCrypto to work?  (http://
code.google.com/appengine/docs/python/tools/libraries.html#PyCrypto)

When I try it on http://shell.appspot.com, I get error messages about
pickling.

- from Crypto.Cipher import DES
- obj=DES.new('abcdefgh', DES.MODE_ECB)


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] PyCrypto Access

[Devel63]

How does one use the new PyCrypto support?  I don't see any module I
can import?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google Help: Index Rebuilding Forever

[Devel63]

Google, can I get some help with this?

On Apr 26, 10:06 am, Devel63 danstic...@gmail.com wrote:
 I have an app whose indexes have been building for days, with very
 little data.

 Can someone from Google please contact me so we can get this reset?
 Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Completely clearing the datastore

[Devel63]

Can you explain this further?  I don't see any reference to key_name
in the sample code.

More importantly, to me, what's the cost differential between using
string representation of keys and key_names?  I've been passing around
key_names to the browser because they're shorter, under the assumption
that the cost to get the corresponding key on the server side was
negligible.

On Apr 25, 9:02 am, Alkis Evlogimenos ('Αλκης Ευλογημένος)
evlogime...@gmail.com wrote:
 Doing it over the remote api means you are going to transfer all your data +
 transmission overhead over the wire. You are probably better off doing
 something like this on the server side through an admin protected handler.

 Also if you happen to know the keys of your data (you used key_name) your
 deletes are going to be a lot more efficient if you give db.delete a list of
 keys instead.



 On Sat, Apr 25, 2009 at 2:41 PM, Sri sri.pan...@gmail.com wrote:

  Hi,

     Is there a way to completely erase the production data store?

  Currently I am using a script like this via the remote api:

  def delete_all_objects(obj_class):
     num_del = 300
     while True:
         try:
             objs = obj_class.all().fetch(1000)
             num_objs = len(objs)
             if num_objs == 0:
                 return
             print Deleting %d/%d objects of class %s % (num_del,
  num_objs, str(obj_class))
             db.delete(objs[:num_del])
         except Timeout:
             print Timeout error - continuing ...

  But with 3 entities in the data store and another 3 million (yep
  thats right) coming, doing a clear this way is extremely slow.

  Any ideas?

  cheers
  Sri

 --

 Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Google Help: Index Rebuilding Forever

[Devel63]

I have an app whose indexes have been building for days, with very
little data.

Can someone from Google please contact me so we can get this reset?
Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Need Index Quota Reset

[Devel63]

I have an app that's giving me a Number Indexes Exceeds Quota message.

My understanding is that vacuumed indexes are not automatically
credited back.

Can someone at Google reset my accounting?  Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Embedded Django 0.96 versus Django 1.0

[Devel63]

We've been using Django 0.96 (since it's already available within App
Engine), mostly just to get i18n support.

What are the drawbacks of going all the way to the latest version?
- We have to upload all the files (and I understand there's a zip way
to do that)
- Startup times are even slower when instantiated on a new machine
(how much?)
- Other?

Any key advantages?
- Gaebar requires 1.0
- The registration module requires 1.0
- Other?

Any sense of when/if Google will make this migration themselves?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Allowing users to login in a non-intimidating, non-confusing way?

[Devel63]

The existing Google user login support works very well, but it's
intimidating to new users who are sometimes reluctant to give an
unknown site their treasured Google ID, especially given the stern
warnings on the Google-supplied page.  And some people don't have a
Google ID, and don't want to make one.

So, I need to provide a friendlier custom login page.  But App Engine
doesn't support https connections from a non-appspot domain, right?
So, it's currently impossible to provide a secure login experience,
correct?

Putting aside https, once I get the user's login info, I suppose I
could validate using the existing user library calls, and just store
the data in Google Accounts ... or simply put it in my own model.

Is there a simple cookbook for handling this whole login issue?  Is
there an accepted practice?  This seems quite fundamental, but I can't
find a good answer anywhere.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: how to return random results from datastore

[Devel63]

Returning a random element is actually much trickier than it sounds.

First, the approach below does not provide a linear distribution.  You
are much better off assigning a one-up serial number to each entity
upon creation, then doing a retrieval the entity whose serial number
= randint(1,max_count)  ... and cycling back to 0 if you get no hits.

I have seen other people espousing the random/random approach below,
but it is fundamentally flawed.  Using the approach below, imagine the
first element getting a random value of 70, and the second
happening to get a random value of 71.   The first element will
get far more random requests than the second.  This is obviously an
extreme example, but it exposes the underlying problem.  The more
elements you have, the less important this problem will seem, as your
initial random distribution covers more of the territory, but the
fundamental flaw will always be there, with some elements getting 2x
or even 100X the exposure of other elements.

Unfortunately, even the approach I first describe above does not
statistically work when you start putting other conditions into the
query.  The problem is that if there are any swaths of non-uniform
distribution in your DB, then entities on the boundaries of these
swaths will get more than their fair share of hits.

The best approach turns out to depend on your particular
circumstances, and unfortunately, there is no correct answer for
certain situations other than to read all satisfying entities into
memory, and then do a random selection from those ... which is usually
prohibitively expensive.  Or, you can give up on perfect
distribution and use an approximation.

I'd be delighted if someone could volunteer a better approach!

On Apr 1, 5:55 am, Barry Hunter barrybhun...@googlemail.com wrote:
 When saving a entity, include a random property. (eg use an Integer,
 and set it to a random value between 1 and 100)

 Then when you want a random record, use the equivlent to ths GQL

 SELECT * FROM RandomEntity WHERE RandomProperty  4456 ORDER BY
 RandomProperty ASC LIMIT 1

 where the 4456 is randomly choosen.

 Or to account for occasional duplications of the RandomProperty could
 include a small offset. (say 1-10)

 SELECT * FROM RandomEntity WHERE RandomProperty  4456 ORDER BY
 RandomProperty ASC LIMIT 5,1

 I believe that should be reasonably efficient in the datastore.

 On 01/04/2009, sagey sage...@gmail.com wrote:



   hello,

   I'd like to be able to return random results from the datastore. I'm
   not sure from the documentation how i would go about doing that. I'd
   appreciate some pointers.

   Thanks in advance

 --
 Barry

 -www.nearby.org.uk-www.geograph.org.uk-
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] __init__ and datastore get?

[Devel63]

A class's __init__() constructor gets called whether you are creating
a new db.Model object in memory for the first time, or retrieving an
existing entity from the datastore.

Is there some accepted way for knowing which case you are in from
within the __init__ call?

If retrieving from the datastore, I don't want to set any values.  But
how do I know?  Should I call object.is_saved?  Is there a better way
of handling this?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: How to overwrite the default save method

[Devel63]

You may want to look at get_value_for_datastore here:
http://code.google.com/appengine/articles/extending_models.html

This gets called in the course of a put, and can be used to accomplish
other actions as well.

On Mar 13, 5:06 am, Michael O'Brien mich...@mcobrien.org wrote:
 If you want to add something to __init__, you can do something like
 this:

 class A(BaseModel):
     def __init__(self, *args, **kwargs):
         # do something here, e.g. adding values to kwargs (property
 values)
         kwargs[canonical_name] = kwargs[name].lower().strip()
         super(A, self).__init__(*args, **kwargs)

 cheers
 Michael

 On Mar 13, 4:45 am, yucc yucc2...@gmail.com wrote:

  Hi all,
  I want to overwrite the default save method of the
  google.appengine.ext.db.Model so as to perform some special work each
  time the entity save(or update),I know in Django model this should be
  like this:(in the model definition)
  def save(self, force_insert=False, force_update=False):
          self.level = self.level+1   #I want to let the level property
  auto add
          super(User, self).save(force_insert, force_update) # Call the
  real save() method.
          #do_something_else()

  But, this doesn't work with google.appengine.ext.db.Model,it comes out
  that the Model's put method takes  exactly 1 argument,so I omit
  them,like this:
  super(User, self).save(),
  Is that right?
  I was wondering how to construct a Model,as there's a init method:
  def __init__(self, parent=None, key_name=None, _app=None,
  _from_entity=False, **kwds)
  It seems super(User, self) is not right here
  Anyone could help ?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: query vs filter()

[Devel63]

I don't know what you mean about running a loop of filter calls, but
something like Class.all().filter().filter()... is very efficient.  My
impression is that the GQL interpretation is slightly more expensive,
but that the difference is negligible.

Note that the call to the data store does not occur until you actually
issue the fetch or get ... and this is what's really expensive.

On Mar 5, 4:44 pm, pedepy paul.ro...@gmail.com wrote:
 is there any difference in performance to use a single query call
 (like in, WHERE prop = value AND prop = value, etc vs running a loop
 of filter() calls and then executing the query ?

 seems to me that if you can build your query all in one shot, you can
 avoid one or many loop(s) in your code ... but everyone seems to be
 all about using all().filter() these days ..

 help?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Query with 'IN' and Associated Index Explosion

[Devel63]

It is indeed a more complex query, and the IN values are keys.  At one
point, I had tried to make them key values in a named reference
collection, but while the data store never complained, it never
returned any results.

Can you confirm that it is impossible to make an entity query that
filters by normal prop vals and by a list of keys in a collection?
e.g. Entity.gql('WHERE prop1 = :val1 AND named_collection
IN :key_list', val1=val, key_list=keys)

Separately, spurred on by your comments, I cleared the datastore and
started again.  Now, the index explosion is not occurring.  I guess
that some interim query experiment requested such indexes, and
stopping the datastore and deleting the indexes out of the index.yaml
file was not sufficient to clear them, so they kept reappearing in
index.yaml.


On Mar 3, 11:29 pm, ryan ryanb+appeng...@google.com wrote:
 On Mar 1, 2:13 pm, Andy Freeman ana...@earthlink.net wrote:

   However, when I do this, the dev_server creates a bunch of index
   definitions, so that I have one with 1 property if there's 1 value in
   the IN set, 15 index properties if there are 15 values in the IN set,
   and so on.

  Are all of these index properties necessary in production?  Will a
  production query with an IN set with 4 members fail if the test set
  used to generate index.yaml has IN sets with 1, 3, and 5 members?

 i have to admit, i don't entirely understand the problem. indices are
 only needed with different combinations of property names, not filter
 values. IN queries fan out to multiple queries on the exact same
 combination of property names; only the filter values change. changing
 the number of filter values shouldn't change the indices needed by the
 subqueries. for example, SELECT * FROM Foo WHERE x in [0, 1] fans out
 to:

 SELECT * FROM Foo WHERE x = 0
 SELECT * FROM Foo WHERE x = 1

 while SELECT * FROM Foo WHERE x in [0, 1, 2, 3]  fans out to

 SELECT * FROM Foo WHERE x = 0
 SELECT * FROM Foo WHERE x = 1
 SELECT * FROM Foo WHERE x = 2
 SELECT * FROM Foo WHERE x = 3

 all of the subqueries use the same index. here, it's a built-in single
 property index, but it could be a composite index if there was an
 additional filter or sort order.

 given that, i suspect there's more going on here, or at least a
 substantially more complicated query. Devel63, any chance you could
 post your exact query and filter values, ie the values for the IN set?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Query with 'IN' and Associated Index Explosion

[Devel63]

This is very promising!  Are you saying that this reference collection
query should work (note the named collection)?

class Foo(db.Model):
  prop1 = db.IntegerProperty()

class Fee(db.Model):
  my_foo = db.ReferenceProperty(Foo, collection_name='my_collection')
  prop2 = db.IntegerProperty()

foos = Foo.all().filter('prop1 =', 32)
results = Fee.gql('WHERE prop2 = 18 AND collection_name IN :1', foos)

That would be awesome, and I'd just have to try a little harder to get
things to work.  Of course in my real example, I'm using db.Searchable
instead of Model, and using search, but I'd be happy just to get the
above to work.


On Mar 4, 12:46 pm, ryan ryanb+appeng...@google.com wrote:
 On Mar 4, 9:11 am, Devel63 danstic...@gmail.com wrote:

  It is indeed a more complex query, and the IN values are keys.  At one
  point, I had tried to make them key values in a named reference
  collection, but while the data store never complained, it never
  returned any results.

 understood. happily, for the purposes of this conversation, the type
 of the IN values actually doesn't matter. IN won't generate an index
 per list length regardless of the IN values' type.

  Can you confirm that it is impossible to make an entity query that
  filters by normal prop vals and by a list of keys in a collection?
  e.g. Entity.gql('WHERE prop1 = :val1 AND named_collection
  IN :key_list', val1=val, key_list=keys)

 no, that's definitely possible. fromhttp://shell.appspot.com/:

  class Foo(db.Expando):
   pass
  Foo.gql('WHERE x = 0 AND y IN (1, 2)').fetch(3)

 [__main__.Foo object at 0xf5ca9957dc3a9c30, __main__.Foo object at
 0xf5ca9957dc3a9fb0, __main__.Foo object at 0xf5ca9957dc3a9cb0]

 again, this is true regardless of the type of the filter values.

  Separately, spurred on by your comments, I cleared the datastore and
  started again.  Now, the index explosion is not occurring.  I guess
  that some interim query experiment requested such indexes, and
  stopping the datastore and deleting the indexes out of the index.yaml
  file was not sufficient to clear them, so they kept reappearing in
  index.yaml.

 hmm. the dev_appserver stores both the datastore and the query
 history, in separate files. i wonder if the query history file somehow
 didn't get deleted. (it defaults to /tmp/
 dev_appserver.datastore.history.) more in the docs:

 http://code.google.com/appengine/docs/python/tools/devserver.html#Com...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Filter against Reference Collection?

[Devel63]

Is it possible to filter against a named reference collection?  The
datastore doesn't complain, but I never get any results...

class Foo(search.SearchableModel):
  prop1 = TextProperty()

class Fee(db.Model):
  prop2 = db.IntegerProperty()
  prop3 = db.ReferenceProperty(Foo, collection_name='my_items')

my_fee = Fee.all().filter('prop2 =', 11).get()
foos = Foo.all().filter('my_items =', my_fee.key())

I have tried every variation I can think of: using keys as above,
using my_fee directly without the key call, putting a str() around the
key, etc.).  I think I saw it work once or twice, but now I'm not sure
if that was just an illusion, and I can't replicate it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: using keys in an IN statement

[Devel63]

I just implemented something like this, and while it ran fine on the
dev_server, I noticed that it defined in index.yaml a new index for
every observed number of items in the IN set.

Since I was approaching the 30 limit, and had other varying elements
of the query, this caused an explosion of defined indexes.  But
perhaps these indexes aren't really needed...

Have either of you tried running an IN query on the production server
WITHOUT this explosion of indexes?


On Mar 3, 2:36 pm, ryan ryanb+appeng...@google.com wrote:
 On Mar 3, 12:26 pm, pedepy paul.ro...@gmail.com wrote:



  isnt the proper syntax WHERE property IN (foo, foo, foo) ?

 if you're providing a list of constants, yes. if you're using a bound
 parameter, though, like in your original post, you should omit the
 parentheses.

 having said that, it looks like we may sometimes handle list bound
 parameters incorrectly, particularly with parentheses around them.
 i'll look into that. thanks for the report!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---