[google-appengine] Re: best away to know numbers of rows in a entities
If you used numeric keys then you can do pagination by key in order to get all items from the datastore and count them. Basically this involves querying the datastore several times, each time filtering on keys that are greater than the highest key from the results of the previous query. You won't want to do this often as it will likely take to long to do it in a single request so you'll probably only want to do it once and store the value in a counter for later use. If you are using key names and don't have a way of pulling unique data, say via a create time DateTimeProperty, then I'm afraid you're kind of out of luck. You could try different filters and pull the data locally and compare the keys to get unique entities but that sounds like a serious pain. Someone else might have a better idea in this case. Ian On Wed, Jun 24, 2009 at 6:52 AM, alf alberto@gmail.com wrote: hi, assuming you have more than 1000 rows and you have no counter store in a entities what is the best away to know how many rows has a entitie. I think is better time to time request numbers of rows per entity to has a incremental counter and always have in mind insert and delete many thaks. -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Model with key_name not have key().id
Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model with key_name not have key().id
Ritesh, Keys can either have a numeric id or a key_name. In order to access the key name for entities that you saved with a key_name you need to use the key_name() method. instead of obj.key().id(), try obj.key().name() Incedentally you can also use obj.key().id_or_name() to get either the id or the name depending on which you are using. On Wed, Jun 24, 2009 at 3:08 PM, Ritesh Nadhani rite...@gmail.com wrote: Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model with key_name not have key().id
Ritesh, Also, BTW, The reason that obj1 and obj2 can have different types of keys is because the key is not saved on a model level. It is saved at the entity level. Each entity can have id or key_names. Apparently obj2 was saved without a key name and was given a numeric key. obj1 was saved with a key_name. So to answer your question, yes, within a model/Kind you can have some entities that have a numeric id and some that have key names. Ian 2009/6/24 Ian Lewis ianmle...@gmail.com Ritesh, Keys can either have a numeric id or a key_name. In order to access the key name for entities that you saved with a key_name you need to use the key_name() method. instead of obj.key().id(), try obj.key().name() Incedentally you can also use obj.key().id_or_name() to get either the id or the name depending on which you are using. On Wed, Jun 24, 2009 at 3:08 PM, Ritesh Nadhani rite...@gmail.com wrote: Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Using the Humanize part of Django templates
I created a settings.py file and added the INSTALLED APPS but nothing helps. (the error, btw, is TemplateSyntaxError: 'humanize' is not a valid tag library: Could not load template library from django.templatetags.humanize, No module named humanize). Remember, I'm not using Django; just the forms. So how can I accomplish this? All I really want to do is automatically put commas into numbers (e.g. 1000 = 1,000), by using templates... On Jun 24, 2:08 am, Ian Lewis ianmle...@gmail.com wrote: Thats the thing about Django. It tends to not work well unless you use the whole framework. INSTALLED_APPS is a setting in the settings.py in django. You'll probably need to create a settings.py and put 'django.contrib.humanize' in the INSTALLED_APPS list. see:http://docs.djangoproject.com/en/dev/ref/settings/#installed-apps On Wed, Jun 24, 2009 at 7:19 AM, MajorProgamming sefira...@gmail.comwrote: I am currently running an app that is ONLY using Django templates (not the whole framework). I was wondering how I can use the django.contrib.humanize part so I can use specialized filters? It mentions something about INSTALLED_APPS, but I have no clue how to do that? Thanks, -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Datastore delete taking long time
Has the delete function of records in the datastore started to take much longer time? One Ajax call I have that deletes up to 20 records often (more often than not perhaps) takes a very long time, where a single request consumes around 8000cpu_ms and around 8000api_cpu_ms. The records deleted are simple with only 10 properties of which only one is a ReferenceProperty and the rest are single-line no-reference properties. And no complicated indexes are used (as far as I know). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model with key_name not have key().id
Actually, what I meant was: I cannot have an entity with both key_name and id. They seem to be mutually exclusive for an entity. 2009/6/23 Ian Lewis ianmle...@gmail.com: Ritesh, Also, BTW, The reason that obj1 and obj2 can have different types of keys is because the key is not saved on a model level. It is saved at the entity level. Each entity can have id or key_names. Apparently obj2 was saved without a key name and was given a numeric key. obj1 was saved with a key_name. So to answer your question, yes, within a model/Kind you can have some entities that have a numeric id and some that have key names. Ian 2009/6/24 Ian Lewis ianmle...@gmail.com Ritesh, Keys can either have a numeric id or a key_name. In order to access the key name for entities that you saved with a key_name you need to use the key_name() method. instead of obj.key().id(), try obj.key().name() Incedentally you can also use obj.key().id_or_name() to get either the id or the name depending on which you are using. On Wed, Jun 24, 2009 at 3:08 PM, Ritesh Nadhani rite...@gmail.com wrote: Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- Ritesh http://www.riteshn.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model with key_name not have key().id
Yes, that is correct. 2009/6/24 Ritesh Nadhani rite...@gmail.com Actually, what I meant was: I cannot have an entity with both key_name and id. They seem to be mutually exclusive for an entity. 2009/6/23 Ian Lewis ianmle...@gmail.com: Ritesh, Also, BTW, The reason that obj1 and obj2 can have different types of keys is because the key is not saved on a model level. It is saved at the entity level. Each entity can have id or key_names. Apparently obj2 was saved without a key name and was given a numeric key. obj1 was saved with a key_name. So to answer your question, yes, within a model/Kind you can have some entities that have a numeric id and some that have key names. Ian 2009/6/24 Ian Lewis ianmle...@gmail.com Ritesh, Keys can either have a numeric id or a key_name. In order to access the key name for entities that you saved with a key_name you need to use the key_name() method. instead of obj.key().id(), try obj.key().name() Incedentally you can also use obj.key().id_or_name() to get either the id or the name depending on which you are using. On Wed, Jun 24, 2009 at 3:08 PM, Ritesh Nadhani rite...@gmail.com wrote: Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How do I limit searchable_text_index using SearchableModel?
Thanks for all the responses.
I'll use Ian's first suggestion on Product, ProductIndex until
key_only query issue is resolved.
How do I change my bulkloader to update the Product and ProductIndex
on one upload?
class ProductLoader(bulkloader.Loader):
def __init__(self):
bulkloader.Loader.__init__(self, 'Product',
[('pid', str),
('title', lambda x: unicode(x,
'utf-8')),
('site', str),
('url', str)
])
Thanks
Jon
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Odd memcache behavior across multiple app instances
Hi Kim,
Can you please provide your app ID?
Also, how are you accessing your app? Only via appspot.com, or also via a
Google Apps domain you've mapped to the app? The more details you can
provide about your load testing - especially the actual code - the easier
this will be to diagnose.
-Nick Johnson
On Tue, Jun 23, 2009 at 2:42 PM, Kim Riber kimsteenri...@gmail.com wrote:
Yes. There is no other activity when running step 3.
The two values I get back, are some of the last ones I write to the
cache (I return all the values I write).
So it looks a lot like the write threads are writing to two instances
of memcache, and when I start reading back, I see the 2 different
values from the 2 insances.
When waiting for an hour or so, when I run step 3 it returns only one
value again.
If I run the heavy load task (and spawn more instances), running step
3 will start reporting the 2 values again. Like the cluster with the
second memcache instance is getting fired up, and reporting it's old
value.
the code for step 3 is something like this:
def read_memcache()
val = memcache.get('my_test_key')
return val
I could try to set up an application with the 3 steps.
Cheers
Kim
On Jun 23, 2:52 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Kim,
Are you able to send us the code you use for step 3? And are you certain
nothing is changing the memcache concurrently with step 3?
On Tue, Jun 23, 2009 at 1:44 PM, Kim Riber kimsteenri...@gmail.com
wrote:
Hi Nick
I run the test in 3 steps (with half a minute in between):
1. Heavy load process to spawn appinstances
2. Write a lot of random values to the same key
3. Read the key from multiple threads.
I can repeat 3rd step, and still get the same result (mostly 2
different values)
It seems like I hit 2 different memcache servers with different view
on what is in the cache at that key
Also, are you running this against the dev_appserver, or in
production?
How do I see that?
We are running towards .appspot.com
If you are testing against the local development server, that's the
dev_appserver. If you're testing code running on appspot.com, that's
production.
-Nick Johnson
http://appspot.com
-Kim
On Jun 23, 12:39 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Kim,
It's not clear from your description exactly how you're performing
your
tests. Without extra information, the most likely explanation would
be
that
you're seeing a race condition in your code, where the key is
modified
between subsequent requests to the memcache API.
Also, are you running this against the dev_appserver, or in
production?
-Nick Johnson
On Tue, Jun 23, 2009 at 7:18 AM, Kim Riber kimsteenri...@gmail.com
wrote:
Just made another test, to confirm the behavior I see.
This example is much simpler, and simply has 10 threads writing
random
values to memcahce to the same key.
I would expect the last value written to be the one left in
memcache.
When afterwards, having 4 threads reading 10 times from that same
key,
they return 2 different values.
This only happens if I prior to the writing threads, run some heavy
tasks, to force gae to spawn more app instances.
It seems like each server cluster might have its own memcache,
independant from each other. I hope this is not true. From a thread
from Ryan
http://groups.google.com/group/google-appengine/browse_thread/thread/.
..
he states that
as for the datastore, and all other current stored data APIs like
memcache, there is a single, global view of data. we go to great
lengths to ensure that these APIs are strongly consistent.
Regards
Kim
On Jun 17, 8:51 pm, Kim Riber kimsteenri...@gmail.com wrote:
To clarify a bit:
one thread from our server runs one loop with a unique id.
each requests stores a value in memcache and returns that value.
In
the following request, the memcache is queried if the value just
written, is in the cache.
This sometimes fail.
My fear is that it is due to the requests changing to another app
instance and then suddently getting wrong data.
instance 1 + +
instance 2 --
Hope this clears out the example above a bit
Cheers
Kim
On Jun 17, 7:52 pm, Kim Riber kimsteenri...@gmail.com wrote:
Hi,
I'm experiencing some rather strange behavior from memcache. I
think
I'm getting different data back from memcache using the same
key
The issue I see is that when putting load on our application,
even
simple memcache queries are starting to return inconsistant
data.
When
running the same request from multiple threads, I get different
results.
I've made a very simple example, that runs
[google-appengine] Re: How do I limit searchable_text_index using SearchableModel?
Hi ogterran,
You'll need to define a custom handle_entity function for your loader class.
This function can return multiple ProductIndex entities in addition to the
Product entity being loaded. See the docstring for details:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/tools/bulkloader.py#3243
-Nick Johnson
On Wed, Jun 24, 2009 at 10:09 AM, ogterran jonathanh...@gmail.com wrote:
Thanks for all the responses.
I'll use Ian's first suggestion on Product, ProductIndex until
key_only query issue is resolved.
How do I change my bulkloader to update the Product and ProductIndex
on one upload?
class ProductLoader(bulkloader.Loader):
def __init__(self):
bulkloader.Loader.__init__(self, 'Product',
[('pid', str),
('title', lambda x: unicode(x,
'utf-8')),
('site', str),
('url', str)
])
Thanks
Jon
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
Hi Anders, How are you deleting the entities? Are you fetching them first, or deleting them by providing the keys? And are you deleting them in a single batch, or one at a time? -Nick Johnson On Wed, Jun 24, 2009 at 7:44 AM, Anders i...@blabline.com wrote: Has the delete function of records in the datastore started to take much longer time? One Ajax call I have that deletes up to 20 records often (more often than not perhaps) takes a very long time, where a single request consumes around 8000cpu_ms and around 8000api_cpu_ms. The records deleted are simple with only 10 properties of which only one is a ReferenceProperty and the rest are single-line no-reference properties. And no complicated indexes are used (as far as I know). -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: best away to know numbers of rows in a entities
2009/6/24 Ian Lewis ianmle...@gmail.com If you used numeric keys then you can do pagination by key in order to get all items from the datastore and count them. Basically this involves querying the datastore several times, each time filtering on keys that are greater than the highest key from the results of the previous query. You won't want to do this often as it will likely take to long to do it in a single request so you'll probably only want to do it once and store the value in a counter for later use. If you are using key names and don't have a way of pulling unique data, say via a create time DateTimeProperty, then I'm afraid you're kind of out of luck. You could try different filters and pull the data locally and compare the keys to get unique entities but that sounds like a serious pain. Someone else might have a better idea in this case. Pagination by key works regardless of if you're using key names or IDs. -Nick Johnson Ian On Wed, Jun 24, 2009 at 6:52 AM, alf alberto@gmail.com wrote: hi, assuming you have more than 1000 rows and you have no counter store in a entities what is the best away to know how many rows has a entitie. I think is better time to time request numbers of rows per entity to has a incremental counter and always have in mind insert and delete many thaks. -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Shifting applications between google and google apps.
Hi Matt, There are two separate issues here: 1. Moving the 'ownership' of the app to your company account 2. Changing authentication settings. For part 1, this is simple: Add the company account as an administrator of the app, then log in using that account, and remove your personal account as an administrator. For part 2, this depends on your requirements. If you're happy with authentication being open (eg, not restricted to a domain), or you are happy to do checks in code, and that's how you set up the app originally, no action is required. If you need the app restricted to only using your company's domain for authentication, it's slightly more problematic. There's no user-end way to change the authentication settings, and you can't delete and re-create an app. Creating a new app may be your best option - bear in mind that you can map any domain to any app, regardless of the app ID. If you're in the latter situation and really need to keep the app ID, contact me and we may be able to sort something out. -Nick Johnson On Tue, Jun 23, 2009 at 12:06 AM, Matt mjwat...@gmail.com wrote: Hi there, I got a small problem that I need to sort out with Google App Engine. When I first signed up to develop for GAE I used my regular Google account and dutifully registered the name of a product my company is developing. I've realized that I should have signed up using my companies Google Apps account instead. So at the moment I've got a registered application under my regular Google account that I need to move across to my companies Google Apps account. What should I do? Can I delete the registered application from within my account and then add it to my apps domain? Will that work? I don't want to lose the registered application name as it's very important to us. I've also read that there is some work in progress sort out issues like this. Is that something I should wait for instead? I'd really appreciate an answer for this as it's potentially holding back our application deployment. Thanks for your time. Matt -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Does static files count for outbound traffic?
Hi Mariano, Yes, static files are included in your daily traffic quota. Bear in mind that although App Engine does not provide last-modified or etag headers, it does set Cache-Control and Expires headers, and you can control the expiration time in app.yaml, so browsers should not re-request static content until it expires. -Nick Johnson On Tue, Jun 23, 2009 at 11:15 PM, Mariano Benitez mari...@benitez.nuwrote: I do have some large static files, and given that appengine do not use eTags or another form of check it is always returning the entire files. If the static files count as part of the outgoing traffic quota (which I think it is reasonable), then either you implement eTag or another method of not returning the entire file if it was not modified. My other alternative is to implement my handlers, move all to dynamic, and handle the 304 myself. I don't like the idea but I have no choice. Regards, MAriano -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
First I get the entities using a query, something like:
entities = entity.Entity.gql('WHERE account_key = :1 AND topic = :2
ORDER BY message_count DESC LIMIT 20', account_key, topic)
Then a loop calling delete() on each entity:
for entity in entities:
entity.delete()
On Jun 24, 12:29 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Anders,
How are you deleting the entities? Are you fetching them first, or deleting
them by providing the keys? And are you deleting them in a single batch, or
one at a time?
-Nick Johnson
On Wed, Jun 24, 2009 at 7:44 AM, Anders i...@blabline.com wrote:
Has the delete function of records in the datastore started to take
much longer time? One Ajax call I have that deletes up to 20 records
often (more often than not perhaps) takes a very long time, where a
single request consumes around 8000cpu_ms and around 8000api_cpu_ms.
The records deleted are simple with only 10 properties of which only
one is a ReferenceProperty and the rest are single-line no-reference
properties. And no complicated indexes are used (as far as I know).
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: best away to know numbers of rows in a entities
2009/6/24 Nick Johnson (Google) nick.john...@google.com Pagination by key works regardless of if you're using key names or IDs. Good to know. Sorry about the misinformation. I had forgotten that entities are ordered in the datastore by their key. BTW, here is an article in the documentation about pagination: http://code.google.com/appengine/articles/paging.html Read the part called Paging without a property for information on how to paginate by key. Here is information about queries on keys and contains the info about how keys are sorted by id/keyname: http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Queries_on_Keys -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Is class Body(db.Model): ok?
Hi Jesse, It sometimes takes a while for new entity types to be reflected in the Admin Console. Are the entities showing up now? -Nick Johnson On Tue, Jun 23, 2009 at 8:51 PM, Jesse Grosjean je...@hogbaysoftware.comwrote: Have you added any Body entities to your datastore? Sometimes I get HTTP 500 errors in the console viewer for models with no existing entities. Yes I've added a lot, and they seem to be working, I just don't see them in the console. I also have an Includes ancestors index that's serving Body entities. Jesse -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
Hi Anders,
You don't actually need the entities in order to delete them - only their
keys. Also, you can do all the deletes in a single batch operation. This
ought to be substantially faster:
db.delete(db.GqlQuery(SELECT __key__ FROM Entity WHERE account_key = :1 AND
topic = :2 ORDER BY message_count DESC, account_key, topic).fetch(20))
or equivalently with the Query api:
db.delete(entity.Entity.all(keys_only=True).filter('account_key',
account_key).filter('topic', topic).order('-message_count').fetch(20))
-Nick Johnson
On Wed, Jun 24, 2009 at 11:45 AM, Anders i...@blabline.com wrote:
First I get the entities using a query, something like:
entities = entity.Entity.gql('WHERE account_key = :1 AND topic = :2
ORDER BY message_count DESC LIMIT 20', account_key, topic)
Then a loop calling delete() on each entity:
for entity in entities:
entity.delete()
On Jun 24, 12:29 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Anders,
How are you deleting the entities? Are you fetching them first, or
deleting
them by providing the keys? And are you deleting them in a single batch,
or
one at a time?
-Nick Johnson
On Wed, Jun 24, 2009 at 7:44 AM, Anders i...@blabline.com wrote:
Has the delete function of records in the datastore started to take
much longer time? One Ajax call I have that deletes up to 20 records
often (more often than not perhaps) takes a very long time, where a
single request consumes around 8000cpu_ms and around 8000api_cpu_ms.
The records deleted are simple with only 10 properties of which only
one is a ReferenceProperty and the rest are single-line no-reference
properties. And no complicated indexes are used (as far as I know).
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Retrieving size of collection set
Hi djidjadji,
This will be less efficient than simply calling .count() on the collection
property, which doesn't require fetching the keys at all.
-Nick Johnson
On Tue, Jun 23, 2009 at 8:04 PM, djidjadji djidja...@gmail.com wrote:
You can construct a __key__ only query and count them up.
Then you don't have to construct all the A objects just for counting.
result = A.all(keys_only=True).filter('refprop =', b.key()).fetch(1000)
numA = len(result)
2009/6/22 Nick Johnson (Google) nick.john...@google.com:
Hi johntray,
On Sun, Jun 21, 2009 at 5:47 PM, johntray john.tur...@gmail.com wrote:
Well yes I could add a count property to the B objects, but I'm really
trying to understand more about how ReferenceProperty works.
ReferenceProperty's collection attributes are just syntactic sugar for
creating and executing a query yourself. As such, all the same
limitations
apply. In the case where you call len() on it, I believe this will result
in
a count query being executed, which doesn't require Python to decode all
the
entities, but does require the datastore to fetch all the index rows.
[snip]
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Application names disappear from Admin console, No new can be created but nevertheless availability counter is decreasing
Hi stelg, All App Engine accounts are Google accounts, and vice-versa. There's no such thing as an account just for App Engine, or just for GMail. What I'm asking is if your 'account X' is an account at a domain that is not gmail.com or googlemail.com. This is more easily solved if you can tell me the accounts and app IDs in question. Please email me privately if you are not comfortable revealing them here. -Nick Johnson On Tue, Jun 23, 2009 at 7:00 PM, stelg stelg.best...@gmail.com wrote: Hi Nick, Account X is an account I created on foloowing the instructions on http://code.google.com/appengine/ My website is just at the stelg.appspot.com (no other domain name) I log in at the page Welcome to Google App Engine http://appengine.google.com/ This does noet work: http://appengine.google.com/a/stelg.appspot.com where stelg.appspot.com equals to yourdomain I can see and work normally with the Admin console on for my first created application: stelg. All the rest is gone I have been able to view them in the past... Regards Stelg On 17 jun, 18:33, Nick Johnson (Google) nick.john...@google.com wrote: Hi stelg, Is account 'X' an Apps for Your Domain account? If so, you need to log in athttp://appengine.google.com/a/yourdomain to see the apps. -Nick Johnson On Wed, Jun 17, 2009 at 5:08 PM, stelg stelg.best...@gmail.com wrote: Hi Folks, I experience a strange situation, which needs some explanation as the sequence op steps might have caused this problem. Step 1: I created more than one year ago my first application in Google App Engine using a Google App Acount (here) called X. Step 2: I created a second, a third and a fourth AppEngine name using account X, so total of 4 applications. All were displayed in the admin console. Step 3 : I created a new Google Account (called Y) -- This is a Gmail account, not an App Engine account! Step 4: I allowed account Y to become administrator of my first Google App Engine application, So far so good: 2 admins! Step 5: Today I tried to create with account X (is admin) a new Google App Engine application. The system accepts the new application name but when i finished this process, no new application name appeared in the admin console (the screen you see when you login). The number of available Google App Engine applications went anyhow down from 10 to 9. Another big surprise i got: all my other applications made with account X (so the second, third and fourth mentioned in step 2) were gone. As far as i know you cannot delete applications. I have never found or seen such a function. My older application names are not listed anymore. Only 1 application is shown and that was my first from step1 !! The application counter shows that I still have 9 applications available... Weird...strange..strange..strange Lucky me these apps were not the important ones... Step 6: I retried to create again a new Google App Engine application with again a new name, but no new things appear in the admin console, nevertheless the counter went down again: The number of available Google App Engine applications went down from 9 to 8. Step 7: after 30-60 minutes waiting NO new Google App Engine application name has appeared in the console. So what is happening here? Logging on with account Y does not show them (which would be even more strange when that would have happened). I am stuck here: 3 apps gone en no new one can be created. Hheelp!!! -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SMS verification trouble.
Hi Patipat, Please try signing out, then back in with your ken account and creating an app. -Nick Johnson On Tue, Jun 23, 2009 at 6:27 PM, Patipat Susumpow keng...@gmail.com wrote: keng...@gmail the same one I've posted here. I'm curious. When I click Create an Application from this page http://appengine.google.com/start, it's still redirect me to http://appengine.google.com/permissions/smssend Cheers, Patipat. On Jun 23, 11:07 pm, Nick Johnson (Google) nick.john...@google.com wrote: What email address are you using? I activated the one you're using to post here. -Nick Johnson On Tue, Jun 23, 2009 at 4:48 PM, Patipat Susumpow keng...@gmail.com wrote: I still redirected tohttp://appengine.google.com/permissions/smssend whenever i try to create new application. How can this be fixed? Cheers, Patipat. On Jun 22, 5:06 pm, Nick Johnson (Google) nick.john...@google.com wrote: Hi Patipat, I've manually activated your account. -Nick Johnson On Sat, Jun 20, 2009 at 10:37 AM, Patipat Susumpow keng...@gmail.com wrote: Hi, I can't verify my account by SMS from http://appengine.google.com/permissions/smssend.do, tried many times with friends' mobile phone no, various supported operators in Thailand, but always get The phone number has been sent too many messages or has already been used to confirm an account. message. I'm wondering that I never use this verification method before, but get this error. Thanks, Patipat. -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Does static files count for outbound traffic?
On 24/06/2009, Nick Johnson (Google) nick.john...@google.com wrote: Hi Mariano, Yes, static files are included in your daily traffic quota. Bear in mind that although App Engine does not provide last-modified or etag headers, it does set Cache-Control and Expires headers, and you can control the expiration time in app.yaml, so browsers should not re-request static content until it expires. Should not and will not are very different things in my experience :) Esp. when faced with users pressing F5. Pressing F5 will often result if a Conditional GET (if there is a etag, or last-modified in the original) I guess that this should be directed into the feature request to allow enabling of last-modified on the static server. http://code.google.com/p/googleappengine/issues/detail?id=575 -Nick Johnson On Tue, Jun 23, 2009 at 11:15 PM, Mariano Benitez mari...@benitez.nu wrote: I do have some large static files, and given that appengine do not use eTags or another form of check it is always returning the entire files. If the static files count as part of the outgoing traffic quota (which I think it is reasonable), then either you implement eTag or another method of not returning the entire file if it was not modified. My other alternative is to implement my handlers, move all to dynamic, and handle the 304 myself. I don't like the idea but I have no choice. Regards, MAriano -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Barry - www.nearby.org.uk - www.geograph.org.uk - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
I have now changed it to the second variant, with the Query api, and
the result for 3 test calls was:
6775cpu_ms 6687api_cpu_ms
6757cpu_ms 6687api_cpu_ms
6765cpu_ms 6687api_cpu_ms
That's perhaps a slight improvement, but I have also done
vacuum_indexes which could perhaps have improved the performance a
bit. In either case, it's still a lot of CPU time for a single Ajax
call. If these calls are made frequently it will consume vast amounts
of CPU time.
On Jun 24, 12:53 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Anders,
You don't actually need the entities in order to delete them - only their
keys. Also, you can do all the deletes in a single batch operation. This
ought to be substantially faster:
db.delete(db.GqlQuery(SELECT __key__ FROM Entity WHERE account_key = :1 AND
topic = :2 ORDER BY message_count DESC, account_key, topic).fetch(20))
or equivalently with the Query api:
db.delete(entity.Entity.all(keys_only=True).filter('account_key',
account_key).filter('topic', topic).order('-message_count').fetch(20))
-Nick Johnson
On Wed, Jun 24, 2009 at 11:45 AM, Anders i...@blabline.com wrote:
First I get the entities using a query, something like:
entities = entity.Entity.gql('WHERE account_key = :1 AND topic = :2
ORDER BY message_count DESC LIMIT 20', account_key, topic)
Then a loop calling delete() on each entity:
for entity in entities:
entity.delete()
On Jun 24, 12:29 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Anders,
How are you deleting the entities? Are you fetching them first, or
deleting
them by providing the keys? And are you deleting them in a single batch,
or
one at a time?
-Nick Johnson
On Wed, Jun 24, 2009 at 7:44 AM, Anders i...@blabline.com wrote:
Has the delete function of records in the datastore started to take
much longer time? One Ajax call I have that deletes up to 20 records
often (more often than not perhaps) takes a very long time, where a
single request consumes around 8000cpu_ms and around 8000api_cpu_ms.
The records deleted are simple with only 10 properties of which only
one is a ReferenceProperty and the rest are single-line no-reference
properties. And no complicated indexes are used (as far as I know).
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number:
368047
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: dev_appserver.py throws SystemError: frexp() result out of range and subsequenty ValueError: bad marshal data
Hi rraj, It's possible you're still pointing to a corrupted datastore. Please try backing up your datastore (if necessary), then running with --clear_datastore and see if that works. -Nick Johnson On Tue, Jun 23, 2009 at 5:43 PM, rraj r.rajku...@gmail.com wrote: No, but I gave a different datastore location (using datastore_path and history_path, as shown in the trace snippet) which I thought was equivalent enough - is it not ?! Regards, R.Rajkumar On Tue, Jun 23, 2009 at 4:17 PM, Nick Johnson (Google) nick.john...@google.com wrote: Hi rraj, Have you tried clearing your datastore? -Nick Johnson On Tue, Jun 23, 2009 at 3:07 AM, rraj r.rajku...@gmail.com wrote: Hi, Had anybody encountered the following error and can guide me on how you fixed it ? When running the development app server, initial run throws SystemError: frexp() result out of range... C:\Program Files\Google\google_appenginedev_appserver.py --datastore_path=C:\gae_data --history_path=C:\gae_data demos\guestbook Traceback (most recent call last): File C:\Program Files\Google\google_appengine\dev_appserver.py, line 60, in module run_file(__file__, globals()) File C:\Program Files\Google\google_appengine\dev_appserver.py, line 57, in run_file execfile(script_path, globals_) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 483, in module sys.exit(main(sys.argv)) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 400, in main SetGlobals() File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 86, in SetGlobals from google.appengine.tools import dev_appserver File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 86, in module from google.appengine.api import datastore_file_stub File C:\Program Files\Google\google_appengine\google\appengine\api\datastore_file_stub.py, line 38, in module import datetime SystemError: frexp() result out of range Subsequent attempts to run applications, throws ValueError: bad marshal data... C:\Program Files\Google\google_appenginedev_appserver.py --datastore_path=C:\gae_data --history_path=C:\gae_data demos\guestbook Traceback (most recent call last): File C:\Program Files\Google\google_appengine\dev_appserver.py, line 60, in module run_file(__file__, globals()) File C:\Program Files\Google\google_appengine\dev_appserver.py, line 57, in run_file execfile(script_path, globals_) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 483, in module sys.exit(main(sys.argv)) File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 400, in main SetGlobals() File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver_main.py, line 86, in SetGlobals from google.appengine.tools import dev_appserver File C:\Program Files\Google\google_appengine\google\appengine\tools\dev_appserver.py, line 86, in module from google.appengine.api import datastore_file_stub ValueError: bad marshal data Python Version :: Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit Intel)] on win32 GAE Version : 1.2.3 (GoogleAppEngine_1.2.3.msi) Was running the application with an earlier version of GAE (1.2.2/1.2.1), when I encountered the bad marshal data problem during a restart of my application. Tried moving to the latest version to see if this has been handled. Removing the datastore_file_stub.pyc and running again reproduces the problem in the same sequence : frexp() result out of range followed by bad marshal data. Tried moving to Python 2.6.2 - did not help. Tried repairing GAE 1.2.3 - did not help. Uninstalled Python 2.6.2, Python 2.5.2 GAE and then installed Python 2.5.2 and GAE 1.2.3 again and tested with demo application and new data-store path, when I got the above traces. Not able to run any GAE apps now :-( Any tips to get me going again will be appreciated. Thanks Regards, R.Rajkumar -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Efficient way to structure my data model
Hi ecognium,
On Tue, Jun 23, 2009 at 6:11 PM, ecognium ecogn...@gmail.com wrote:
Thanks again - this is very helpful. I will let you know if i run into
any future index creation errors as it could have been caused by any
number of other entries - i mistakenly thought it was all these categ
list-based entries.
So if i understand it right even with a 10 element list for keywords,
there will only be 10 rows when 4 categ fields are used.
Correct - 4C4 * 10C1 index entries for the custom index you specified
earlier.
In the event
I use 'categ' only once in my query along with keywords field, it
will have up to 40 rows (10 from keywords and 4C1 from categ list).
Correct.
Am
I adding these up right?
I do not see myself going beyond 6 elements in the categ list at this
point (I guess the max will be 6C3 = 20 under such a situation).
In the case of your index with 4 occurrences, this will rather be 6C4.
The
keyword list will be probably go into the 20s but do not see anything
beyond that and will always be used only once in the query.
Thanks,
-e
On Jun 23, 3:53 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi ecognium,
On Tue, Jun 23, 2009 at 1:35 AM, ecognium ecogn...@gmail.com wrote:
Thanks, Nick. Let me make sure I understand your comment correctly.
Suppose I have the following data:
ID BlobProp1 BlobProp2-N Keywords
Categ
=
123 blahblahtag1,tag2,tag3
Circle,
Red, Large, Dotted
345 blahblahtag3,tag4,tag5
Square, Blue, Small, Solid
678 blahblahtag1,tag3,tag4
Circle, Blue, Small, Solid
-
The field categ (list) contains four different types - Shape, Color,
Size and Line Type. Suppose the user wants to retrieve all entities
that are Small Dotted Blue Circles then the query will be:
Select * From MyModel where categ = Circle AND categ = Small AND
categ = Blue AND categ = Dotted
When I was reading about exploding indexes the example indicated the
issue was due to Cartesian product of two list elements. I thought the
same will hold true with one list field when used multiple times in a
query.
That is indeed true, though it's not quite the cartesian product - the
datastore won't bother indexing (Circle, Circle, Circle, Circle), or
(Dotted, Dotted, Dotted, Dotted) - it only indexes every unique
combination,
which is a substantially smaller number than the cartesian product. It's
still only tractable for small lists, though, such as the 4 item lists
you're dealing with.
Are you saying the above query will not need {Circle, Red,
Large, Dotted} * {Circle, , , } * {Circle, , , } * {Circle, , , }
number of index entities for entity ID=123?
Correct - if you're not specifying a sort order, you can execute the
query
without any composite indexes whatsoever. The datastore satisfies
equality-only queries using a merge join strategy.
I was getting index errors
when I was using the categ list property four times in my index
specification and that's why I was wondering if I should restructure
things.
How many items did you have in the list you were indexing in that case?
If
your list has 4 items and your index specification lists it 4 times, you
should only get one index entry.
so I am guessing the following spec should not cause any index
issues in the future?
Again, that depends on the number of entries in the 'categ' list. With 4
entries, this will only generate a single index entry, but the number of
entries will expand exponentially as the list increases in size.
-Nick Johnson
- kind: MyModel
properties:
- name: categ
- name: categ
- name: categ
- name: categ
- name: keywords
- name: __key__ # used for paging
Thanks,
-e
On Jun 22, 2:10 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi ecognium,
If I understand your problem correctly, every entity will have 0-4
entries
in the 'categ' list, corresponding to the values for each of 4
categories
(eg, Color, Size, Shape, etc)?
The sample query you give, with only equality filters, will be
satisfiable
using the merge join query planner, which doesn't require custom
indexes,
so
you won't have high indexing overhead. There will simply be one index
entry
for each item in each list.
If you do need custom indexes, the number of index entries, isn't
4^4, as
you suggest, but rather smaller. Assuming you want to be able to
query
with
any number of categories from 0 to 4, you'll need 3 or 4 custom
indexes
(depending on if the 0-category
[google-appengine] Re: [ANN]: full-text search for app-engine-patch/Django
Hi Waldemar, Impressive! I'm curious - what's the dependency on Django / App Engine patch? This seems like it should be doable as a purely datastore-oriented library. -Nick Johnson On Tue, Jun 23, 2009 at 5:12 PM, Waldemar Kornewald app-eng...@scholardocs.com wrote: Hi, we'd like to announce the immediate availablility of our full-text search package. It's called gae-search: http://gae-full-text-search.appspot.com/ See it in action by searching our documentation (which is indexed with gae-search). We also have a few demos. Note that gae-search requires app-engine-patch (Django). Features: * index only specific properties (instead of all string/text properties like in SearchableModel) * Porter stemmers (increase search quality) * sort your results (at least a little bit) via chain-sorting * make DISTINCT queries using a so-called values index * auto-completion via a jQuery plugin * key-based pagination (fully unit-tested implementation of Ryan Barrett's algorithm) * easy to use views and templates (add search support in just a few lines) Since it took a lot of effort to implement all features and make them easy to use we can't give this away for free, though. We initially implemented it for our own projects, but after so many people complained about the lack of full-text search we though we could provide it to others - for a little compensation. Bye, Waldemar Kornewald Thomas Wanschik (the creators of app-engine-patch) -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can't enable authentication for a Google Apps Premier domain that is a subdomain
Hi Dudinha, What error do you get when you try and do this? And is Google Apps set up on spread.com.br, or g.spread.com.br? If Apps itself is set up on the parent domain, you will need to enter that as the authentication domain, not the subdomain you intend to run your app on. -Nick Johnson On Tue, Jun 23, 2009 at 2:02 PM, Dudinha sombr...@gmail.com wrote: Hi all, I can't create an App Engine application to authenticate the users of a Google Apps Premier Edition domain, g.spread.com.br. It is a subdomain of spread.com.br. The strange is that if I put a domain that does not exist, it accept! How can I solve it? Best regards, Eduardo Bortoluzzi Junior Spread Teleinformática Ltda Google Reseller in Brazil -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SMS Verfication Troubles
Hi felurkinda, Done. -Nick Johnson 2009/6/23 felurkinda tengjiaoz...@gmail.com Hi Nick: I've the same problem as Otto. Could manually active my account？ Thanks. = Best Regards, Zhao Tengjiao -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Help me, I have problem to receive sms, when signup for an App Engine account
Hi saltfactory, I've manually activated your account. -Nick Johnson On Tue, Jun 23, 2009 at 5:33 AM, saltfactory saltfact...@gmail.com wrote: I'm student and live in the South of the Korea, I want to learning and developing app on google app engine. so, I signed up app engine account, but I had a problem that I can't received sms to notify account. but I really inputing correct phone number with my country code . when send my phone number to google, I have a errors. please help me, I want to have account to learn google app engine, but don't know how to have account -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Index deletion/building not happening
Hi Scott, Your admin console doesn't show any indexes in 'building' or 'error' state, or any indexes at all under UserItemSection. Can you try uploading your index again now? -Nick Johnson On Tue, Jun 23, 2009 at 4:47 PM, Scott sderick...@gmail.com wrote: Hi, I'm having a little trouble getting a new index built. It's for application skrit, and the model is called UserItemSection, and I'm trying to build a single index for it. Here's how things have gone so far. 1. I updated the index to the server two weeks ago. The index page showed it as 'building'. 2. I checked today and it was still 'building'. 3. I tried vacuuming it, but I think appengine was having index troubles at the time because every time I used any of the commands 'vacuum_indexes', 'update_indexes', or 'update', it would error over the indexes. For example, the output when I used 'update' was: -- Uploading index definitions. Error 500: --- begin server output --- Server Error (500) A server error has occurred. --- end server output --- Your app was updated, but there was an error updating your indexes. Please retry later with appcfg.py update_indexes. -- The other two commands had similar error messages. 4. I successfully ran the 'update_indexes' command, so I guess appengine got over its hiccup. I then tried 'vacuum_indexes' and told it to delete the VocabListSection index. It outputted this: -- Fetching index definitions diff. This index is no longer defined in your index.yaml file. kind: VocabListSection properties: - name: list - name: sect Are you sure you want to delete this index? (N/y/a): y Deleting selected index definitions. 2009-06-23 11:44:43,198 WARNING appcfg.py:697 An index was not deleted. Most likely this is because it no longer exists. kind: VocabListSection properties: - name: list - name: sect -- But on the index page on the dashboard, it's still shown as 'building', so no change from before. Running 'update' or 'update_indexes' has no effect. How can I get this index unstuck and built? -Scott -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Is class Body(db.Model): ok?
It sometimes takes a while for new entity types to be reflected in the Admin Console. Are the entities showing up now? No, I'm still seeing the same errors in the console. My app ID is hogbaytaskpaper. Jesse --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SMS Verfication Troubles
Hi Nick: Thanks!!! 2009/6/24 Nick Johnson (Google) nick.john...@google.com Hi felurkinda, Done. -Nick Johnson 2009/6/23 felurkinda tengjiaoz...@gmail.com Hi Nick: I've the same problem as Otto. Could manually active my account？ Thanks. = Best Regards, Zhao Tengjiao -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- Best Regards, Zhao Tengjiao --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Deploying Application on Google Apps Domain
Can we have a consistent url like *WebApp*.*mydomain.com* instead of WebApp.appspot.com. I have admin rights to the Google Apps domain. -- Thanks and Regards, Nikhil B. Bharadwaj I have felt it and lived it and now it leaves me here, love is the ultimate pain and joy, without it you die with it you perish. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Deploying Application on Google Apps Domain
Nikihil, The appsport names are very consistent. once you have set up your Google Apps Account and hooked in your Google App Engine account it is very map webapp.mydomain.com - you have to remember that google have no control over your DNS so they can only manage appspot.com and also allow you to map your own CNAME to the service. Paul 2009/6/24 nikhil bharadwaj shrinikhilbharad...@gmail.com Can we have a consistent url like *WebApp*.*mydomain.com* instead of WebApp.appspot.com. I have admin rights to the Google Apps domain. -- Thanks and Regards, Nikhil B. Bharadwaj I have felt it and lived it and now it leaves me here, love is the ultimate pain and joy, without it you die with it you perish. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: [ANN]: full-text search for app-engine-patch/Django
Hi Nick. On Jun 24, 1:28 pm, Nick Johnson (Google) nick.john...@google.com wrote: Hi Waldemar, Impressive! Thanks a lot! It's nice to hear that from a Google developer. :) We hope it's enough to bridge the gap until Google releases super- powerful full-text search support (with ranking and spellchecking and phonetic matching and ...). I'm curious - what's the dependency on Django / App Engine patch? This seems like it should be doable as a purely datastore-oriented library. Some low-level parts *should* work directly with any framework (as we just derive from db.StringListProperty). We haven't tested it, but I'm pretty sure if you rip out the values index the Base package would run with webapp, too, though a few app-engine-patch/ragendja utility functions might still be needed. The values index (or DISTINCT index) requires special app-engine- patch extensions because it has to be compatible with transactions. If you modify an entity within a transaction gae-search has to wait for the transaction to finish and only then it's possible to update the values index (which would be in different entity groups). Also, we have to know whether the transaction finished successfully before we can actually update the values index. Here, app-engine-patch provides a few extra signals (post_save_committed, post_delete_committed) which we handle in order to update the index. We plan to port the updater to the new background processing API and maybe that can obviate the need for those signals. So, a future release of the Base package might even support webapp (if there is enough interest). The higher-level parts in the Premium package (e.g., the views and templates) require more complete Django support (more than what django- helper can provide). Though, we already removed the dependency on generic views, so it might be possible to support django-helper in a future release, too. Finally, the auto-complete jQuery plugin is independent of any framework, of course. Bye, Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Efficient way to structure my data model
Thanks very much! I think I understand indexing with Lists much better
now...
In the case of your index with 4 occurrences, this will rather be 6C4.
Yup, I was looking at the max situation when I use 3 categ fields.
Need to be careful as it grows pretty quickly.
On Jun 24, 4:19 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi ecognium,
On Tue, Jun 23, 2009 at 6:11 PM, ecognium ecogn...@gmail.com wrote:
Thanks again - this is very helpful. I will let you know if i run into
any future index creation errors as it could have been caused by any
number of other entries - i mistakenly thought it was all these categ
list-based entries.
So if i understand it right even with a 10 element list for keywords,
there will only be 10 rows when 4 categ fields are used.
Correct - 4C4 * 10C1 index entries for the custom index you specified
earlier.
In the event
I use 'categ' only once in my query along with keywords field, it
will have up to 40 rows (10 from keywords and 4C1 from categ list).
Correct.
Am
I adding these up right?
I do not see myself going beyond 6 elements in the categ list at this
point (I guess the max will be 6C3 = 20 under such a situation).
In the case of your index with 4 occurrences, this will rather be 6C4.
The
keyword list will be probably go into the 20s but do not see anything
beyond that and will always be used only once in the query.
Thanks,
-e
On Jun 23, 3:53 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi ecognium,
On Tue, Jun 23, 2009 at 1:35 AM, ecognium ecogn...@gmail.com wrote:
Thanks, Nick. Let me make sure I understand your comment correctly.
Suppose I have the following data:
ID BlobProp1 BlobProp2-N Keywords
Categ
=
123 blah blah tag1,tag2,tag3
Circle,
Red, Large, Dotted
345 blah blah tag3,tag4,tag5
Square, Blue, Small, Solid
678 blah blah tag1,tag3,tag4
Circle, Blue, Small, Solid
-
The field categ (list) contains four different types - Shape, Color,
Size and Line Type. Suppose the user wants to retrieve all entities
that are Small Dotted Blue Circles then the query will be:
Select * From MyModel where categ = Circle AND categ = Small AND
categ = Blue AND categ = Dotted
When I was reading about exploding indexes the example indicated the
issue was due to Cartesian product of two list elements. I thought the
same will hold true with one list field when used multiple times in a
query.
That is indeed true, though it's not quite the cartesian product - the
datastore won't bother indexing (Circle, Circle, Circle, Circle), or
(Dotted, Dotted, Dotted, Dotted) - it only indexes every unique
combination,
which is a substantially smaller number than the cartesian product. It's
still only tractable for small lists, though, such as the 4 item lists
you're dealing with.
Are you saying the above query will not need {Circle, Red,
Large, Dotted} * {Circle, , , } * {Circle, , , } * {Circle, , , }
number of index entities for entity ID=123?
Correct - if you're not specifying a sort order, you can execute the
query
without any composite indexes whatsoever. The datastore satisfies
equality-only queries using a merge join strategy.
I was getting index errors
when I was using the categ list property four times in my index
specification and that's why I was wondering if I should restructure
things.
How many items did you have in the list you were indexing in that case?
If
your list has 4 items and your index specification lists it 4 times, you
should only get one index entry.
so I am guessing the following spec should not cause any index
issues in the future?
Again, that depends on the number of entries in the 'categ' list. With 4
entries, this will only generate a single index entry, but the number of
entries will expand exponentially as the list increases in size.
-Nick Johnson
- kind: MyModel
properties:
- name: categ
- name: categ
- name: categ
- name: categ
- name: keywords
- name: __key__ # used for paging
Thanks,
-e
On Jun 22, 2:10 am, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi ecognium,
If I understand your problem correctly, every entity will have 0-4
entries
in the 'categ' list, corresponding to the values for each of 4
categories
(eg, Color, Size, Shape, etc)?
The sample query you give, with only equality filters, will be
satisfiable
using the merge join query planner, which doesn't require custom
indexes,
[google-appengine] Re: Java Eclipse plugin dependent on org.eclipse.wst.xml.core.
Thanks for the tip savanna! On May 25, 4:13 pm, Dzsó dzsodzs...@gmail.com wrote: Thx 4 the solution! Dzs On ápr. 15, 04:23, savanna sava...@gmail.com wrote: I had the same problem too. The solution that I found out is checking the Google plugins first. They Eclipse shows the error message for the missing xml plugin. Then I hit the Select Required button on the right side and some packages in the Ganymede Discovery Site are automatically selected and I was good to go. On Apr 9, 10:24 am, Andrew andrew.do...@computer.org wrote: I had the same problem. My solution was to use the Update Manager to install new updates from the Europa Discovery Site (I’m using Europa , rather than Ganymede). I wasn’t very discriminate and selected everything. Download took some time, but after restarting Eclipse, I could download and install the Google App Engine plugins. The version of Eclipse I has using had not been updated for sometime (the machine was being used for other purposes). On Apr 8, 6:45 pm, Alex Neth a...@liivid.com wrote: But I'm not sure where to find that: Google Plugin for Eclipse 3.4 (1.0.0.v200904062334) requires plug-in org.eclipse.wst.xml.core. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: The API call mail.Send() required more quota than is available.
I also eager to know how do you know that You can send about 4600 a minute when it's enabled.. On May 5, 12:28 am, Ray Malone rayish...@gmail.com wrote: I figured out my issue. I needed to enable billing to send more than 32 a minute. You can send about 4600 a minute when it's enabled. On May 4, 11:10 am, Ray Malone rayish...@gmail.com wrote: I don't understand what's happened. It allowed me to send about 100 emails then stopped with this error. Is it because I made multiple requests at the same time? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can't enable authentication for a Google Apps Premier domain that is a subdomain
Hi Nick! I only receive Auth domain, without any further explanation. Both domains (spread.com.br and g.spread.com.br) are in Google Apps Premier Edition. Best regards, Eduardo On Jun 24, 8:33 am, Nick Johnson (Google) nick.john...@google.com wrote: Hi Dudinha, What error do you get when you try and do this? And is Google Apps set up on spread.com.br, or g.spread.com.br? If Apps itself is set up on the parent domain, you will need to enter that as the authentication domain, not the subdomain you intend to run your app on. -Nick Johnson On Tue, Jun 23, 2009 at 2:02 PM, Dudinha sombr...@gmail.com wrote: Hi all, I can't create an App Engine application to authenticate the users of a Google Apps Premier Edition domain, g.spread.com.br. It is a subdomain of spread.com.br. The strange is that if I put a domain that does not exist, it accept! How can I solve it? Best regards, Eduardo Bortoluzzi Junior Spread Teleinformática Ltda Google Reseller in Brazil -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Any interest in a Boston-area users group?
Are there any other AppEngine developers here who are located in the greater Boston area who would be interested in starting a Users Group? I am willing to organize such a thing if there is sufficient interest. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Bucket Size example ?
Hi, I don't really understand the bucket_size parameter in the task config yaml. Is it possible de have an example ? For example, I've 100 tasks in the queue. (rate 1/s) What does it change if my bucket_size = 1 or 10 ? and if I have only 5 tasks ? (rate 1/s) What does it change if my bucket_size = 1 or 10 ? Thank you Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
Sometimes delete operations are implemented to be allowed to take more time than put operations. Could it be that the delete operation in GAE has been allowed to be slow in order to make put operations and/or queries faster? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: why there is no way to build a key id?
Hey Jeff - sorry for the confusion. The idea was that one would be
able to get the unique id from an GOOG and then do a db.put with that
id as an arg. For example, let's say I want to create two entities,
with each referring to each other. I need to do three db.put
operations:
a = Foo()
db.put(a)
b = Foo()
b.ref = a.key()
db.put(b)
a.ref = b.key()
db.put(a)
One would hope to be able to do this with two db.puts.
Thanks,
Jeff
2009/6/16 Jeff S (Google) j...@google.com:
The datastore does not allow key_names which begin with a digit in
order to avoid confusion with an ID, which is numerical. If you want
to use numeric key names, you could add a one letter prefix :-)
Happy coding,
Jeff
On Jun 16, 1:17 am, cryb cbuti...@gmail.com wrote:
Hi Jeff.
Thanks for your reply.. I really hope that in the near future
appengine will support setting key ids for entities.
You mentioned that I can use hooks in order to achieve my goal..
However I was more interested in a solution based on appengine java
sdk, and not on python hooks. Does appengine java sdk provide hooks or
some other similar mechanism?
It seems that for the moment I'll stick to generating key names.
One more question: I've tried to generate some entities with key names
on my local devappserver and I got a strange exception stating that I
can't create key names that start with a digit (?!?)... this holds on
google appengine production servers too or it's just a bug of
devappserver?
Thanks
On Jun 16, 2:45 am, Jeff S (Google) j...@google.com wrote:
Hi cryb,
As you noted, we do not currently allow the ID for a key to be set, as we
ensure that the ID is unique for each existing entity. I recommend using a
key name instead of an ID, as Antoniov suggeted, if possible.
It is technically possible to modify the key of an entity as it is being
converted to a protocol buffer message before it is sent to the datastore.
You could do this using hooks in the API proxy as described in this
article:http://code.google.com/appengine/articles/hooks.htmlAlsoit is
possible to
construct the key for the desired object if you know the ID in advance.
class X(db.Model):
pass
# If you've already created the entity so you have the ID.
x_id = X().put().id()
# Instead of getting by ID, you can create the key manually.
k = db.Key.from_path('X', x_id)
Now you have the desired key without having fetched the object, but the
part
which the model class does not allow is setting the key yourself. So you
could modify the protocol buffer message before it is written to the
datastore, but I don't recommend it.
The decision to allow setting key_names but not IDs is something we may
revisit.
Happy coding,
Jeff
2009/6/12 cryb cbuti...@gmail.com
Hi.. that is to build key names... What I asked was why I can't build
a key ID..
On Jun 12, 5:35 am, Antoniov nio@gmail.com wrote:
Use the code:
s = Story(key_name=xzy123)
Then you create an entity with the key name xzy123.
Check this:
http://code.google.com/intl/en-US/appengine/docs/python/datastore/key...
On 6月12日, 上午1时28分, cryb cbuti...@gmail.com wrote:
Does anyone know why it is possible to build a key name but NOT a key
id? I know key IDs are used as autoincrements, but why can't I just
override this mechanism and build my own key id?
Suppose I want to overwrite an existent entry in my table that has a
key id I know, and also I want to keep that key id after update...
because I can't just build a key id, I am forced to fetch that
entity,
modify it and write it back, instead of just write the updated entity
with the key id I already know - so an additional read to the
datastore.
Is there an obscure reason for that? (both key names and key ids are
prefixed with appid/kind as far as I know so there is no chance of
collision with other apps/kinds)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Datastore delete taking long time
Hmm... On a second thought, the put operation is perhaps also costly in terms of CPU time. I haven't tried doing 20 puts in one page access. Maybe that would also consume a lot of CPU time. On a practical level, this is usually not a big problem when the traffic to the application is low, but imagine an application like Twitter being implemented using GAE and with that kind of huge traffic Twitter has. Millions of datastore updates every minute. If an application like that would run on GAE, the Quota costs for CPU time alone would become astronomical! P.S. Twitter is actually using the Amazon Elastic Compute Cloud at the moment I think. ;-) On Jun 24, 6:36 pm, Anders i...@blabline.com wrote: Sometimes delete operations are implemented to be allowed to take more time than put operations. Could it be that the delete operation in GAE has been allowed to be slow in order to make put operations and/or queries faster? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Model with key_name not have key().id
Thanks. That is what I was confirming. 2009/6/24 Ian Lewis ianmle...@gmail.com: Yes, that is correct. 2009/6/24 Ritesh Nadhani rite...@gmail.com Actually, what I meant was: I cannot have an entity with both key_name and id. They seem to be mutually exclusive for an entity. 2009/6/23 Ian Lewis ianmle...@gmail.com: Ritesh, Also, BTW, The reason that obj1 and obj2 can have different types of keys is because the key is not saved on a model level. It is saved at the entity level. Each entity can have id or key_names. Apparently obj2 was saved without a key name and was given a numeric key. obj1 was saved with a key_name. So to answer your question, yes, within a model/Kind you can have some entities that have a numeric id and some that have key names. Ian 2009/6/24 Ian Lewis ianmle...@gmail.com Ritesh, Keys can either have a numeric id or a key_name. In order to access the key name for entities that you saved with a key_name you need to use the key_name() method. instead of obj.key().id(), try obj.key().name() Incedentally you can also use obj.key().id_or_name() to get either the id or the name depending on which you are using. On Wed, Jun 24, 2009 at 3:08 PM, Ritesh Nadhani rite...@gmail.com wrote: Hi When I save a model with (key_name=...) value, the key does not seem to have an id. Eg. Mode1(key_name='xxx').put() obj.key().id() - None Model2() obj2.key().id() - 1 Is this intented? Can I have both key_name and id in a mode? -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- Ritesh http://www.riteshn.com -- === 株式会社ビープラウド イアン・ルイス 〒150-0012 東京都渋谷区広尾1-11-2アイオス広尾ビル604 email: ianmle...@beproud.jp TEL：03-5795-2707 FAX：03-5795-2708 http://www.beproud.jp/ === -- Ritesh http://www.riteshn.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Request: Task Queue quota increase
Right now the Task Queue quota is set to 10k tasks, which is rather small. Can this be bumped to at least 100k? Thanks Vivek --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any interest in a Boston-area users group?
I would attend sporadically. Probably not the level of enthusiasm you're looking for, but at least it's honest :) On Jun 24, 11:29 am, Adam adam.crossl...@gmail.com wrote: Are there any other AppEngine developers here who are located in the greater Boston area who would be interested in starting a Users Group? I am willing to organize such a thing if there is sufficient interest. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Problem installing PIL 1.1.6, on Windows Vista 64 development machine
Hi All, I've scoured the internet looking for info to help resolve this issue. I've also tried a number of things on my local installation to trying to resolve this issue, but both have been unsuccessful. I am encountering an error running the Python Imaging Library 1.1.6 for Python 2.5 (Windows only) on my local machine. The installer is giving the following error: Cannot install Python version 2.5 required, which was not found in the registry. I have Python 2.5.4 installed, 64 bit version. I have checked the registry and it seems to be properly set. Here are the values for a couple of the keys: key name: HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore \2.5\PythonPath (Default) value: C:\Python25\Lib;C:\Python25\DLLs;C:\Python25\Lib \lib-tk key name: HKEY_LOCAL_MACHINE\SOFTWARE\Python\PythonCore \2.5\InstallPath (Default) value: C:\Python25\ I encounter the same issue when running the setuptools 0.6c9 installer. Any help would be greatly appreciated. Thanks, Scott --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Odd memcache behavior across multiple app instances
Hi Kim,
Can you please try flushing your memcache and running your tests again? The
issue should now be fixed.
-Nick
On Tue, Jun 23, 2009 at 2:42 PM, Kim Riber kimsteenri...@gmail.com wrote:
Yes. There is no other activity when running step 3.
The two values I get back, are some of the last ones I write to the
cache (I return all the values I write).
So it looks a lot like the write threads are writing to two instances
of memcache, and when I start reading back, I see the 2 different
values from the 2 insances.
When waiting for an hour or so, when I run step 3 it returns only one
value again.
If I run the heavy load task (and spawn more instances), running step
3 will start reporting the 2 values again. Like the cluster with the
second memcache instance is getting fired up, and reporting it's old
value.
the code for step 3 is something like this:
def read_memcache()
val = memcache.get('my_test_key')
return val
I could try to set up an application with the 3 steps.
Cheers
Kim
On Jun 23, 2:52 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Kim,
Are you able to send us the code you use for step 3? And are you certain
nothing is changing the memcache concurrently with step 3?
On Tue, Jun 23, 2009 at 1:44 PM, Kim Riber kimsteenri...@gmail.com
wrote:
Hi Nick
I run the test in 3 steps (with half a minute in between):
1. Heavy load process to spawn appinstances
2. Write a lot of random values to the same key
3. Read the key from multiple threads.
I can repeat 3rd step, and still get the same result (mostly 2
different values)
It seems like I hit 2 different memcache servers with different view
on what is in the cache at that key
Also, are you running this against the dev_appserver, or in
production?
How do I see that?
We are running towards .appspot.com
If you are testing against the local development server, that's the
dev_appserver. If you're testing code running on appspot.com, that's
production.
-Nick Johnson
http://appspot.com
-Kim
On Jun 23, 12:39 pm, Nick Johnson (Google) nick.john...@google.com
wrote:
Hi Kim,
It's not clear from your description exactly how you're performing
your
tests. Without extra information, the most likely explanation would
be
that
you're seeing a race condition in your code, where the key is
modified
between subsequent requests to the memcache API.
Also, are you running this against the dev_appserver, or in
production?
-Nick Johnson
On Tue, Jun 23, 2009 at 7:18 AM, Kim Riber kimsteenri...@gmail.com
wrote:
Just made another test, to confirm the behavior I see.
This example is much simpler, and simply has 10 threads writing
random
values to memcahce to the same key.
I would expect the last value written to be the one left in
memcache.
When afterwards, having 4 threads reading 10 times from that same
key,
they return 2 different values.
This only happens if I prior to the writing threads, run some heavy
tasks, to force gae to spawn more app instances.
It seems like each server cluster might have its own memcache,
independant from each other. I hope this is not true. From a thread
from Ryan
http://groups.google.com/group/google-appengine/browse_thread/thread/.
..
he states that
as for the datastore, and all other current stored data APIs like
memcache, there is a single, global view of data. we go to great
lengths to ensure that these APIs are strongly consistent.
Regards
Kim
On Jun 17, 8:51 pm, Kim Riber kimsteenri...@gmail.com wrote:
To clarify a bit:
one thread from our server runs one loop with a unique id.
each requests stores a value in memcache and returns that value.
In
the following request, the memcache is queried if the value just
written, is in the cache.
This sometimes fail.
My fear is that it is due to the requests changing to another app
instance and then suddently getting wrong data.
instance 1 + +
instance 2 --
Hope this clears out the example above a bit
Cheers
Kim
On Jun 17, 7:52 pm, Kim Riber kimsteenri...@gmail.com wrote:
Hi,
I'm experiencing some rather strange behavior from memcache. I
think
I'm getting different data back from memcache using the same
key
The issue I see is that when putting load on our application,
even
simple memcache queries are starting to return inconsistant
data.
When
running the same request from multiple threads, I get different
results.
I've made a very simple example, that runs fine on 1-200
threads,
but
if I put load on the app (with some heavier requests) just
before I
run my test, I see different values coming back from memcache
using
[google-appengine] Re: Over Quota: Datastore Indices Count
Jeff, I have the same issue, my app id is neatschool. Could you please assist me as well? Thanks On Jun 23, 9:46 am, Jeff S (Google) j...@google.com wrote: Hi Thomas, I've reset the index count, apologies for the inconvenience. Happy coding, Jeff On Tue, Jun 23, 2009 at 9:12 AM, Thomas McKay -www.winebythebar.com thomasfmc...@gmail.com wrote: My app is getting Your application is exceeding a quota: Datastore Indices Count after a vacuum. Could I get a reset of indices count please? App: winebythebar On May 23, 11:50 pm, fedestabile stabile.feder...@gmail.com wrote: Thanks Jeff, it's working fine now :) Cheers, Fred On May 20, 5:48 am, Jeff S (Google) j...@google.com wrote: Fred, I've reset the indexcountso you should be all set. Kaspars I've reset the indexquotaforyourapp as well and it looks like you had some indexes which were stuck in the building state so I've moved them to error. Happy coding, Jeff On May 19, 2:11 am, kaspars...@gmail.com kaspars...@gmail.com wrote: I've only 8 indexes at the moment and I'm also getting Your applicationisexceedingaquota:DatastoreIndicesCount. The applicationis basically stuck for couple days. Could you please reset thequota, app ID is lasi2. Thanks, Kaspars- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: why there is no way to build a key id?
Hi Jeff,
This is an idea that we're aware of as well. If you could reserve the next
ID in advance, then you could actually do this in one put since multiple
entities could be sent in one batch :-) The workaround available now is to
use the key_name, but the difficulty becomes ensuring that the key_name is
unique. Here is a feature request which I'm aware of which is along these
lines (though the approach differs slightly):
http://code.google.com/p/googleappengine/issues/detail?id=1003
Thank you,
Jeff
2009/6/24 Jeff Enderwick jeff.enderw...@gmail.com
Hey Jeff - sorry for the confusion. The idea was that one would be
able to get the unique id from an GOOG and then do a db.put with that
id as an arg. For example, let's say I want to create two entities,
with each referring to each other. I need to do three db.put
operations:
a = Foo()
db.put(a)
b = Foo()
b.ref = a.key()
db.put(b)
a.ref = b.key()
db.put(a)
One would hope to be able to do this with two db.puts.
Thanks,
Jeff
2009/6/16 Jeff S (Google) j...@google.com:
The datastore does not allow key_names which begin with a digit in
order to avoid confusion with an ID, which is numerical. If you want
to use numeric key names, you could add a one letter prefix :-)
Happy coding,
Jeff
On Jun 16, 1:17 am, cryb cbuti...@gmail.com wrote:
Hi Jeff.
Thanks for your reply.. I really hope that in the near future
appengine will support setting key ids for entities.
You mentioned that I can use hooks in order to achieve my goal..
However I was more interested in a solution based on appengine java
sdk, and not on python hooks. Does appengine java sdk provide hooks or
some other similar mechanism?
It seems that for the moment I'll stick to generating key names.
One more question: I've tried to generate some entities with key names
on my local devappserver and I got a strange exception stating that I
can't create key names that start with a digit (?!?)... this holds on
google appengine production servers too or it's just a bug of
devappserver?
Thanks
On Jun 16, 2:45 am, Jeff S (Google) j...@google.com wrote:
Hi cryb,
As you noted, we do not currently allow the ID for a key to be set, as
we
ensure that the ID is unique for each existing entity. I recommend
using a
key name instead of an ID, as Antoniov suggeted, if possible.
It is technically possible to modify the key of an entity as it is
being
converted to a protocol buffer message before it is sent to the
datastore.
You could do this using hooks in the API proxy as described in this
article:http://code.google.com/appengine/articles/hooks.htmlAlsoit is
possible to
construct the key for the desired object if you know the ID in
advance.
class X(db.Model):
pass
# If you've already created the entity so you have the ID.
x_id = X().put().id()
# Instead of getting by ID, you can create the key manually.
k = db.Key.from_path('X', x_id)
Now you have the desired key without having fetched the object, but
the part
which the model class does not allow is setting the key yourself. So
you
could modify the protocol buffer message before it is written to the
datastore, but I don't recommend it.
The decision to allow setting key_names but not IDs is something we
may
revisit.
Happy coding,
Jeff
2009/6/12 cryb cbuti...@gmail.com
Hi.. that is to build key names... What I asked was why I can't
build
a key ID..
On Jun 12, 5:35 am, Antoniov nio@gmail.com wrote:
Use the code:
s = Story(key_name=xzy123)
Then you create an entity with the key name xzy123.
Check this:
http://code.google.com/intl/en-US/appengine/docs/python/datastore/key...
On 6月12日, 上午1时28分, cryb cbuti...@gmail.com wrote:
Does anyone know why it is possible to build a key name but NOT
a key
id? I know key IDs are used as autoincrements, but why can't I
just
override this mechanism and build my own key id?
Suppose I want to overwrite an existent entry in my table that
has a
key id I know, and also I want to keep that key id after
update...
because I can't just build a key id, I am forced to fetch that
entity,
modify it and write it back, instead of just write the updated
entity
with the key id I already know - so an additional read to the
datastore.
Is there an obscure reason for that? (both key names and key ids
are
prefixed with appid/kind as far as I know so there is no chance
of
collision with other apps/kinds)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this
[google-appengine] Re: why there is no way to build a key id?
see also
http://groups.google.com/group/google-appengine/browse_thread/thread/3f8cfeaf7dc2eb72/d5d599180fe47e02?lnk=gstq=ryan+allocate#d5d599180fe47e02
On Jun 24, 11:33 am, Jeff S (Google) j...@google.com wrote:
Hi Jeff,
This is an idea that we're aware of as well. If you could reserve the next
ID in advance, then you could actually do this in one put since multiple
entities could be sent in one batch :-) The workaround available now is to
use the key_name, but the difficulty becomes ensuring that the key_name is
unique. Here is a feature request which I'm aware of which is along these
lines (though the approach differs slightly):
http://code.google.com/p/googleappengine/issues/detail?id=1003
Thank you,
Jeff
2009/6/24 Jeff Enderwick jeff.enderw...@gmail.com
Hey Jeff - sorry for the confusion. The idea was that one would be
able to get the unique id from an GOOG and then do a db.put with that
id as an arg. For example, let's say I want to create two entities,
with each referring to each other. I need to do three db.put
operations:
a = Foo()
db.put(a)
b = Foo()
b.ref = a.key()
db.put(b)
a.ref = b.key()
db.put(a)
One would hope to be able to do this with two db.puts.
Thanks,
Jeff
2009/6/16 Jeff S (Google) j...@google.com:
The datastore does not allow key_names which begin with a digit in
order to avoid confusion with an ID, which is numerical. If you want
to use numeric key names, you could add a one letter prefix :-)
Happy coding,
Jeff
On Jun 16, 1:17 am, cryb cbuti...@gmail.com wrote:
Hi Jeff.
Thanks for your reply.. I really hope that in the near future
appengine will support setting key ids for entities.
You mentioned that I can use hooks in order to achieve my goal..
However I was more interested in a solution based on appengine java
sdk, and not on python hooks. Does appengine java sdk provide hooks or
some other similar mechanism?
It seems that for the moment I'll stick to generating key names.
One more question: I've tried to generate some entities with key names
on my local devappserver and I got a strange exception stating that I
can't create key names that start with a digit (?!?)... this holds on
google appengine production servers too or it's just a bug of
devappserver?
Thanks
On Jun 16, 2:45 am, Jeff S (Google) j...@google.com wrote:
Hi cryb,
As you noted, we do not currently allow the ID for a key to be set, as
we
ensure that the ID is unique for each existing entity. I recommend
using a
key name instead of an ID, as Antoniov suggeted, if possible.
It is technically possible to modify the key of an entity as it is
being
converted to a protocol buffer message before it is sent to the
datastore.
You could do this using hooks in the API proxy as described in this
article:http://code.google.com/appengine/articles/hooks.htmlAlsoitis
possible to
construct the key for the desired object if you know the ID in
advance.
class X(db.Model):
pass
# If you've already created the entity so you have the ID.
x_id = X().put().id()
# Instead of getting by ID, you can create the key manually.
k = db.Key.from_path('X', x_id)
Now you have the desired key without having fetched the object, but
the part
which the model class does not allow is setting the key yourself. So
you
could modify the protocol buffer message before it is written to the
datastore, but I don't recommend it.
The decision to allow setting key_names but not IDs is something we
may
revisit.
Happy coding,
Jeff
2009/6/12 cryb cbuti...@gmail.com
Hi.. that is to build key names... What I asked was why I can't
build
a key ID..
On Jun 12, 5:35 am, Antoniov nio@gmail.com wrote:
Use the code:
s = Story(key_name=xzy123)
Then you create an entity with the key name xzy123.
Check this:
http://code.google.com/intl/en-US/appengine/docs/python/datastore/key...
On 6月12日, 上午1时28分, cryb cbuti...@gmail.com wrote:
Does anyone know why it is possible to build a key name but NOT
a key
id? I know key IDs are used as autoincrements, but why can't I
just
override this mechanism and build my own key id?
Suppose I want to overwrite an existent entry in my table that
has a
key id I know, and also I want to keep that key id after
update...
because I can't just build a key id, I am forced to fetch that
entity,
modify it and write it back, instead of just write the updated
entity
with the key id I already know - so an additional read to the
datastore.
Is there an obscure reason for that? (both key names and key ids
are
prefixed with appid/kind as far as I know so there is no chance
of
collision with other apps/kinds)- Hide quoted text -
- Show quoted text -
[google-appengine] Re: : full-text search for app-engine-patch/Django
Hi Waldemar Kudos for trying to address a serious limitation with google app engine. What could we do to convince you to open-source your search technology? By we, I mean the AppEngine dev community. If it is a monetary issue, maybe Google could cut your company a cheque to cover your investment and we could finally put this crippled search issue to rest. On Jun 24, 5:45 am, Waldemar Kornewald wkornew...@gmail.com wrote: Hi Nick. On Jun 24, 1:28 pm, Nick Johnson (Google) nick.john...@google.com wrote: Hi Waldemar, Impressive! Thanks a lot! It's nice to hear that from a Google developer. :) We hope it's enough to bridge the gap until Google releases super- powerful full-text search support (with ranking and spellchecking and phonetic matching and ...). I'm curious - what's the dependency on Django / App Engine patch? This seems like it should be doable as a purely datastore-oriented library. Some low-level parts *should* work directly with any framework (as we just derive from db.StringListProperty). We haven't tested it, but I'm pretty sure if you rip out the values index the Base package would run with webapp, too, though a few app-engine-patch/ragendja utility functions might still be needed. The values index (or DISTINCT index) requires special app-engine- patch extensions because it has to be compatible with transactions. If you modify an entity within a transaction gae-search has to wait for the transaction to finish and only then it's possible to update the values index (which would be in different entity groups). Also, we have to know whether the transaction finished successfully before we can actually update the values index. Here, app-engine-patch provides a few extra signals (post_save_committed, post_delete_committed) which we handle in order to update the index. We plan to port the updater to the new background processing API and maybe that can obviate the need for those signals. So, a future release of the Base package might even support webapp (if there is enough interest). The higher-level parts in the Premium package (e.g., the views and templates) require more complete Django support (more than what django- helper can provide). Though, we already removed the dependency on generic views, so it might be possible to support django-helper in a future release, too. Finally, the auto-complete jQuery plugin is independent of any framework, of course. Bye, Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Any interest in a Boston-area users group?
I'd be interested. On Jun 24, 11:29 am, Adam adam.crossl...@gmail.com wrote: Are there any other AppEngine developers here who are located in the greater Boston area who would be interested in starting a Users Group? I am willing to organize such a thing if there is sufficient interest. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Creating new account in Google App Engine - Phone number has already been used message
Hi, I'm trying to create a new app engine account\application, but I fail to pass the activation by SMS screen. I keep getting the following message: The phone number has been sent too many messages or has already been used to confirm an account. I got this message on my first attempt to open the account. can anyone help? Thanks, Shahar --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: UnacceptableVersionError when trying to import Django 1.0
I'm experiencing the same error on both the dev server and when
deployed.
I've created a new issue at:
http://code.google.com/p/googleappengine/issues/detail?id=1758
On Jun 19, 10:37 pm, James thelevybre...@gmail.com wrote:
I added this code to my main.py file:
from google.appengine.dist import use_library
use_library('django', '1.0')
And this error is returned:
class 'google.appengine.dist._library.UnacceptableVersionError':
django 1.0 was requested, but 0.96.1.None is already in use
args = ('django 1.0 was requested, but 0.96.1.None is already in
use',)
message = 'django 1.0 was requested, but 0.96.1.None is already
in use'
My local django installation is 1.02, but there may be some important
step I forgot. (And yes, tried restarting the dev server, etc.)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: 403 Application Over Quota Problem - Not True!
same thing here, my daily Outgoing Bandwidth shows a limit of 1GB on my quota details page. google mentions though there is a daily limit of 10GB even if billing is not enabled. http://code.google.com/appengine/docs/quotas.html GOOLE please HELP app id: purpleimageapp thanx! On Jun 22, 3:19 pm, Devel63 danstic...@gmail.com wrote: All of a sudden, my app is returning 403 application overquota whenever I do anything a bit strenuous. All of the quotas are WAY under, but things that used to work fine are now triggering this message. A guess is that the budgeting process has become much more fine- grained, and is mistakenly extrapolating from one request that may do a number of DB writes and take 10 seconds. But these are extremely rare. The app name is judysapps-qa. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 403 Application Over Quota Problem - Not True!
answering my own problem here. app engine quotas were reduced rom 10GB to 1GB per day outgoing bandwidth — crazyy reco On Jun 24, 6:12 pm, reco r...@nex9.com wrote: same thing here, my daily Outgoing Bandwidth shows a limit of 1GB on myquotadetails page. google mentions though there is a daily limit of 10GB even if billing is not enabled. http://code.google.com/appengine/docs/quotas.html GOOLE please HELP app id: purpleimageapp thanx! On Jun 22, 3:19 pm, Devel63 danstic...@gmail.com wrote: All of a sudden, my app is returning 403 application overquota whenever I do anything a bit strenuous. All of the quotas are WAY under, but things that used to work fine are now triggering this message. A guess is that the budgeting process has become much more fine- grained, and is mistakenly extrapolating from one request that may do a number of DB writes and take 10 seconds. But these are extremely rare. The app name is judysapps-qa. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 403 Application Over Quota Problem - Not True!
he cc, yes this is crazy at least if you enable billing they could give you 5GB outgoing bandwidth for free. have the free accounts limited to 1GB i think is fine. reco On Jun 22, 10:08 pm, cc c...@gamegiants.net wrote: Yep same problem here 403s everywhere! Try using the Remote API it is now useless with the newquota. Problem seems to be that the burstquotahas also dropped by a factor of 10 along with the dailyquota. Google has the burstquotaset to try to help keep you under the dailyquota, spreading thequotaover a 24 hour period. This feature seems to render most apps unusable now. Anyone trying out app engine under the freequotais quickly going to decide not to use app engine for their web app. Kind of defeats the purpose of even offering the free version. Google should have kept the burstquotaconstant and only reduced the dailyquota. This is just further proof that the accountants are now running the show at Google. On Jun 22, 3:06 pm, Mike Wesner m...@konsole.net wrote: enabling billing seems to have sped things up and so far has stopped the 403's. I still think something is fishy since we had not warnings in the appspot dashboard and are way under free quotas. On Jun 22, 4:58 pm, Mike Wesner m...@konsole.net wrote: Several of our appspot instances are having this exact same issue. We are way underquota, hardly hitting the appid at all and we see 403 on static files and other things. Random 500 errors too. We are enabling billing on a few of our test instances which we hope will help, but I can't see how it will make a difference since we are so far underquota/usage rates. ANY GOOGLERS READING THIS? This is a serious issue and we get ZERO information or support from google. How can a company use this stuff when its so flakey? On Jun 22, 2:19 pm, Devel63 danstic...@gmail.com wrote: All of a sudden, my app is returning 403 application overquota whenever I do anything a bit strenuous. All of the quotas are WAY under, but things that used to work fine are now triggering this message. A guess is that the budgeting process has become much more fine- grained, and is mistakenly extrapolating from one request that may do a number of DB writes and take 10 seconds. But these are extremely rare. The app name is judysapps-qa. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] How does Django (on GAE) handle contrib libs such as User?
I am currently using app-engine-patch, but now that Django 1.0 has been released on GAE, I'd like to port my app over. App-engine-patch zipimports Django and patches a lot of things to make it work out of the box, for example the User model is modified to use GAE models, and memcache is used for sessions. How does the Django lib on GAE provide this functionality? What happens when I invoke 'request.user'? The SDK just uses my local installation of Django, which is obviously unpatched. IIRC, app-engine- patch creates a session model and stores user sessions in the datastore and caches each session in memcache. 'request.user' pulls the session and the user from either memcache or datastore if the cache is purged. I can only think of this issue when porting my app to the Django provided by GAE. Anything else I should be aware of? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Does cron jobs take more time to init than normal handlers
It could be that the cron jobs are run on a different farm of computers. If you call cron every 5 min it means the server has to start cold for every cron request. If you use the regular URL from a browser or such you likely have a warm server running. 2009/6/24 Mariano Benitez mari...@benitez.nu: Hello, Now that I got cron, I moved something I used to do in a normal handler to use a cache and refresh every 5 minutes. What I discovered now is that what used to take 400ms in the normal handler is now taking 800+ms in the cron handler. (I do the exact same thing, really) I don't know if cron handlers are being cached or since I do it not very frequently I have to pay that price. Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Frequent Timeouts - do they count towards quota??
Now that the quota has been reduced for free accounts, I am much more concerned over the timeouts and if the CPU time for these timeouts count towards the quota. The CPU time used when a timeout occurs is much longer than a successful request. I have yet to figure out why I get timeouts on certain requests, requests that normally work during any given day. I see jumps from 300 to 3000 when a timeout happens. Anyone know if these count? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Python Vs Java in GAE
In order to address this issue properly, one really needs to step back a bit and focus on the bigger picture. The real issue is not so much Java vs Python but rather statically- typed vs dynamically-typed languages. In the days of old, the Java high-priests blew off dynamically-typed languages because they thought they were tinker-toys...fun, distracting things that couldn't scale. The market has proven them quite wrong. Can you say Google App Engine, Python and scalability? Now I'm not gonna bash Java 'cause if people can use it and truly create value for their stakeholders, then I say all the power to them. But from my perspective I would have to say that Python makes a lot of *business* sense. It's easy to learn and easy to use, which means everyday, ordinary people can make sense of the code. So does this really matter? The answer depends on your philosophy. If you believe in the power of many and the wisdom of crowds, then Python is the way to go. Cheers. On Jun 11, 9:56 am, Nash-t timna...@gmail.com wrote: David, Good summary but I'd add one more thing. As long as Google remains committed to GWT , without releasing a comparable widget set for all languages, I think the writing is on the wall. Already the Google Wave framework favorsjava. That is probably a glimpse of the future for Google, which is a shame because I preferpythontojavafor most projects. -Tim On Jun 11, 8:08 am, neil foreverneilyo...@googlemail.com wrote: David, perfect summary, great post. Exactly my thoughts on this. There is also an Eclipse version forPython, not used yet. BTW: Is runtime performance comparable? Thanks On 11 Jun., 15:18, Lawrence Fitzpatrick fitzp...@gmail.com wrote: I believe that the GAE team has said they are committed to both, so you can probably take them at their word and expect to just timing differences as to when feature x appears on each platform. Therefore what remains is your learning curve and the type of language you prefer. Do you knowPythonorJavabetter? Why not start there? Do you prefer the more static type checking ofJavaor the more runtime type checking ofPython? I've found that most people naturally prefer, and hence are more productive, in one than the other. And lastly, which community do you prefer :) -fitz --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Using the Humanize part of Django templates
Forget the settings.py file.
Have you tried to import the humanize file in the .py file that uses
the templates that needs these filters
import django.contrib.humanize.templatetags.humanize
The code will register the filters during the load.
NO need to use
{% load humanize %}
because the filters are already registered
2009/6/24 MajorProgamming sefira...@gmail.com:
I created a settings.py file and added the INSTALLED APPS but nothing
helps. (the error, btw, is TemplateSyntaxError: 'humanize' is not a
valid tag library: Could not load template library from
django.templatetags.humanize, No module named humanize). Remember,
I'm not using Django; just the forms.
So how can I accomplish this? All I really want to do is automatically
put commas into numbers (e.g. 1000 = 1,000), by using templates...
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Getting copies of all sent mail
Does anybody has any idea on this? On Jun 23, 6:06 pm, anjs jnanj...@gmail.com wrote: Is it a limitation from google that it sends the email to sender in all cases? Although the google doc says it sends email to sender only in case of error and bouunce messages, http://code.google.com/appengine/docs/python/mail/overview.html but the above thread discussion mentions that it is intended behavior and email will be sent in all cases. Can anybody please confirm. We dont want the sender to be received emails. Is this possible? thanks On Jun 7 2008, 5:05 am, Brian Hartvigsen tre...@gmail.com wrote: Sorry apparently I'm majorly dense today ;-) I swear I read that. Looks like I'll be setting up a filter in Gmail. -- Brian On Jun 6, 4:07 pm, Daniel O'Brien d...@google.com wrote: Hi Brian, This is the intended behavior, and discussed in the Mail API Overview:http://code.google.com/appengine/docs/mail/overview.html Daniel On Jun 6, 10:20 am, Brian Hartvigsen tre...@gmail.com wrote: So I'm sending out a validation link to users when they submit an new email address to my app. However everytime an email is sent I'm gettinga copy in my email as well. This happens even if the address is valid and the message was properly delivered, The headers of the message sent to the valid email and them message I receive are almost identical, but not quite.. Original Message, properly delivered: Delivered-To: brian.and...@brianandjenny.com Received: by 10.90.35.15 with SMTP id i15cs27574agi; Thu, 5 Jun 2008 17:24:28 -0700 (PDT) Received: by 10.100.154.19 with SMTP id b19mr3584989ane. 115.1212711867929; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Return-Path: 3u4nisayjdm8eczd8317v36.x97wc3v8.v8yczhwc3v8v8y4z88j@apphosting.bounces.google.com Received: from an-out-0910.google.com (an-out-0910.google.com [209.85.132.190]) by mx.google.com with ESMTP id d19si8287797and. 17.2008.06.05.17.24.27; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Received-SPF: pass (google.com: domain of 3u4nisayjdm8eczd8317v36.x97wc3v8.v8yczhwc3v8v8y4z88j@apphosting.bounces.google.com designates 209.85.132.190 as permitted sender) client- ip=209.85.132.190; Authentication-Results: mx.google.com; spf=pass (google.com: domain of 3u4nisayjdm8eczd8317v36.x97wc3v8.v8yczhwc3v8v8y4z88j@apphosting.bounces.google.com designates 209.85.132.190 as permitted sender) smtp.mail=3u4nisayjdm8eczd8317v36.x97wc3v8.v8yczhwc3v8v8y4z88j@apphosting.bounces.google.com Received: by an-out-0910.google.com with SMTP id c35so4004851ana.3 for brian.and...@brianandjenny.com; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.90.79.12 with SMTP id c12mr1313031agb. 29.1212711867576; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Message-ID: 0016361e886814f06e044ef47...@google.com Date: Thu, 05 Jun 2008 17:24:27 -0700 Subject: Confirm email address From: tre...@gmail.com To: brian.and...@brianandjenny.com Content-Type: text/plain; charset=ISO-8859-1; Format=Flowed Content-Transfer-Encoding: 7bit Copy received at developer account: Delivered-To: tre...@gmail.com Received: by 10.151.10.15 with SMTP id n15cs34681ybi; Thu, 5 Jun 2008 17:24:27 -0700 (PDT) Received: by 10.90.93.13 with SMTP id q13mr2366598agb. 78.1212711867663; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Return-Path: 3u4nisayjbs8eczd8317v36.x97eczd8317v36@apphosting.bounces.google.com Received: from an-out-0910.google.com (an-out-0910.google.com [209.85.132.184]) by mx.google.com with ESMTP id 36si4223174aga. 18.2008.06.05.17.24.27; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Received-SPF: pass (google.com: domain of 3u4nisayjbs8eczd8317v36.x97eczd8317v36@apphosting.bounces.google.com designates 209.85.132.184 as permitted sender) client- ip=209.85.132.184; Authentication-Results: mx.google.com; spf=pass (google.com: domain of 3u4nisayjbs8eczd8317v36.x97eczd8317v36@apphosting.bounces.google.com designates 209.85.132.184 as permitted sender) smtp.mail=3u4nisayjbs8eczd8317v36.x97eczd8317v36@apphosting.bounces.google.com Received: by an-out-0910.google.com with SMTP id c25so3855286anc.6 for tre...@gmail.com; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) MIME-Version: 1.0 Received: by 10.90.79.12 with SMTP id c12mr1313031agb. 29.1212711867576; Thu, 05 Jun 2008 17:24:27 -0700 (PDT) Message-ID: 0016361e886814f06e044ef47...@google.com Date: Thu, 05 Jun 2008 17:24:27 -0700 Subject: Confirm email address From: tre...@gmail.com To: brian.and...@brianandjenny.com Content-Type: text/plain; charset=ISO-8859-1; Format=Flowed Content-Transfer-Encoding: 7bit The sending code
[google-appengine] Re: Request: Task Queue quota increase
Seconded. For the queues to be useful for some of the applications I currently use client-side polling to accomplish, the daily number of tasks would have to be several orders of magnitude larger (100M). Ben On Jun 24, 10:33 am, vivpuri vivpu...@gmail.com wrote: Right now the Task Queue quota is set to 10k tasks, which is rather small. Can this be bumped to at least 100k? Thanks Vivek --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
