[google-appengine] Sluggish Request Latency
I experience round-trip latency from a browser of around one second. (My request log indicates < 25ms for many requests in the application) Is this explainable/expected/considered acceptable? It would certainly make "as you type" updates as per the Wave demos hard to emulate! I am using Java, so maybe this is not such a problem for python apps and is related to routing to the JVM, I don't know. Should I be able to plan for significantly reduced request latency in the near future? - Martyn --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How can I know if the entity is created or just updated in post call hook?
On Tue, Aug 11, 2009 at 7:29 AM, Takashi Matsuo wrote:
>
> Hi Nick,
>
> On Thu, Jul 30, 2009 at 4:48 PM, Nick Johnson
> (Google) wrote:
>> Hi Takashi,
>>
>> If the entity being put has neither key name nor id, a new id will be
>> created for it when it's put, so it will always result in a new entity being
>> inserted rather than an existing entity being replaced. Likewise, if you see
>> an entity with an id being put, it is almost certainly an update to an
>> existing entity (unless someone was messing around with the low-level API),
>> or the entity was deleted and is now being reinserted.
>>
>> If the entity being put has a key name, the only way to tell for certain if
>> it's replacing an existing entity is to do the put operation in a
>> transaction, and do a get on the key first.
>
> Thank you for clarification. Here comes another question.
> Is there any way to tell if particular PUT operation is in a
> transaction or not, and
> tell if the transaction is committed or rollbacked?
>
> I'd like to add a capability for adding arbitrary callback that will
> be executed only when entities are actually put (that won't be
> executed when the put operation is rollback-ed).
You probably want to look into the datastore's support for hooks:
http://code.google.com/appengine/articles/hooks.html
-Nick Johnson
>
> --- Takashi
>
>> -Nick Johnson
>>
>> On Thu, Jul 30, 2009 at 7:19 AM, Takashi Matsuo
>> wrote:
>>>
>>> Hi,
>>>
>>> I'm writing a capability for registering db_hook dynamically. The code
>>> snippet follows the mail body.
>>> It works well except my code can not distinguish newly created entity
>>> from updated entity.
>>>
>>> I'd like to know how can I know if the entity is created or just
>>> updated. What is the most appropriate way?
>>>
>>> Regards,
>>>
>>>
>>> --
>>> from google.appengine.api import apiproxy_stub_map
>>> from google.appengine.api import datastore
>>> from google.appengine.ext import db
>>>
>>> post_save_hooks = {}
>>>
>>> def register_post_save_hook(func, model):
>>> global post_save_hooks
>>> kind = model.kind()
>>> func_list = post_save_hooks.get(kind, None)
>>> if func_list is None:
>>> func_list = []
>>> func_list.append(func)
>>> post_save_hooks[kind] = func_list
>>>
>>>
>>> def execute_hooks(kind, key, entity):
>>> func_list = post_save_hooks.get(kind, None)
>>> if func_list is not None:
>>> entity.key_.CopyFrom(key)
>>> e = datastore.Entity._FromPb(entity)
>>> instance = db.class_for_kind(kind).from_entity(e)
>>> for func in func_list:
>>> func(instance)
>>>
>>> def db_hook(service, call, request, response):
>>> if call == 'Put':
>>> from kay.utils.db_hook import execute_hooks
>>> for key, entity in zip(response.key_list(), request.entity_list()):
>>> kind = model_name_from_key(key)
>>> execute_hooks(kind, key, entity)
>>>
>>> apiproxy_stub_map.apiproxy.GetPostCallHooks().Append(
>>> 'db_hook', db_hook, 'datastore_v3')
>>>
>>> --
>>>
>>> --
>>> Takashi Matsuo
>>>
>>>
>>
>>
>>
>> --
>> Nick Johnson, Developer Programs Engineer, App Engine
>>
>> >
>>
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Over Quota: Datastore Indices Count
I've reset quotas for me-trics and labpixiesdata. I can't reset NiklasRTZ's index quota, since he didn't provide an App ID. -Nick Johnson On Tue, Aug 11, 2009 at 7:24 AM, Guy Rutenberg wrote: > > Hi Jeff, > > We are experiencing the same issue with app-id: labpixiesdata. It > reports Datastore Indices Count is 100 of 100, while the index page > only lists 3 indexes and our entities are small without many > properties. I don't think we did an index vacuum (at least not > recently). > > I would appreciate if you could look at it and sort it out. Also would > you be able to say how can we avoid this situation at all? > > > Thanks, > > Guy > > On Jun 30, 3:05 am, "Jeff S (Google)" wrote: >> Hi Thomas, >> >> I've rest this quota, apologies for the delay. >> >> Thank you, >> >> Jeff >> >> On Sun, Jun 28, 2009 at 8:03 PM, Thomas McKay -www.winebythebar.com< >> >> thomasfmc...@gmail.com> wrote: >> >> > Dashboard says over quota on indices again. No updates or vacuums >> > today. >> > App: winebythebar >> >> > On Jun 23, 12:46 pm, "Jeff S (Google)" wrote: >> > > Hi Thomas, >> > > I've reset the index count, apologies for the inconvenience. >> >> > > Happy coding, >> >> > > Jeff >> >> > > On Tue, Jun 23, 2009 at 9:12 AM, Thomas McKay -www.winebythebar.com< >> >> > > thomasfmc...@gmail.com> wrote: >> >> > > > My app is getting "Your application is exceeding a quota: Datastore >> > > > Indices Count" after a vacuum. Could I get a reset of indices count >> > > > please? >> > > > App: winebythebar >> >> > > > On May 23, 11:50 pm, fedestabile wrote: >> > > > > Thanks Jeff, it's working fine now :) >> >> > > > > Cheers, >> > > > > Fred >> >> > > > > On May 20, 5:48 am, "Jeff S (Google)" wrote: >> >> > > > > > Fred, I've reset the indexcountso you should be all set. Kaspars >> > > > > > I've reset the indexquotaforyourapp as well and it looks like you >> > > > > > had some indexes which were stuck in the building state so I've >> > moved >> > > > > > them to error. >> >> > > > > > Happy coding, >> >> > > > > > Jeff >> >> > > > > > On May 19, 2:11 am, "kaspars...@gmail.com" >> > > > > > wrote: >> >> > > > > > > I've only 8 indexes at the moment and I'm also getting "Your >> > > > > > >applicationisexceedingaquota:DatastoreIndicesCount". The >> > > > > > >applicationis basically stuck for couple days. >> >> > > > > > > Could you please reset thequota, app ID is lasi2. >> >> > > > > > > Thanks, >> > > > > > > Kaspars >> >> > > > -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty vs StringListProperty
On Tue, Aug 11, 2009 at 7:04 AM, Jeff Enderwick wrote:
>
> So, if I want StringListProperty, but I don't want/need the indexing,
> then I ought use ListProperty(db.Text)?
A better option would be to use StringListProperty(indexed=False).
-Nick Johnson
>
> Thanks again,
> Jeff
>
> On Mon, Aug 10, 2009 at 11:35 AM, Nick Johnson
> (Google) wrote:
>>
>> On Mon, Aug 10, 2009 at 7:25 PM, Jeff Enderwick
>> wrote:
>>>
>>> Thanks Nick. So in the case below, will the datastore then maintain an
>>> index on alist by default?
>>
>> Yes - str and unicode instances get indexed by default.
>>
>> -Nick
>>
>>>
>>> On Mon, Aug 10, 2009 at 3:35 AM, Nick Johnson
>>> (Google) wrote:
Hi Jeff,
A StringListProperty is exactly equivalent, as far as the datastore is
concerned, to multiple individual String properties. That is, an
entity like this:
MyEntity(astring="foo", alist=["foo", "bar"])
is stored in the datastore analagous to this:
(("astring", "foo"), ("alist", "foo"), ("alist", "bar"))
-Nick Johnson
On Sun, Aug 9, 2009 at 6:33 PM, Jeff Enderwick
wrote:
>
> Hey all, is there any datastop/API overhead difference between these
> two? Is there any difference in index (or lack thereof) maintenance by
> the datastore?
>
> Thanks,
> Jeff
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
>
>>>
>>> >
>>>
>>
>>
>>
>> --
>> Nick Johnson, Developer Programs Engineer, App Engine
>>
>> >
>>
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Regarding current google app engine customers
Hi Liang, For a few examples - by no means a comprehensive list - you can see the developers who attended the App Engine developer sandbox at the most recent Google I/O: http://code.google.com/events/io/sandbox.html -Nick Johnson On Tue, Aug 11, 2009 at 3:52 AM, Liang Han wrote: > > This is not a technical question, but I want to know if Google has a > list of current app engine big customers, which is not > confidential. In other words, what big customers are using google > app engine to build their websites? > > Thanks in advance. > > Liang Han > > > -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Get data based on "Key Name"
On Mon, Aug 10, 2009 at 6:09 PM, Ste wrote:
>
> Hi,
>
> I'm a total newbie to App Engine, and to web apps in general, with a
> background in front end web design.
>
> Anyway, I've worked through the (python) "Getting Started" guide, and
> it makes sense. Now I'm trying to change small bits and pieces to
> better my understanding.
>
> For instance, I'm now using a descriptive primary key for the
> datastore by providing a user-submitted "Key Name":
>
> greeting = Greeting(key_name="%s" % self.request.get('ID'))
>
> This works fine.
>
> However, I'm at a loss as to how to get a row back out of the
> datastore by using its Key Name. What variable should I include in my
> GQL, or am I missing the point?
When you know the key you want to retrieve a record by, you don't need
to use a query. Simply do MyModel.get_by_key_name(somename).
Alternately, you can construct Key objects using
Key.from_path("modelname", "keyname") and fetch them using db.get() or
MyModel.get().
-Nick Johnson
>
> Many thanks, and excuse my ignorance,
> Stephen
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Indexes stuck in building
On Mon, Aug 10, 2009 at 8:46 PM, Peecho wrote: > > Thanks, I could vacuum them now. > > However, my second attempt does not seem to do much either. I am a > Python newbie, so I fear I do something wrong... > For some reason, I see only 4 indexes 'building' in my console, while > my index.yaml actually describes 7 indexes. Does that mean some error > in the other three? Have you run 'update_indexes' or 'update' since you last vacuumed? > > This is my index.yaml (standard Bloog - I added only one line of code > to fix the search - see note below): > > indexes: > > - kind: Article > properties: > - name: __searchable_text_index > - name: published > direction: desc > > - kind: Article > properties: > - name: article_type > - name: published > direction: desc > > - kind: Article > properties: > - name: article_type > - name: title > > - kind: Article > properties: > - name: display_type > - name: published > direction: desc > > - kind: Article > properties: > - name: published > direction: desc > > - kind: Article > properties: > - name: tags > - name: published > direction: desc > > - kind: Comment > properties: > - name: article > - name: thread > > Note: I added the following line in def _ToPb (search.py) at line > number 270: > > prop.set_multiple(len(keywords) > 1) > > Does this compromise the indexes, you think? No, it doesn't. Have you checked if bloog has fixed this, however? You may not need to make the change yourself. -Nick Johnson > > Thanks again! > > Sander > On Aug 10, 2:59 pm, "Nick Johnson (Google)" > wrote: >> HiPeecho, >> >> I've moved your indexes to 'error'. Try vacuuming them now. >> >> -Nick Johnson >> >> On Sat, Aug 8, 2009 at 2:48 PM,Peecho wrote: >> >> > Hi, >> > My indexes seem stuck in Building for over 4 days now. I tried to >> > vacuum them but alas. Could anybody help me out, please? >> > My app's name is peechotest. >> >> > Thanks! >> >> -- >> Nick Johnson, Developer Programs Engineer, App Engine > > > -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: SEO and Google App Engine
Well for starters I'd welcome more than 100 composite indexes limit, a real paging support (next, previous, jump to a page no.), and rock solid indexes so that app customer data don't get mixed on an index accident. Then a big file service and freetext search indexing. But it's still the sexiest hosting platform on the net - pls let me know if you know a better one. On Aug 10, 9:28 pm, Holger wrote: > This flow chart page may be a nice > example:http://infosthetics.com/archives/2008/11/life_flow_chart_community_au... > > On the other hand it seems clear that Appengine still needs time and > massive investment by Google if Appengine shall become a kind of > blockbuster application earning money for Google. > > To become more popular it needs a better documentation understandable > not only for experts. > > To attract high traffic sites it needs far better datastore handling > with professional snapshot backups, high speed bulk data up/downloads > of the whole datastore (not only single kinds), single command high > speed deletion of whole kinds and some professional framework has to > be developed to an extend that it runs equally on Appengine as on Non- > Appengine infrastructure to guarantee the possibility not being stuck > to Google. > > On Aug 10, 8:39 pm, Barry Hunter wrote: > > > AppEngine itself is just hosting - so it doesnt really have much effect on > > SEO. > > > Dont forget that you can access a appengine app via a custom domain - > > so you might not even know a site is AppEngine. > > > As to indexing them - yes:http://www.google.com/search?q=site:appspot.com > > and a random examplehttp://www.google.com/search?q=life+flow+charts > > > 2009/8/10 student_thesis : > > > > Hello > > > > A big decision in moving our high volume website completely to google > > > app engine is to see, how does GAE handle SEO of a website. > > > > We dont want to lose on our SEO rankings with the move. > > > > I cant find one popular SEO website in google app engine. > > > > I tried typing a few GAE app names in google and the app does not come > > > up. > > > > So does google ever index GAE apps? > > > > We are willing to make an investment in GAE development, if someone > > > can shed light on the SEO issue. > > > > Thanks > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: GAE costs more than 10 times than expected.
Hi Sylvain, would it be possible for you to pay for App Engine hosting from a PayPal account (instead of the bank account), which you would be filling up monthly? Sorry if this is a stupid question, I haven't used App Engine billing yet. J. On Aug 10, 4:49 pm, Sylvain wrote: > Hi, > > Here is the background : I've a "small" app that needs more than 1Go, > currently : 1.27 GB. > So each week, I'm charged by GAE/Chekout : $0.08 (0.056€) > > But at the end I pay to my bank : $1.00 (0.71€). So, it is more than > 10 times than expected. > > I know about taxes (http://code.google.com/intl/fr/appengine/docs/ > billing.html#A_Note_About_Taxes), but here the gap between the "GAE > price" and the "Bank price" is too big : 92% is only taxes. > > I think billing needs to be changed/adapted asap in several ways to > reduce the taxes "part" : > > 1 - add new currencies (Euro, etc,...) > > 2 - do not charge weekly but monthly. About this, I think it is not > really logical to charge weekly. Most services charge weekly and for > example, if you use "Google Adsense" to "pay" you GAE site : Adsense > will pay monthly. Else add an option (weekly/monthly) > > 3 - Do not charge under a certain amount ($1 or more) > > 4 - Add a "credit system". For example, I credit my account with 50€, > then GAE/Chekout use this money > > I know that banks need money ;), but here, I think this is "very" bad > for users and GAE if it doesn't change and particularly for small > project from non US users. > > I want too give my money to Google, not to banks :) > > Regards. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: urllib2 and cookies in GAE environment
Hello,
I had the same problem. As I understand, in production urllib2 is
emulated on GAE through urlfetch and the cookie-related features don't
work. As a workaround you can set "Cookie" header explicitly. Here's
some code I used:
import Cookie
#get the cookie from the response:
cookie = Cookie.SimpleCookie(result.headers.get('set-cookie', ''))
#a routine to construct cookie header for requests:
def make_cookie_header(cookie):
ret = ""
for val in cookie.values():
ret+="%s=%s; "%(val.key, val.value)
return ret
# using the cookie in subsequent requests:
result = urlfetch.fetch(url=url,
headers={'Cookie' : make_cookie_header(cookie)})
Hope this helps.
Best Regards,
Alex
On Sun, Aug 9, 2009 at 9:09 AM, aig wrote:
>
> Prompt please, I wish to make automatic mail check through GAE,
> however the given script works on my computer but does not work in GAE
> enviroment (unable to login):
>
> #!/usr/bin/env python
>
> import cookielib
> import urllib
> import urllib2
>
> cj = cookielib.CookieJar()
> opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
> urllib2.install_opener(opener)
>
> user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
> headers = { 'User-Agent' : user_agent }
> values = {'Email' : 'lo...@gmail.com',
> 'Passwd' : 'password',
> 'PersistentCookie' : 'yes',
> 'null': 'Sign in',
> 'rm': 'false',
> 'hl': 'en',
> 'service': 'mail',
> 'continue': 'https://mail.google.com/mail/h/'}
>
> data = urllib.urlencode(values)
>
> url = 'https://www.google.com/accounts/ServiceLoginAuth'
> req = urllib2.Request(url, data, headers)
>
> handle = urllib2.urlopen(req)
>
> req = urllib2.Request('https://mail.google.com/mail/h/')
> handle = urllib2.urlopen(req)
>
> print 'Content-Type: text/plain'
> print ''
>
> print handle.read()
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: SEO and Google App Engine
> But it's still the sexiest hosting platform on the net - Sure the sexiest. That's why we're here and try to convince Google to add what lacks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How can I know if the entity is created or just updated in post call hook?
Hi Nick,
On Tue, Aug 11, 2009 at 7:50 PM, Nick Johnson
(Google) wrote:
>
> On Tue, Aug 11, 2009 at 7:29 AM, Takashi Matsuo
> wrote:
>>
>> Hi Nick,
>>
>> On Thu, Jul 30, 2009 at 4:48 PM, Nick Johnson
>> (Google) wrote:
>>> Hi Takashi,
>>>
>>> If the entity being put has neither key name nor id, a new id will be
>>> created for it when it's put, so it will always result in a new entity being
>>> inserted rather than an existing entity being replaced. Likewise, if you see
>>> an entity with an id being put, it is almost certainly an update to an
>>> existing entity (unless someone was messing around with the low-level API),
>>> or the entity was deleted and is now being reinserted.
>>>
>>> If the entity being put has a key name, the only way to tell for certain if
>>> it's replacing an existing entity is to do the put operation in a
>>> transaction, and do a get on the key first.
>>
>> Thank you for clarification. Here comes another question.
>> Is there any way to tell if particular PUT operation is in a
>> transaction or not, and
>> tell if the transaction is committed or rollbacked?
>>
>> I'd like to add a capability for adding arbitrary callback that will
>> be executed only when entities are actually put (that won't be
>> executed when the put operation is rollback-ed).
>
> You probably want to look into the datastore's support for hooks:
> http://code.google.com/appengine/articles/hooks.html
Thanks for your suggestion, but I've already read this article. In
case I've missed something, I read it again carefully, but what I want
to know is not written in the article.
I should have explain my question more precise and concretely.
OK, here is pseudo codes I'm working on.
-
def db_hook(service, call, request, response):
if call == 'Put':
from kay.utils.db_hook import execute_hooks
for key, entity in zip(response.key_list(), request.entity_list()):
kind = model_name_from_key(key)
execute_hooks(kind, key, entity)
apiproxy_stub_map.apiproxy.GetPostCallHooks().Append(
'db_hook', db_hook, 'datastore_v3')
--
When I tested with dev_appserver, this execute_hooks function was
invoked even if the transaction that contains this PUT operation, was
rollbacked.
I want to invoke execute_hooks function only when this PUT operation
is committed.
Is there any way to do that?
-- Takashi
> -Nick Johnson
>
>>
>> --- Takashi
>>
>>> -Nick Johnson
>>>
>>> On Thu, Jul 30, 2009 at 7:19 AM, Takashi Matsuo
>>> wrote:
Hi,
I'm writing a capability for registering db_hook dynamically. The code
snippet follows the mail body.
It works well except my code can not distinguish newly created entity
from updated entity.
I'd like to know how can I know if the entity is created or just
updated. What is the most appropriate way?
Regards,
--
from google.appengine.api import apiproxy_stub_map
from google.appengine.api import datastore
from google.appengine.ext import db
post_save_hooks = {}
def register_post_save_hook(func, model):
global post_save_hooks
kind = model.kind()
func_list = post_save_hooks.get(kind, None)
if func_list is None:
func_list = []
func_list.append(func)
post_save_hooks[kind] = func_list
def execute_hooks(kind, key, entity):
func_list = post_save_hooks.get(kind, None)
if func_list is not None:
entity.key_.CopyFrom(key)
e = datastore.Entity._FromPb(entity)
instance = db.class_for_kind(kind).from_entity(e)
for func in func_list:
func(instance)
def db_hook(service, call, request, response):
if call == 'Put':
from kay.utils.db_hook import execute_hooks
for key, entity in zip(response.key_list(), request.entity_list()):
kind = model_name_from_key(key)
execute_hooks(kind, key, entity)
apiproxy_stub_map.apiproxy.GetPostCallHooks().Append(
'db_hook', db_hook, 'datastore_v3')
--
--
Takashi Matsuo
>>>
>>>
>>>
>>> --
>>> Nick Johnson, Developer Programs Engineer, App Engine
>>>
>>> >
>>>
>>
>> >
>>
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google
[google-appengine] Re: Indexes stuck in building
Yes, I have run these commands, but the indexes are still stuck (again). Since the indexes are standard Bloog (and I don't see anybody else complaining), would it be worth a try to deploy the code into another app to see if it would work there? Or is there anything else I could do? Thanks for your help so far, anyway. By the way, the Bloog coders did not deploy the search fix into the code yet, but I did send Bill Katz a message. On Aug 11, 1:18 pm, "Nick Johnson (Google)" wrote: > On Mon, Aug 10, 2009 at 8:46 PM,Peecho wrote: > > > Thanks, I could vacuum them now. > > > However, my second attempt does not seem to do much either. I am a > > Python newbie, so I fear I do something wrong... > > For some reason, I see only 4 indexes 'building' in my console, while > > my index.yaml actually describes 7 indexes. Does that mean some error > > in the other three? > > Have you run 'update_indexes' or 'update' since you last vacuumed? > > > > > > > > > This is my index.yaml (standard Bloog - I added only one line of code > > to fix the search - see note below): > > > indexes: > > > - kind: Article > > properties: > > - name: __searchable_text_index > > - name: published > > direction: desc > > > - kind: Article > > properties: > > - name: article_type > > - name: published > > direction: desc > > > - kind: Article > > properties: > > - name: article_type > > - name: title > > > - kind: Article > > properties: > > - name: display_type > > - name: published > > direction: desc > > > - kind: Article > > properties: > > - name: published > > direction: desc > > > - kind: Article > > properties: > > - name: tags > > - name: published > > direction: desc > > > - kind: Comment > > properties: > > - name: article > > - name: thread > > > Note: I added the following line in def _ToPb (search.py) at line > > number 270: > > > prop.set_multiple(len(keywords) > 1) > > > Does this compromise the indexes, you think? > > No, it doesn't. Have you checked if bloog has fixed this, however? You > may not need to make the change yourself. > > -Nick Johnson > > > > > > > > > Thanks again! > > > Sander > > On Aug 10, 2:59 pm, "Nick Johnson (Google)" > > wrote: > >> HiPeecho, > > >> I've moved your indexes to 'error'. Try vacuuming them now. > > >> -Nick Johnson > > >> On Sat, Aug 8, 2009 at 2:48 PM,Peecho wrote: > > >> > Hi, > >> > My indexes seem stuck in Building for over 4 days now. I tried to > >> > vacuum them but alas. Could anybody help me out, please? > >> > My app's name is peechotest. > > >> > Thanks! > > >> -- > >> Nick Johnson, Developer Programs Engineer, App Engine > > -- > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Sharing an app between different Google apps domains
Hi Nick, Hooks are cool! Thanks for bringing my attention to them. Can you expand a bit on how one might handle a multi-account scenario using hooks? All I can imagine is a datastore hook that is smart enough to insert the proper filter on a query before it is executed -- is that what you had in mind? Ben On Aug 10, 1:30 am, "Nick Johnson (Google)" wrote: > On Mon, Aug 10, 2009 at 9:02 AM, Sudhir wrote: > > > Heres the background - I'm developing an app, lets say accounting > > software, that I'd like to sell to organizations using Google Apps. > > > How do I manage deployments? If I just create one app, it is possible > > for all my clients to import it into their dashboard and use it like > > their own app? > > Yes, though currently their users will have to have Google accounts > (you can create a Google account for any email address by going > tohttps://www.google.com/accounts/NewAccount) in order to sign in, since > you can only set your app to work with Google accounts or one specific > domain's accounts. Alternately, you can work around this limitation by > using OpenID, but this is rather cumbersome. > > > > > I don't want to create, maintain and update a separate app for each > > and every one of my customers... how can I share the app with my > > client domains keeping a separate datastore and auth setup for each of > > them? > > If you're using the approach outlined above, you'll need to enforce > separation in your datastore yourself - for example, by restricting > records based on the Host header the site is accessed through, and/or > the hostname of the user who's logged in. One way to do this is with > Hooks:http://code.google.com/appengine/articles/hooks.html > > -Nick Johnson > > > > > Sudhir > > -- > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Indexes stuck in building
I created an extra app, named peechobloog. The indexes get stuck in that one, too. However, I played around with the actual order of the indexes in the index.yaml file, and it seems that this index... - kind: Article properties: - name: published direction: desc ...is causing problems. It doesn't show up in the list of indexes, and everything after it is mentioned doesn't show up either. If I put the indexes below it on top of the list, they do show up - but not this one. Isn't that weird? It seems like the simplest index. On Aug 11, 4:28 pm, Peecho wrote: > Yes, I have run these commands, but the indexes are still stuck > (again). Since the indexes are standard Bloog (and I don't see anybody > else complaining), would it be worth a try to deploy the code into > another app to see if it would work there? Or is there anything else I > could do? > > Thanks for your help so far, anyway. > > By the way, the Bloog coders did not deploy the search fix into the > code yet, but I did send Bill Katz a message. > > On Aug 11, 1:18 pm, "Nick Johnson (Google)" > wrote: > > > > > On Mon, Aug 10, 2009 at 8:46 PM,Peecho wrote: > > > > Thanks, I could vacuum them now. > > > > However, my second attempt does not seem to do much either. I am a > > > Python newbie, so I fear I do something wrong... > > > For some reason, I see only 4 indexes 'building' in my console, while > > > my index.yaml actually describes 7 indexes. Does that mean some error > > > in the other three? > > > Have you run 'update_indexes' or 'update' since you last vacuumed? > > > > This is my index.yaml (standard Bloog - I added only one line of code > > > to fix the search - see note below): > > > > indexes: > > > > - kind: Article > > > properties: > > > - name: __searchable_text_index > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: article_type > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: article_type > > > - name: title > > > > - kind: Article > > > properties: > > > - name: display_type > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: tags > > > - name: published > > > direction: desc > > > > - kind: Comment > > > properties: > > > - name: article > > > - name: thread > > > > Note: I added the following line in def _ToPb (search.py) at line > > > number 270: > > > > prop.set_multiple(len(keywords) > 1) > > > > Does this compromise the indexes, you think? > > > No, it doesn't. Have you checked if bloog has fixed this, however? You > > may not need to make the change yourself. > > > -Nick Johnson > > > > Thanks again! > > > > Sander > > > On Aug 10, 2:59 pm, "Nick Johnson (Google)" > > > wrote: > > >> HiPeecho, > > > >> I've moved your indexes to 'error'. Try vacuuming them now. > > > >> -Nick Johnson > > > >> On Sat, Aug 8, 2009 at 2:48 PM,Peecho wrote: > > > >> > Hi, > > >> > My indexes seem stuck in Building for over 4 days now. I tried to > > >> > vacuum them but alas. Could anybody help me out, please? > > >> > My app's name is peechotest. > > > >> > Thanks! > > > >> -- > > >> Nick Johnson, Developer Programs Engineer, App Engine > > > -- > > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Sharing an app between different Google apps domains
On Tue, Aug 11, 2009 at 5:31 PM, Ben Nevile wrote: > > Hi Nick, > > Hooks are cool! Thanks for bringing my attention to them. > > Can you expand a bit on how one might handle a multi-account scenario > using hooks? All I can imagine is a datastore hook that is smart > enough to insert the proper filter on a query before it is executed -- > is that what you had in mind? Yes, that's about the size of it. On puts, you add the 'domain' property to all entities automatically, and on query, you automatically add the required filter. -Nick Johnson > > Ben > > > On Aug 10, 1:30 am, "Nick Johnson (Google)" > wrote: >> On Mon, Aug 10, 2009 at 9:02 AM, Sudhir wrote: >> >> > Heres the background - I'm developing an app, lets say accounting >> > software, that I'd like to sell to organizations using Google Apps. >> >> > How do I manage deployments? If I just create one app, it is possible >> > for all my clients to import it into their dashboard and use it like >> > their own app? >> >> Yes, though currently their users will have to have Google accounts >> (you can create a Google account for any email address by going >> tohttps://www.google.com/accounts/NewAccount) in order to sign in, since >> you can only set your app to work with Google accounts or one specific >> domain's accounts. Alternately, you can work around this limitation by >> using OpenID, but this is rather cumbersome. >> >> >> >> > I don't want to create, maintain and update a separate app for each >> > and every one of my customers... how can I share the app with my >> > client domains keeping a separate datastore and auth setup for each of >> > them? >> >> If you're using the approach outlined above, you'll need to enforce >> separation in your datastore yourself - for example, by restricting >> records based on the Host header the site is accessed through, and/or >> the hostname of the user who's logged in. One way to do this is with >> Hooks:http://code.google.com/appengine/articles/hooks.html >> >> -Nick Johnson >> >> >> >> > Sudhir >> >> -- >> Nick Johnson, Developer Programs Engineer, App Engine > > > -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Get data based on "Key Name"
I'm a newb myself, but I know one way. There is a method,
get_by_key_name(), where you get an entity by key_name like this:
greeting = Greeting.get_by_key_name(somekeyname)
If you give it one key_name string, it gives you one instance.
See: http://code.google.com/appengine/docs/python/datastore/modelclass.html
in the section: Model.get_by_key_name
Cheers!
UK
On Aug 10, 10:09 am, Ste wrote:
> Hi,
>
> I'm a total newbie to App Engine, and to web apps in general, with a
> background in front end web design.
>
> Anyway, I've worked through the (python) "Getting Started" guide, and
> it makes sense. Now I'm trying to change small bits and pieces to
> better my understanding.
>
> For instance, I'm now using a descriptive primary key for the
> datastore by providing a user-submitted "Key Name":
>
> greeting = Greeting(key_name="%s" % self.request.get('ID'))
>
> This works fine.
>
> However, I'm at a loss as to how to get a row back out of the
> datastore by using its Key Name. What variable should I include in my
> GQL, or am I missing the point?
>
> Many thanks, and excuse my ignorance,
> Stephen
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Building index stucked
My app id is vietnamix by the way. On 10 Tháng Tám, 22:04, "Nick Johnson (Google)" wrote: > Hi Bryan, > > I've reset your indexes again. > > We are working on improving theindexbuilding system to the point > where this will no longer be an issue. > > -Nick Johnson > > > > > > On Mon, Aug 10, 2009 at 3:29 PM, Bryan Pendleton wrote: > > It was cleared, but now I seem to have re-introduced the problem as a result > > of a poor re-shuffle of myindex.yaml file. > > Is there a reason that either: > > 1) More reporting than "Error" is not provided? > > 2) Users can't be offered a self-help un-stick, perhaps limited to trying > > 1x/hour or something? Or am I doing something so wrong that it should never > > happen and, thus, stick the indexes? > > -- > > Bryan A. P. Pendleton > > > On Mon, Aug 10, 2009 at 9:27 AM, Nick Johnson (Google) > > wrote: > > >> Hi Bryan, > > >> Please try vacuuming and rebuilding your indexes now. > > >> -Nick Johnson > > >> On Sat, Aug 8, 2009 at 6:29 PM, Bryan Pendleton > >> wrote: > >> > My indexes are stuck, on application "aquarium-bp". Oneindexis in > >> > error, > >> > the rest are "building". When I attempt to vacuum with a > >> > no-indexes-defined > >> >index.yaml file, I'm offered the opportunity to delete each, in turn. > >> > If I only choose "yes" for the Errorindex, I get the "Deleting selected > >> >indexdefinitions.", but theindexnever disappears (and remains > >> > "Error"). > >> > If I say "yes" to delete any other indexes, I'm given the somewhat more > >> > confusing "2009-08-08 13:20:20,689 WARNING appcfg.py:704 Anindexwas > >> > not > >> > deleted. Most likely this is because it no longer exists." message, and > >> > still nothing changes. > >> > I have a fairly small amount of data in my Datastore, but this broken > >> > application state is making it impossible for me to continue > >> > development/deployment of my app, which is now (further) behind > >> > schedule. > >> > Could someone from the AppEngine team take a look and maybe knock my > >> > indexes > >> > loose? > >> > -- > >> > Bryan A. P. Pendleton > >> > Graduate Student, Carnegie Mellon Human Computer Interaction Institute > >> > Ph: (877) geek-1-bp > > >> > On Thu, Jun 4, 2009 at 1:40 PM, Nick Johnson (Google) > >> > wrote: > > >> >> Hi, > > >> >> I've reset your indexes to 'error' state. You can now vacuum and > >> >> recreate > >> >> them if you wish. > > >> >> -Nick Johnson > > >> >> On Wed, Jun 3, 2009 at 10:07 PM, mmiy...@gmail.com > >> >> wrote: > > >> >>> Hi, > >> >>> I've got a problem while building a newindexfor my applicaiton. > >> >>>index.yaml is correctly uploaded but its status is stacked in > >> >>> "Building" for a week. Something must be wrong. > >> >>> I've tried vaccuming theindexby appcfg.py but failed... I need some > >> >>> help. My application ID is "codepasture-rest". > >> >>> Thanks. > > >> -- > >> Nick Johnson, Developer Programs Engineer, App Engine > > -- > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Building index stucked
Hi, My indexes (2 of 2) are unfortunately stuck on "Building" even though I have little data and have tried a few update and vacuum index commands. Could someone from the AppEngine team take a look and reset my indexes to "Error" ? Thanks heaps. On 10 Tháng Tám, 22:04, "Nick Johnson (Google)" wrote: > Hi Bryan, > > I've reset your indexes again. > > We are working on improving theindexbuilding system to the point > where this will no longer be an issue. > > -Nick Johnson > > > > > > On Mon, Aug 10, 2009 at 3:29 PM, Bryan Pendleton wrote: > > It was cleared, but now I seem to have re-introduced the problem as a result > > of a poor re-shuffle of myindex.yaml file. > > Is there a reason that either: > > 1) More reporting than "Error" is not provided? > > 2) Users can't be offered a self-help un-stick, perhaps limited to trying > > 1x/hour or something? Or am I doing something so wrong that it should never > > happen and, thus, stick the indexes? > > -- > > Bryan A. P. Pendleton > > > On Mon, Aug 10, 2009 at 9:27 AM, Nick Johnson (Google) > > wrote: > > >> Hi Bryan, > > >> Please try vacuuming and rebuilding your indexes now. > > >> -Nick Johnson > > >> On Sat, Aug 8, 2009 at 6:29 PM, Bryan Pendleton > >> wrote: > >> > My indexes are stuck, on application "aquarium-bp". Oneindexis in > >> > error, > >> > the rest are "building". When I attempt to vacuum with a > >> > no-indexes-defined > >> >index.yaml file, I'm offered the opportunity to delete each, in turn. > >> > If I only choose "yes" for the Errorindex, I get the "Deleting selected > >> >indexdefinitions.", but theindexnever disappears (and remains > >> > "Error"). > >> > If I say "yes" to delete any other indexes, I'm given the somewhat more > >> > confusing "2009-08-08 13:20:20,689 WARNING appcfg.py:704 Anindexwas > >> > not > >> > deleted. Most likely this is because it no longer exists." message, and > >> > still nothing changes. > >> > I have a fairly small amount of data in my Datastore, but this broken > >> > application state is making it impossible for me to continue > >> > development/deployment of my app, which is now (further) behind > >> > schedule. > >> > Could someone from the AppEngine team take a look and maybe knock my > >> > indexes > >> > loose? > >> > -- > >> > Bryan A. P. Pendleton > >> > Graduate Student, Carnegie Mellon Human Computer Interaction Institute > >> > Ph: (877) geek-1-bp > > >> > On Thu, Jun 4, 2009 at 1:40 PM, Nick Johnson (Google) > >> > wrote: > > >> >> Hi, > > >> >> I've reset your indexes to 'error' state. You can now vacuum and > >> >> recreate > >> >> them if you wish. > > >> >> -Nick Johnson > > >> >> On Wed, Jun 3, 2009 at 10:07 PM, mmiy...@gmail.com > >> >> wrote: > > >> >>> Hi, > >> >>> I've got a problem while building a newindexfor my applicaiton. > >> >>>index.yaml is correctly uploaded but its status is stacked in > >> >>> "Building" for a week. Something must be wrong. > >> >>> I've tried vaccuming theindexby appcfg.py but failed... I need some > >> >>> help. My application ID is "codepasture-rest". > >> >>> Thanks. > > >> -- > >> Nick Johnson, Developer Programs Engineer, App Engine > > -- > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Access from non-browser-based application?
Hello! the last days i read a lot about google app engine but I couldn't find the answere to one question. I searching a solution where I can access with a non-browser-based application the database (read/write). Is this possible with google app engine? I found this example in the Cookbock[1]. This seems like something I'm searchning for. But if i want to deploy the app to more users it would be bad to put my google auth-data into the app. The other users should only be able to access the application as they would use it in a browser but with a stand-alone client. So my question: Is it possible to create a google app which allows other users with a special usename/password to access the app from a non-browser-based application but with no further rights? Can you point me to some examples or documentation if this is possible and if it's not possible do you know a solution which would fit my needs? [1] http://appengine-cookbook.appspot.com/recipe/authenticate-from-non-browser-based-application/?id=ahJhcHBlbmdpbmUtY29va2Jvb2tyngELEgtSZWNpcGVJbmRleCJAYWhKaGNIQmxibWRwYm1VdFkyOXZhMkp2YjJ0eUdnc1NDRU5oZEdWbmIzSjVJZ3hWVWt4R1pYUmphQ0JCVUVrTQwLEgZSZWNpcGUiQWFoSmhjSEJsYm1kcGJtVXRZMjl2YTJKdmIydHlHZ3NTQ0VOaGRHVm5iM0o1SWd4VlVreEdaWFJqYUNCQlVFa00yDA Thanks a lot! Markus --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Write-behind cache demo from Brett Slatkin
Here's all of the code from the presentation:
http://code.google.com/events/io/sessions/OfflineProcessingAppEngine.html
class Counter(db.Model):
count = db.IntegerProperty(indexed=False)
class CounterHandler(webapp.RequestHandler):
def post(self):
key = self.request.get('key')
if (memcache.incr(key) is None and
not memcache.add(key, 1)):
memcache.incr(key)
if memcache.add(key + '_dirty', 1):
taskqueue.add(url='/worker',
params={'key': key})
class PageHitWorker(webapp.RequestHandler):
def post(self):
key = self.request.get('key')
memcache.delete(key + '_dirty'):
value = memcache.get(key)
if value is None:
logging.error('Failure for %s', key)
return
Counter(key_name=key, count=value).put()
One thing is that is bothering me is the ':' after memcache.delete in
PageHitWorker. I wonder if an 'if' statement is missing to only update
the counter entity when in fact was dirty. Most likely it's just an
extra character because nothing is indented and the main issue is to
make sure you don't spin more than one task at a time. Moving on.
Brett talks about this solution when you're willing to sacrifice
"near" real-time resolution down to whatever the queue configuration
might be. But you're also are sacrificing the possibility of data loss
because this method assumes that you'll never lose that key from
memcache.
I've begun writing another example that increments the entity counter
instead of replacing it. This way, at most, I'll only lose the current
value of my memcache key/counter. For that, I now have to wrap the
worker in a transaction and also memcache.decr the counter by the
amount added to the entity counter. But I wanted to hear what others
are doing now that sharded counters has been deemed too expensive.
I'm trying to keep several counters per-entity so I'm thinking this
whole thing can get out of hand too quickly. I was then thinking of an
alternative approach using memcache. What if I keep a counter in
memcache as a log index (using memcache.incr) and insert my requests
as name_{index_from_memcache} then simply have the task queue read
from memcache in batches and do all of the processing and counting
within a single transaction/entity.
Any thoughts?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Indexes stuck in building
Hi Nick I also have 3 indexes in my app (alin) which are stuck since two days in the building state. Could you please help me set them to the error state so I can perform a vacuum. Is there actually a way to set the stuck indexes to the error state myself so I and maybe others dont have to harass :) you about this anymore? Already tried running vacuum und update_indexes and then removed all my application files but didn't help. Thanks Endri On 11 Aug., 16:28, Peecho wrote: > Yes, I have run these commands, but the indexes are stillstuck > (again). Since the indexes are standard Bloog (and I don't see anybody > else complaining), would it be worth a try to deploy the code into > another app to see if it would work there? Or is there anything else I > could do? > > Thanks for your help so far, anyway. > > By the way, the Bloog coders did not deploy the search fix into the > code yet, but I did send Bill Katz a message. > > On Aug 11, 1:18 pm, "Nick Johnson (Google)" > wrote: > > > On Mon, Aug 10, 2009 at 8:46 PM,Peecho wrote: > > > > Thanks, I could vacuum them now. > > > > However, my second attempt does not seem to do much either. I am a > > > Python newbie, so I fear I do something wrong... > > > For some reason, I see only 4 indexes 'building' in my console, while > > > myindex.yaml actually describes 7 indexes. Does that mean some error > > > in the other three? > > > Have you run 'update_indexes' or 'update' since you last vacuumed? > > > > This is myindex.yaml (standard Bloog - I added only one line of code > > > to fix the search - see note below): > > > > indexes: > > > > - kind: Article > > > properties: > > > - name: __searchable_text_index > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: article_type > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: article_type > > > - name: title > > > > - kind: Article > > > properties: > > > - name: display_type > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: published > > > direction: desc > > > > - kind: Article > > > properties: > > > - name: tags > > > - name: published > > > direction: desc > > > > - kind: Comment > > > properties: > > > - name: article > > > - name: thread > > > > Note: I added the following line in def _ToPb (search.py) at line > > > number 270: > > > > prop.set_multiple(len(keywords) > 1) > > > > Does this compromise the indexes, you think? > > > No, it doesn't. Have you checked if bloog has fixed this, however? You > > may not need to make the change yourself. > > > -Nick Johnson > > > > Thanks again! > > > > Sander > > > On Aug 10, 2:59 pm, "Nick Johnson (Google)" > > > wrote: > > >> HiPeecho, > > > >> I've moved your indexes to 'error'. Try vacuuming them now. > > > >> -Nick Johnson > > > >> On Sat, Aug 8, 2009 at 2:48 PM,Peecho wrote: > > > >> > Hi, > > >> > My indexes seemstuckinBuildingfor over 4 days now. I tried to > > >> > vacuum them but alas. Could anybody help me out, please? > > >> > My app's name is peechotest. > > > >> > Thanks! > > > >> -- > > >> Nick Johnson, Developer Programs Engineer, App Engine > > > -- > > Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] How to integrate Payment module with google app engine
Hi, I am working on a project which needed payment so that anybody can use his/her credit card to purchase my service, As I am new to Google App Engine, I need your help for this part, if somebody can suggest the text related to it that would be appreciable.thanks. -- Rahul --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Number of Entities in DataStore
Hi All, I'm really new to python and databases period, so please take it easy on me, im trying to learn. im trying to roll my own energy monitor and i cant really figure out how to get the number of entities in my data store. class PowerData(db.Model): date = db.DateTimeProperty(auto_now_add=True) # timestamp kw = db.FloatProperty() # current kilowatt data (0.01 scale) my database will have more than 1000 entires in it quickly. i want to grab 1000 data points and send it to the google annotated timeline display, but i know my query will be limited to 1000 results. I want to write a query that will return the number of entities in my datastore, then write a function that decimates (get every Nth one) the data based on a hop factor that will give me less than 100 data points. 1- i have no idea how to get the full size of the datastore if its greater than 1000 entities 2- i dont know how to grab the decimated data in a way that wont be computationaly expensive im not looking for handout code, just throw me a bone for something to read up on? is this a decent way to structure the data base? should i change it before i got further? thanks, adam --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to integrate Payment module with google app engine
Your basic choices are: 1) Google Checkout 2) Paypal 3) Get your own merchant account with your bank and then find a company that does online processing such as Authorize.Net or PayFloPro (from Paypal). Usually you will get Visa/MC with your bank, then have to apply with Amex/Discover separately. 4) 2CheckOut.com I just did an implementation of Paypal, it probably took 40-60 hours all together, just because of the learning curve. We are also selling subscriptions, and someone told me that Google Checkout recently added this feature. Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Number of Entities in DataStore
Google Data Store doesn't offer a record count of all rows of a "Kind" on BigTable; the typical recommendation is to keep your own counter up- to-date each time you store a record (assuming you really need it). If you are worried about concurrency when doing this, there is a "sharding" technique here: http://code.google.com/appengine/articles/sharding_counters.html Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Access from non-browser-based application?
This maybe what you are asking about? http://code.google.com/appengine/articles/remote_api.html Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: PDF Generation
I have been using ReportLab. Apart from the inability to embed pictures I have been pretty happy with it. Best Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty vs StringListProperty
Thanks again Nick. Sorry to be a PITA. We've been told that
TextProperty is effectively the same as StringProperty w/out the
indexing (although the online doc indicates some max length
differences).
What are the differences between ListProperty(db.Text) and
StringListProperty(indexed=False), and what makes the latter better?
On Tue, Aug 11, 2009 at 3:55 AM, Nick Johnson
(Google) wrote:
>
> On Tue, Aug 11, 2009 at 7:04 AM, Jeff Enderwick
> wrote:
>>
>> So, if I want StringListProperty, but I don't want/need the indexing,
>> then I ought use ListProperty(db.Text)?
>
> A better option would be to use StringListProperty(indexed=False).
>
> -Nick Johnson
>
>>
>> Thanks again,
>> Jeff
>>
>> On Mon, Aug 10, 2009 at 11:35 AM, Nick Johnson
>> (Google) wrote:
>>>
>>> On Mon, Aug 10, 2009 at 7:25 PM, Jeff Enderwick
>>> wrote:
Thanks Nick. So in the case below, will the datastore then maintain an
index on alist by default?
>>>
>>> Yes - str and unicode instances get indexed by default.
>>>
>>> -Nick
>>>
On Mon, Aug 10, 2009 at 3:35 AM, Nick Johnson
(Google) wrote:
>
> Hi Jeff,
>
> A StringListProperty is exactly equivalent, as far as the datastore is
> concerned, to multiple individual String properties. That is, an
> entity like this:
>
> MyEntity(astring="foo", alist=["foo", "bar"])
>
> is stored in the datastore analagous to this:
>
> (("astring", "foo"), ("alist", "foo"), ("alist", "bar"))
>
> -Nick Johnson
>
> On Sun, Aug 9, 2009 at 6:33 PM, Jeff Enderwick
> wrote:
>>
>> Hey all, is there any datastop/API overhead difference between these
>> two? Is there any difference in index (or lack thereof) maintenance by
>> the datastore?
>>
>> Thanks,
>> Jeff
>>
>> >
>>
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
>
> >
>
>
>>>
>>>
>>>
>>> --
>>> Nick Johnson, Developer Programs Engineer, App Engine
>>>
>>> >
>>>
>>
>> >
>>
>
>
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] user_id() always None
I am trying to store some users in my app, but when I create a User
instance using a valid Google account e-mail on my server, the object
that is returned as no user_id (None is returned when I apply the
user_id()-method). This is what I do, right before I want to store the
user:
user = users.User('someth...@gmail.com')
logging.info('Adding: ' + user.email() + ' with id ' + user.user_id())
Everytime it fails in the second line, saying that user.user_id() is
None. This is the case on both my dev. server and on the Google-hosted
app. What am I doing wrong?
In the documentation it is stated that "When running under the
development web server, all User objects are assumed to represent
valid Google accounts when stored in the (simulated) datastore. The
User object for a valid user can provide a unique ID value for the
user that stays the same even if the user changes her email address.
The user_id() method returns this ID, a str value." So a valid user
means one with an ID, and as all users are valid on the dev. server,
all should return an ID, right?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty vs StringListProperty
On Tue, Aug 11, 2009 at 7:46 PM, Jeff Enderwick wrote:
>
> Thanks again Nick. Sorry to be a PITA. We've been told that
> TextProperty is effectively the same as StringProperty w/out the
> indexing (although the online doc indicates some max length
> differences).
That's correct. The length limitation on StringProperty is due to the indexing.
>
> What are the differences between ListProperty(db.Text) and
> StringListProperty(indexed=False), and what makes the latter better?
A ListProperty requires the elements to be the exact type specified,
so you'd have to explicitly cast all your items to db.Text. A
StringListProperty, however, will happily accept str, unicode, and (I
think) db.Text instances.
-Nick Johnson
>
> On Tue, Aug 11, 2009 at 3:55 AM, Nick Johnson
> (Google) wrote:
>>
>> On Tue, Aug 11, 2009 at 7:04 AM, Jeff Enderwick
>> wrote:
>>>
>>> So, if I want StringListProperty, but I don't want/need the indexing,
>>> then I ought use ListProperty(db.Text)?
>>
>> A better option would be to use StringListProperty(indexed=False).
>>
>> -Nick Johnson
>>
>>>
>>> Thanks again,
>>> Jeff
>>>
>>> On Mon, Aug 10, 2009 at 11:35 AM, Nick Johnson
>>> (Google) wrote:
On Mon, Aug 10, 2009 at 7:25 PM, Jeff Enderwick
wrote:
>
> Thanks Nick. So in the case below, will the datastore then maintain an
> index on alist by default?
Yes - str and unicode instances get indexed by default.
-Nick
>
> On Mon, Aug 10, 2009 at 3:35 AM, Nick Johnson
> (Google) wrote:
>>
>> Hi Jeff,
>>
>> A StringListProperty is exactly equivalent, as far as the datastore is
>> concerned, to multiple individual String properties. That is, an
>> entity like this:
>>
>> MyEntity(astring="foo", alist=["foo", "bar"])
>>
>> is stored in the datastore analagous to this:
>>
>> (("astring", "foo"), ("alist", "foo"), ("alist", "bar"))
>>
>> -Nick Johnson
>>
>> On Sun, Aug 9, 2009 at 6:33 PM, Jeff Enderwick
>> wrote:
>>>
>>> Hey all, is there any datastop/API overhead difference between these
>>> two? Is there any difference in index (or lack thereof) maintenance by
>>> the datastore?
>>>
>>> Thanks,
>>> Jeff
>>>
>>> >
>>>
>>
>>
>>
>> --
>> Nick Johnson, Developer Programs Engineer, App Engine
>>
>> >
>>
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
>
>>>
>>> >
>>>
>>
>>
>>
>> --
>> Nick Johnson, Developer Programs Engineer, App Engine
>>
>> >
>>
>
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty vs StringListProperty
On Aug 11, 10:57 pm, "Nick Johnson (Google)" wrote: > That's correct. The length limitation on StringProperty is due to the > indexing. Does this mean that a StringProperty(indexed=False) can have unlimited length? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: java.rmi.server.UID error
GAE uses a white-list of classes allowed for you to use. this jva.rmi.server.UID may be in the black-list, because it says: "java.lang.NoClassDefFoundError: java.rmi.server.UID *is a restricted **class.* " you can find the white-list classes here: http://code.google.com/appengine/docs/java/jrewhitelist.html Att Bruno Bilescky Wants to learn GWT? Read my blog / Quer aprender a programar? leia meu blog http://tcninja.blogspot.com On Mon, Aug 10, 2009 at 6:18 PM, Icarus wrote: > java.lang.NoClassDefFoundError: java.rmi.server.UID is a restricted > class. > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ListProperty vs StringListProperty
On Tue, Aug 11, 2009 at 9:38 PM, Peter Petrov wrote: > > On Aug 11, 10:57 pm, "Nick Johnson (Google)" > wrote: >> That's correct. The length limitation on StringProperty is due to the >> indexing. > > Does this mean that a StringProperty(indexed=False) can have unlimited > length? No - the length limitation is applied to unindexed strings, too. -Nick > > > -- Nick Johnson, Developer Programs Engineer, App Engine --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: user_id() always None
A user object needs to be stored in the datastore before it gets its id
populated.
On Tue, Aug 11, 2009 at 9:28 PM, epb wrote:
>
> I am trying to store some users in my app, but when I create a User
> instance using a valid Google account e-mail on my server, the object
> that is returned as no user_id (None is returned when I apply the
> user_id()-method). This is what I do, right before I want to store the
> user:
>
> user = users.User('someth...@gmail.com')
> logging.info('Adding: ' + user.email() + ' with id ' + user.user_id())
>
> Everytime it fails in the second line, saying that user.user_id() is
> None. This is the case on both my dev. server and on the Google-hosted
> app. What am I doing wrong?
>
> In the documentation it is stated that "When running under the
> development web server, all User objects are assumed to represent
> valid Google accounts when stored in the (simulated) datastore. The
> User object for a valid user can provide a unique ID value for the
> user that stays the same even if the user changes her email address.
> The user_id() method returns this ID, a str value." So a valid user
> means one with an ID, and as all users are valid on the dev. server,
> all should return an ID, right?
> >
>
--
Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: user_id() always None
Ok, thanks. So there is no way to check if the user is a valid Google
account before I store it?
On Aug 11, 5:10 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος)
wrote:
> A user object needs to be stored in the datastore before it gets its id
> populated.
>
>
>
> On Tue, Aug 11, 2009 at 9:28 PM, epb wrote:
>
> > I am trying to store some users in my app, but when I create a User
> > instance using a valid Google account e-mail on my server, the object
> > that is returned as no user_id (None is returned when I apply the
> > user_id()-method). This is what I do, right before I want to store the
> > user:
>
> > user = users.User('someth...@gmail.com')
> > logging.info('Adding: ' + user.email() + ' with id ' + user.user_id())
>
> > Everytime it fails in the second line, saying that user.user_id() is
> > None. This is the case on both my dev. server and on the Google-hosted
> > app. What am I doing wrong?
>
> > In the documentation it is stated that "When running under the
> > development web server, all User objects are assumed to represent
> > valid Google accounts when stored in the (simulated) datastore. The
> > User object for a valid user can provide a unique ID value for the
> > user that stays the same even if the user changes her email address.
> > The user_id() method returns this ID, a str value." So a valid user
> > means one with an ID, and as all users are valid on the dev. server,
> > all should return an ID, right?
>
> --
>
> Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Rest server: user-group access
Hi Jeff, I am no longer using the rest server interface. Instead I have made my own custom interface that handles http get/post. In addition to creating this interface, I followed one of the approaches you described and added a list of users to the entity that I am sharing. This works fine, as the sharing is on a per-user basis. Thanks! epb On Aug 10, 6:31 pm, "Jeff S (Google)" wrote: > Yes, what you are describing sounds like Access Control Lists (ACLs) and it > is possible to check within your code to see if the current user has > necessary permissions before carrying out an action. I'm not familiar with > the rest server module that you are using but there might be something in it > already which provides ACLs. If not, you can roll your own by adding a list > (or lists) of authorized users to an entity, or you could create groupings > for users (roles) and each entity could have a list of authorized roles. > There are quite a few design possibilities, some designs call require more > complex ACL implementations than others. Is sharing mostly on a per user > basis or are there identifiable types of users (admins, editors, readers, > etc.)? > > Thank you, > > Jeff > > On Thu, Aug 6, 2009 at 8:39 PM, epb wrote: > > > Hi > > > I am using the app engine rest server module to access different > > instances of a datamodel on my GAE app. Login is required on the app > > but I would like the system to use instance-wise authentication so > > that only a specific set of users A (and not all users) has access to > > a specific datamodel instance B via the rest server interface. Fx. the > > datamodel instance with key 1234 can only be accessed by a specfic set > > of users viahttp://myapp.com/rest/DATAMODELNAME/1234, both with HTTP > > GET and POST/PUT. Can this be done? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: user_id() always None
Not that I know of no.
2009/8/11 epb
>
> Ok, thanks. So there is no way to check if the user is a valid Google
> account before I store it?
>
> On Aug 11, 5:10 pm, Alkis Evlogimenos ('Αλκης Ευλογημένος)
> wrote:
> > A user object needs to be stored in the datastore before it gets its id
> > populated.
> >
> >
> >
> > On Tue, Aug 11, 2009 at 9:28 PM, epb wrote:
> >
> > > I am trying to store some users in my app, but when I create a User
> > > instance using a valid Google account e-mail on my server, the object
> > > that is returned as no user_id (None is returned when I apply the
> > > user_id()-method). This is what I do, right before I want to store the
> > > user:
> >
> > > user = users.User('someth...@gmail.com')
> > > logging.info('Adding: ' + user.email() + ' with id ' + user.user_id())
> >
> > > Everytime it fails in the second line, saying that user.user_id() is
> > > None. This is the case on both my dev. server and on the Google-hosted
> > > app. What am I doing wrong?
> >
> > > In the documentation it is stated that "When running under the
> > > development web server, all User objects are assumed to represent
> > > valid Google accounts when stored in the (simulated) datastore. The
> > > User object for a valid user can provide a unique ID value for the
> > > user that stays the same even if the user changes her email address.
> > > The user_id() method returns this ID, a str value." So a valid user
> > > means one with an ID, and as all users are valid on the dev. server,
> > > all should return an ID, right?
> >
> > --
> >
> > Alkis
> >
>
--
Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Fetching gzipped resource, but comes out unzipped.
I'm trying to load a remote gzip file onto the Python environment, but
the downloaded file is already unzipped. Hence it gets chopped (or get
a ResponseTooLargeError).
Here is my code.
from google.appengine.ext import webapp
import logging
class Test(webapp.RequestHandler):
def get(self):
import urllib2, gzip, StringIO
req = urllib2.Request('http://atsushi-takayama.com/siteinfo/
wedataAutoPagerize.json.gz')
req.add_header('Accept', 'application/x-gzip')
response = urllib2.urlopen(req)
raw_data = response.read()
logging.debug('data length :' + str(len(raw_data)))
stream = StringIO.StringIO(raw_data)
decompressor = gzip.GzipFile(fileobj=stream)
json = decompressor.read()
logging.debug('json length :' + str(len(json)))
application = webapp.WSGIApplication([(r'/siteinfo/test',
Test)],debug=False)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
This is the error I get.
DEBUG:root:Could not import "icglue": Disallowed C-extension or built-
in module
WARNING:root:Stripped prohibited headers from URLFetch request:
['Host']
DEBUG:root:Making HTTP request: host = atsushi-takayama.com, url =
http://atsushi-takayama.com/siteinfo/wedataAutoPagerize.json.gz,
payload = None, headers = {'Accept-Encoding': 'gzip', 'Connection':
'close', 'Accept': 'application/x-gzip', 'User-Agent': 'Python-urllib/
2.5 AppEngine-Google; (+http://code.google.com/appengine)', 'Host':
'atsushi-takayama.com', 'Referer': 'http://localhost/'}
DEBUG:root:data length :1062763
ERROR:root:Not a gzipped file
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/ext/webapp/__init__.py", line 507, in __call__
handler.get(*groups)
File "/Users/atsushi/Dropbox/Programming/gae/Atsushuu/siteinfo.py",
line 279, in get
json = decompressor.read()
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/gzip.py", line 220, in read
self._read(readsize)
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/gzip.py", line 263, in _read
self._read_gzip_header()
File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/
python2.5/gzip.py", line 164, in _read_gzip_header
raise IOError, 'Not a gzipped file'
IOError: Not a gzipped file
The data length 1062763 is the file size of the unzipped file.
In the remote environment, I get this.
D
08-11 02:03PM 40.316
data length :1048576
E
08-11 02:03PM 40.320
Not a gzipped file
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.py", line 507, in __call__
handler.get(*groups)
File "/base/data/home/apps/atsushuu/1.335550907989224563/
siteinfo.py", line 279, in get
json = decompressor.read()
File "/base/python_dist/lib/python2.5/gzip.py", line 220, in read
self._read(readsize)
File "/base/python_dist/lib/python2.5/gzip.py", line 263, in _read
self._read_gzip_header()
File "/base/python_dist/lib/python2.5/gzip.py", line 164, in
_read_gzip_header
raise IOError, 'Not a gzipped file'
IOError: Not a gzipped file
It doesn't matter if I do req.add_header('Accept', 'application/x-
gzip'), req.add_header('Accept-encoding', 'gzip') or don't set
anything extra at all. (The request is already gzip acceptable)
Notice now the data length is 1048576 (=1MB). If I use urlfetch
instead of urllib2, the local error message is the same, but the
remote server returns ResponseTooLargeError. This is exactly what
happens when I try to download a unzipped file.
Does anyone know how to download a gzip file and expand at the App
Engine environment?
I'm using App Engine SDK 1.2.4, Mac OS X 10.5.7, Python 2.5.1
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Group search problem
Hi jd, You might have better luck using the search box on http://code.google.com/appengine/. This uses a Custom Search Engine which has a refinement tab for the discussion groups, for example I searched for transcript refined to the discussion groups and got http://www.google.com/custom?client=google-coop-np&boostcse=0&q=transcript+more:discussion_groups&cref=http://code.google.com/cse/googlecode-context.xml&sa=N&ei=3BGCSqqRJ42YsgPu05Ro&oi=coopctx&resnum=0&ct=col2&cd=1 Cheers, Jeff On Mon, Aug 10, 2009 at 6:39 AM, jd wrote: > > Hi, > > When I "search this group" for "transcript" there are no results. But > clearly there are results e.g. > > > http://groups.google.com/group/google-appengine/browse_thread/thread/ae99786ea4baee2/0c6c2fab728da13d > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: "You do not have the permissions necessary to install this application."
Hi Brad, Are you signed in as the admin for the Google Apps domain? I think only a domain admin can add App Engine apps to a Google Apps domain. Thank you, Jeff On Mon, Aug 10, 2009 at 7:27 AM, Brad Allison wrote: > > I am getting the above message when I try to add my app to Google > Apps. I've tried using my Google account and using my Google Apps > account to add the app but I still get the same message. Has anyone > seen this before? I've tried searching for the error message but > didn't come up with much. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] api_cpu_ms (need to be sure I'm not being stupid)
long story short, I had a URL that was running 900/1500ms (app/api) CPU, so I had to dig into it. I was surprised by just how much CPU time some simple things are taking (or how much I think they're taking ;-). For example, with this class: class AssocObject(db.Model): urlPath = db.TextProperty() fileName = db.TextProperty() contentType = db.TextProperty() blob = db.BlobProperty() # Unused in query history -- copied from input. - kind: AssocObject ancestor: yes and the following snippet: fileObj = AssocObject(parent = node) db.put(fileObj) Just the single db.put() costs 250/210ms of CPU. Getting rid of the 'parent =' saves on 10s of ms. The total data set size is in the 10s of objects. I am not setting any of the variables in the class instance; the code really is create obj and store it at this point. Am I missing something? Is this level of CPU burn to be expected? Thanks, Jeff --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
