[google-appengine] Re: updating schema
Djidjadji, Thanks very much for the reply. I ended up needing to use both strftime() and strptime() as you can see in my updated code. Another wrinkle I had to overcome was I had to reverse the sort order and the comparison inequality to pass through *all* the instances; without such a change, I only altered half of them. In the code below I left in the original sort order and comparison lines, but commented them out, so it would be clearer what was required. http://dpaste.com/18328/ Wouldn't my code be good to show others as a code recipe or something, because I bet dates are more unique then names? Even if you agree, I don't know how to publish such items. On Mar 23, 8:50 pm, djidjadji wrote: > You must convert the text version of the date back to a > datetime.datetime object. > > Use the datetime.datetime.strptime(date_string, format) method > > 2009/3/24 thebrianschott : > > > [snip] > > > next_url = '/update_datastore?date=%s' % urllib.quote(next_date) > > res = map(safe_map.__getitem__, s) > > TypeError: argument 2 to map() must support iteration > > >http://dpaste.com/18238/ > > > Can somebody show me what is wrong with my code, please? > > > Brian in Atlanta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: @login_required usage
For a normal dictionary you use the following to delete a key-value pair
del request.session['myid']
2009/3/24 arnie :
>
> For a logged in user I have created a session as below
> request.session['myid'] = somevalue
> This will create an entry in Session datastore table.
> When the user logout I need to clear this session key
> how can I do this?
> request.session.delete[request.session.get('myid')] does not work
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: updating schema
You can always use the __key__ attribute instead of the date. Look at [2] section "Paging without a property" You can make an entry in the GAE cookbook [1]. [1] http://appengine-cookbook.appspot.com/ [2] http://code.google.com/appengine/articles/paging.html 2009/3/24 thebrianschott : > > Djidjadji, > > Thanks very much for the reply. I ended up needing to use > both strftime() and strptime() as you can see in my updated > code. Another wrinkle I had to overcome was I had to reverse > the sort order and the comparison inequality to pass through > *all* the instances; without such a change, I only altered half > of them. > > In the code below I left in the original sort order and > comparison lines, but commented them out, so it would be > clearer what was required. > > http://dpaste.com/18328/ > > Wouldn't my code be good to show others as a code recipe > or something, because I bet dates are more unique then > names? Even if you agree, I don't know how to publish such items. > > On Mar 23, 8:50 pm, djidjadji wrote: >> You must convert the text version of the date back to a >> datetime.datetime object. >> >> Use the datetime.datetime.strptime(date_string, format) method >> >> 2009/3/24 thebrianschott : >> >> > [snip] >> >> > next_url = '/update_datastore?date=%s' % urllib.quote(next_date) >> > res = map(safe_map.__getitem__, s) >> > TypeError: argument 2 to map() must support iteration >> >> >http://dpaste.com/18238/ >> >> > Can somebody show me what is wrong with my code, please? >> >> > Brian in Atlanta --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Trouble importing PIL module
Hello,
I'd like to use the PIL module. I know it's not supported, so I
copied the PIL source files into a directory in my app's source
directory.
At this point, "import Image, ImageDraw" didn't work.
I thought it might be a package name conflict and found this other
thread:
http://groups.google.com/group/google-appengine/browse_thread/thread/db910ed40ae495fc/f41873a476db744e?lnk=gst&q=adding+Python+modules#f41873a476db744e
Then I tried doing as suggested in that posting. I created a file
called import_fixer.py with those contents mentioned. I removed
"import Image, ImageDraw" and then added the following to my
application file:
import import_fixer
import_fixer.FixImports('Image')
import_fixer.FixImports('ImageDraw')
Now I am getting the error:
: No module named Image
args = ('No module named Image',)
message = 'No module named Image'
Any advice? Thanks in advance,
Audrey
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: hard to fetch a single entry
I found there cannot be too much delete operation to a datastore. it will take a whole day for server to "consume" the changes. there must be some hidden quota. On Mar 23, 4:38 pm, djidjadji wrote: > It probably is a Datastore timeout. If you retry it mostly works. > > 2009/3/23 neoedmund : > > > > > why this happens? > > > a model has 100,000+ entries in it. > > q=Foo.all() > > e=q.get() > > > error occurs: > > > File "/base/python_lib/versions/1/google/appengine/ext/db/ > > __init__.py", line 1346, in get > > results = self.fetch(1) > > File "/base/python_lib/versions/1/google/appengine/ext/db/ > > __init__.py", line 1390, in fetch > > raw = self._get_query().Get(limit, offset) > > File "/base/python_lib/versions/1/google/appengine/api/datastore.py" > > , line 942, in Get > > return self._Run(limit, offset)._Next(limit) > > File "/base/python_lib/versions/1/google/appengine/api/datastore.py" > > , line 886, in _Run > > _ToDatastoreError(err) > > File "/base/python_lib/versions/1/google/appengine/api/datastore.py" > > , line 1965, in _ToDatastoreError > > raise errors[err.application_error](err.error_detail) > > Timeout --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Really weird performance issues
Hey all,
I have two really weird performance issues that I can't seem to fix.
- With our rest api the first request is always slow (~1500ms) and all
subsequent requests are way faster (~200ms). They are the exact same
requests just with different data. When I stop making requests for
like 10 secs the first one is slow again. In a request 2 memcache
objects (<2kb) are fetched and returned.
- We have a django app that renders pages, and 80% of the time
rendering of the page takes ~2000ms. It fetches a list of objects and
it's template from memcache and outputs the page. Nothing fancy. What
I don't get is that when I profile the page, the profiles states I
only use 200ms
03-24 02:38AM 16.853 / 200 801ms 1711ms-cpu 1kb Mozilla/5.0
(Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/528.16
(KHTML, like Gecko) Version/4.0 Safari/528.16,gzip(gfe)
80.56.5.134 - - [24/Mar/2009:02:38:17 -0700] "GET HTTP/1.1" 200
1863 "" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us)
AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16,gzip
(gfe)"
Profile data:
45653 function calls (44818 primitive calls) in 0.168 CPU
seconds
Ordered by: internal time
List reduced from 661 to 20 due to restriction <20>
ncalls tottime percall cumtime percall filename:lineno
(function)
70.0430.0060.0430.006
{google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait}
36/350.0120.0000.0290.001 {__import__}
50.0080.0020.0540.011 /base/python_dist/lib/
python2.5/pickle.py:839(load)
86620.0060.0000.0060.000 {method 'read' of
'cStringIO.StringO' objects}
6820.0040.0000.0070.000 /base/python_dist/lib/
python2.5/pickle.py:977(load_binunicode)
8380.0030.0000.0070.000 /base/python_dist/lib/
python2.5/pickle.py:1166(load_long_binput)
19190.0030.0000.0040.000 /base/python_dist/lib/
python2.5/struct.py:77(unpack)
8130.0030.0000.0050.000 /base/python_dist/lib/
python2.5/pickle.py:1147(load_binget)
36560.0030.0000.0030.000 {isinstance}
10.0030.0030.1140.114 /base/data/home/apps/
sofawebstore/1.332292765683284581/shop/views.py:50(browse)
10.0030.0030.0050.005 /base/data/home/apps/
sofawebstore/1.332292765683284581/shop/views.py:1()
10.0030.0030.0030.003 /base/data/home/apps/
sofawebstore/1.332292765683284581/shop/loader.py:1()
385/130.0030.0000.0240.002 /base/data/home/apps/
sofawebstore/1.332292765683284581/models/base.py:199(toData)
23720.0030.0000.0030.000 {repr}
37950.0020.0000.0020.000 {method 'append' of
'list' objects}
10.0020.0020.0080.008 /base/data/home/apps/
sofawebstore/1.332292765683284581/shop/urls.py:2()
2880.0020.0000.0060.000 /base/python_lib/
versions/1/google/appengine/api/datastore_types.py:1051
(ValidateProperty)
1001/7510.0020.0000.0050.000 {getattr}
5110.0010.0000.0030.000 /base/python_lib/
versions/1/google/appengine/ext/db/__init__.py:377(__get__)
4300.0010.0000.0030.000 /base/python_dist/lib/
python2.5/pickle.py:1161(load_binput)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] No module named tools
Exception Type: ImportError Exception Value: No module named tools Exception Location: /base/python_lib/versions/1/google/appengine/ext/ remote_api/remote_api_stub.py in , line 74 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Trouble importing PIL module
Hi
You really need to read the docs.
http://code.google.com/appengine/docs/python/images/overview.html
Not the bit about installing PIL (only for you dev environment), it
also tells you how to install PIL in your dev environment.
T
On Mar 24, 5:16 pm, Audrey M Roy wrote:
> Hello,
>
> I'd like to use the PIL module. I know it's not supported, so I
> copied the PIL source files into a directory in my app's source
> directory.
> At this point, "import Image, ImageDraw" didn't work.
>
> I thought it might be a package name conflict and found this other
> thread:http://groups.google.com/group/google-appengine/browse_thread/thread/...
>
> Then I tried doing as suggested in that posting. I created a file
> called import_fixer.py with those contents mentioned. I removed
> "import Image, ImageDraw" and then added the following to my
> application file:
> import import_fixer
> import_fixer.FixImports('Image')
> import_fixer.FixImports('ImageDraw')
>
> Now I am getting the error:
> : No module named Image
> args = ('No module named Image',)
> message = 'No module named Image'
>
> Any advice? Thanks in advance,
> Audrey
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Django 1.1beta1: testers needed!
Hi, please download the latest app-engine-patch / Django beta release and help with finding bugs! This release comes with Django 1.1beta1, so some bugs might be in Django itself. Here's our project site: http://code.google.com/p/app-engine-patch/ Thanks! Bye, Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: @login_required usage
I have already tried this and it is giving me error --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: No module named tools
Development Console
Interactive Console#!/usr/bin/python
import code
import getpass
import sys
from google.appengine.ext.remote_api import remote_api_stub
from google.appengine.ext import db
def auth_func():
return 'service.g2...@gmail.com', 'google$$free'
app_id = 'myapp'
host = '%s.appspot.com' % app_id
remote_api_stub.ConfigureRemoteDatastore(app_id, '/remote_api', auth_func,
host)
code.interact('App Engine interactive console for %s' % (app_id,), None,
locals())
Run Program
Traceback (most recent call last):
File "/base/python_lib/versions/1/google/appengine/ext/admin/__init__.py",
line 194, in post
exec(compiled_code, globals())
File "", line 6, in
File
"/base/python_lib/versions/1/google/appengine/ext/remote_api/remote_api_stub.py",
line 74, in
from google.appengine.tools import appengine_rpc
ImportError: No module named tools
Best Regards
Tom Wu
2009/3/24 Tom Wu
>
> Exception Type: ImportError
> Exception Value:
>
> No module named tools
>
> Exception Location: /base/python_lib/versions/1/google/appengine/ext/
> remote_api/remote_api_stub.py in , line 74
>
>
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] memcache, What types can it contain?
Can memcache store semi-complex object instances? Or is it basically limited to only simple data structures? Because at the moment, i have been unable to store a complex series of class instances into memcache. Resulting in an error involving "Can't Pickle cStringIO.StringI". For the full error (mostly): http://dpaste.com/18449/ Any comments would be appreciated, thanks, -- Lee Olayvar --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: What does Model.to_xml() returns?
You could use Live HTTP Headers in Firefox (for instance) to make sure
that the browser is getting valid data.
Maybe you need to set the content-type of the response to 'text/xml'
for the browser to initialize responseXML:
self.response.headers['Content-Type'] = 'text/xml'
On Mar 23, 11:10 am, Marcelo Sena wrote:
> So I tried a more direct approach to this problem:
>
> function stateChanged()
> {
> if (xmlHttp.readyState==4)
> {
> if (xmlHttp.responseXML==null)alert("Null");
> var xmlDoc=xmlHttp.responseXML.documentElement;
> document.getElementById("companyname").innerHTML=
> xmlDoc.getElementsByTagName("compname")[0].childNodes[0].nodeValue;
> document.getElementById("contactname").innerHTML=
> xmlDoc.getElementsByTagName("contname")[0].childNodes[0].nodeValue;
> document.getElementById("address").innerHTML=
> xmlDoc.getElementsByTagName("address")[0].childNodes[0].nodeValue;
> document.getElementById("city").innerHTML=
> xmlDoc.getElementsByTagName("city")[0].childNodes[0].nodeValue;
> document.getElementById("country").innerHTML=
> xmlDoc.getElementsByTagName("country")[0].childNodes[0].nodeValue;
>
> }
> }
>
> The responseXML is always null so the line after the alert box always
> crashes.
>
> Here is the response handler I'm using:
>
> class getANote(webapp.RequestHandler):
> def get(self):
> #query=Models.NoteModel.all()
> #fnote=query.get()
>
> self.response.out.write(''' encoding='ISO-8859-1'?>
>
> Foo
> bar
> ball
> bwal
> XD
>
> ''')
>
> The xml is said well formatted using w3c validator. Now what I'm doing
> wrong, is this a app engine error? Help please!
>
> On Mar 20, 9:53 pm, Marcelo Sena wrote:
>
> > Here is my model:
> > class NoteModel(db.Model):
> > title = db.StringProperty(multiline=False)
> > content = db.ListProperty(type('string'))
> > And the values are:
> > title:'Foo'
> > content: ['bar']
>
> > And the result of:
>
> > class getANote(webapp.RequestHandler):
> > def get(self):
> > query=Models.NoteModel.all()
> > fnote=query.get()
> > xmlrep=fnote.to_xml()
>
> > self.response.out.write(xmlrep)
>
> > called from:
>
> > function stateChanged()
> > {
> > if (xmlHttp.readyState==4)
> > {
> > var xmlDoc=xmlHttp.responseXML;
> > document.getElementById("txtHint").innerHTML=xmlDoc;
>
> > }
>
> > }
>
> > But the result is:
>
> > Exactly, nothing. What am I doing wrong?
>
> > On Mar 20, 4:25 pm, Marcelo Sena wrote:
>
> > > I know what the docs say, but what kind of xml is that, is a string or
> > > a object and what is the type of that object? I tried using type() on
> > > it and it returned nothing.
> > > I need to turn my model into a XML to use it in a javascript.
> > > Regards,
> > > Marcelo Sena.
>
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: memcache, What types can it contain?
Looks like you're trying to pickle a cStringIO object. That won't work. Options are: find the code using cStringIO and convert to using StringIO (I'm guessing this will work fine), or consider implementing __getstate__ [1] instead. David [1] http://www.python.org/doc/2.5.2/lib/pickle-inst.html 2009/3/24 Lee Olayvar : > Can memcache store semi-complex object instances? Or is it basically limited > to only simple data structures? > > Because at the moment, i have been unable to store a complex series of class > instances into memcache. Resulting in an error involving "Can't Pickle > cStringIO.StringI". > > For the full error (mostly): http://dpaste.com/18449/ > > Any comments would be appreciated, thanks, > -- > Lee Olayvar > > > > -- It is better to be wrong than to be vague. — Freeman Dyson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Really weird performance issues
On 24 Mrz., 10:56, Koen Bok wrote: > I have two really weird performance issues that I can't seem to fix. > > - With our rest api the first request is always slow (~1500ms) and all > subsequent requests are way faster (~200ms). They are the exact same > requests just with different data. When I stop making requests for > like 10 secs the first one is slow again. In a request 2 memcache > objects (<2kb) are fetched and returned. What you see is that the app engine loads your code and imports all modules that you import in your app. This takes in that case about 1300ms. Requests short after that are faster, because your app is in memory. If you don't have a request for like 10 secs the app is removed from the cache so the next request has to load your app again. A way to maybe improve your app is that, if you just want to return something from the memcache, don't import apis that you don't need to serve your cached data. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Odd problem with urlfetch from picasa servers
I think the problem is that URLFetch times out in 5 seconds, and a big
picture won't get downloaded. When it's timed out, it's cut, while on
devserver, downloading will be complete even if a timeout occurs.
On Feb 26, 10:32 am, "G. Nyman" wrote:
> Hello everybody,
>
> I have been banging my head against this problem for a while now and I
> can't seem to understand why it does what it does.
>
> I am trying to import pictures from Picasa using the code appended at
> the end of this post. It works fine in the dev environment but for
> some reason picasa returns 404 when i try to run it online.
>
> The devserver-logs say:
> INFO 2009-02-26 10:24:35,093 svgae.py] Got the following
> url:http://lh6.ggpht.com/_mUcXljy1w9g/SaQ-uXbkY4I/ACE/DJtdqCz6u50...,
> data was truncated? 0
> INFO 2009-02-26 10:24:35,093 svgae.py] Big thumb got status 200
> and lenght: 99720
> INFO 2009-02-26 10:24:35,108 dev_appserver.py] "GET /
> no_go_with_picasa HTTP/1.1" 200 -
>
> The real thing says:
> 02-26 02:25AM 24.551 /no_go_with_picasa 200 927ms 951ms-cpu 0kb
> 130.232.90.98 - - [26/Feb/2009:02:25:25 -0800] "GET /no_go_with_picasa
> HTTP/1.1" 200 104 - -
> I 02-26 02:25AM 25.473
> Got the following
> url:http://lh6.ggpht.com/_mUcXljy1w9g/SaQ-uXbkY4I/ACE/DJtdqCz6u50...,
> data was truncated? 0
> I 02-26 02:25AM 25.474
> Big thumb got status 404 and lenght: 1421
>
> This might be a picasa problem but I tought I'll try here first. The
> url to the photo works in both cases so I don't understand the 404.
> The most interesting thing is that urlfetch on photo.media.thumbnail
> [0].url works fine.
>
> Thankfull for all assistance
>
> /G. Nyman
>
> Code to replicate the problem:
> ---
> class PicasaProblem(webapp.RequestHandler):
> def get(self):
> gd_client = gdata.photos.service.PhotosService()
> gdata.alt.appengine.run_on_appengine(gd_client)
> username = 'tehviu'
> albumid = 'SigmaTest'
> photos = gd_client.GetFeed(
> '/data/feed/api/user/%s/album/%s?
> kind=photo&thumbsize=64,1024&imgmax=d' % (
> username, albumid))
> for photo in photos.entry:
> img_thumb_big = None
> try:
> img_thumb_big =
> urlfetch.fetch(photo.media.thumbnail[1].url)
> except Exception, e:
> logging.exception(e)
> error = 'Image importing error'
>
> logging.info("Got the following url: " +
> photo.media.thumbnail
> [1].url + ", data was truncated? " + str
> (img_thumb_big.content_was_truncated))
> logging.info("Big thumb got status "+ str
> (img_thumb_big.status_code) + " and lenght: " + img_thumb_big.headers
> ['content-length'])
> break
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Trouble importing PIL module
I have already installed PIL in my dev environment, and it works
fine. The Images API works fine for me (my app already resizes images
and saves them to the datastore with no problem) - that's not the
issue.
The issue is that I'd like to use PIL in my app for drawing
operations, e.g. drawing rainbows.
Audrey
On Mar 24, 3:12 am, Tim Hoffman wrote:
> Hi
>
> You really need to read the docs.
> http://code.google.com/appengine/docs/python/images/overview.html
> Not the bit about installing PIL (only for you dev environment), it
> also tells you how to install PIL in your dev environment.
>
> T
>
> On Mar 24, 5:16 pm, Audrey M Roy wrote:
>
> > Hello,
>
> > I'd like to use the PIL module. I know it's not supported, so I
> > copied the PIL source files into a directory in my app's source
> > directory.
> > At this point, "import Image, ImageDraw" didn't work.
>
> > I thought it might be a package name conflict and found this other
> > thread:http://groups.google.com/group/google-appengine/browse_thread/thread/...
>
> > Then I tried doing as suggested in that posting. I created a file
> > called import_fixer.py with those contents mentioned. I removed
> > "import Image, ImageDraw" and then added the following to my
> > application file:
> > import import_fixer
> > import_fixer.FixImports('Image')
> > import_fixer.FixImports('ImageDraw')
>
> > Now I am getting the error:
> > : No module named Image
> > args = ('No module named Image',)
> > message = 'No module named Image'
>
> > Any advice? Thanks in advance,
> > Audrey
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Trouble importing PIL module
You are aware that even if you get PIL working in the SDK*, it will
not work in production
http://code.google.com/appengine/docs/python/runtime.html
(you can only import Pure Python code)
* Because you can circumvent the sandbox.
On 24/03/2009, Audrey M Roy wrote:
>
> I have already installed PIL in my dev environment, and it works
> fine. The Images API works fine for me (my app already resizes images
> and saves them to the datastore with no problem) - that's not the
> issue.
>
> The issue is that I'd like to use PIL in my app for drawing
> operations, e.g. drawing rainbows.
>
>
> Audrey
>
>
> On Mar 24, 3:12 am, Tim Hoffman wrote:
> > Hi
> >
> > You really need to read the docs.
> http://code.google.com/appengine/docs/python/images/overview.html
> > Not the bit about installing PIL (only for you dev environment), it
> > also tells you how to install PIL in your dev environment.
> >
> > T
> >
> > On Mar 24, 5:16 pm, Audrey M Roy wrote:
> >
> > > Hello,
> >
> > > I'd like to use the PIL module. I know it's not supported, so I
> > > copied the PIL source files into a directory in my app's source
> > > directory.
> > > At this point, "import Image, ImageDraw" didn't work.
> >
> > > I thought it might be a package name conflict and found this other
> > >
> thread:http://groups.google.com/group/google-appengine/browse_thread/thread/...
> >
> > > Then I tried doing as suggested in that posting. I created a file
> > > called import_fixer.py with those contents mentioned. I removed
> > > "import Image, ImageDraw" and then added the following to my
> > > application file:
> > > import import_fixer
> > > import_fixer.FixImports('Image')
> > > import_fixer.FixImports('ImageDraw')
> >
> > > Now I am getting the error:
> > > : No module named Image
> > > args = ('No module named Image',)
> > > message = 'No module named Image'
> >
> > > Any advice? Thanks in advance,
> > > Audrey
> >
>
--
Barry
- www.nearby.org.uk - www.geograph.org.uk -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: App Engine Patch - dumpdata then loaddata
I'm working with the most recent GAE Patch but still seem to have the problem. This issue here with GAE-Helper describes what I'm seeing exactly: Issue 70:Deserializer does not decode datetime fields properly http://code.google.com/p/google-app-engine-django/issues/detail?id=70 Is there any convenient way I can setup my development environment so django is used in its unzipped form? This way I can edit pieces and see if that solves my problem, without having to unzip django1.0, make the edit, and rezip? Thanks Stu On Mar 21, 7:08 am, Waldemar Kornewald wrote: > On Mar 20, 3:31 pm, Big Stu wrote: > > > Hi Waldemar, > > Thanks for following up, and thanks for all your hard work with App > > Engine Patch. I'm having a lot of fun learning about app engine and > > django in my spare time, and hope to one day move my full time career > > into this stuff. Django, GAE, and your stuff will have a huge hand in > > that. > > > Anyway, the property I'm tryign to dump is DateTimeProperty: > > date = db.DateTimeProperty() > > That should work with the repository version. > > Bye, > Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine Patch - dumpdata then loaddata
I just found out how insanely easy it is to open, edit, and save a file within a zip file under vim. I came across it on this link: http://blog.vorpal.cc/category/linux/vim-can-open-zip-files.html On Mar 24, 12:34 pm, Big Stu wrote: > I'm working with the most recent GAE Patch but still seem to have the > problem. This issue here with GAE-Helper describes what I'm seeing > exactly: > > Issue 70: Deserializer does not decode datetime fields > properlyhttp://code.google.com/p/google-app-engine-django/issues/detail?id=70 > > Is there any convenient way I can setup my development environment so > django is used in its unzipped form? This way I can edit pieces and > see if that solves my problem, without having to unzip django1.0, make > the edit, and rezip? > > Thanks > > Stu > > On Mar 21, 7:08 am, Waldemar Kornewald wrote: > > > On Mar 20, 3:31 pm, Big Stu wrote: > > > > Hi Waldemar, > > > Thanks for following up, and thanks for all your hard work with App > > > Engine Patch. I'm having a lot of fun learning about app engine and > > > django in my spare time, and hope to one day move my full time career > > > into this stuff. Django, GAE, and your stuff will have a huge hand in > > > that. > > > > Anyway, the property I'm tryign to dump is DateTimeProperty: > > > date = db.DateTimeProperty() > > > That should work with the repository version. > > > Bye, > > Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Cold app startup stats in case you're curious
very useful. Any thoughts on how this compares to the start-up of
regular GAE (0.96)?
On Mar 23, 9:27 pm, cz wrote:
> I added some timing code to see how expensive a cold app startup is.
> This is a Django1.x app so YMMV.
>
> The typical numbers are:
>
> Zipimport of Django: 180ms
> Appengine-django-helper monkeypatching: 430ms
> Misc imports, app-specific patching, etc: 20ms
>
> Total app startup: 630ms
>
> This is for startup only, not the time it takes to then process the
> request, so this is a fixed expense. Also, this is real time not cpu
> quota time which is much higher.
>
> The biggest hog is the django-helper monkeypatcher. I'm eventually
> going to try appengine-patch with the hope that it will be faster.
>
> Additionally, on a cold startup my app processes a request in about
> 500ms, but on a warm app it only takes 10ms due to some aggressive
> caching. I think that when all your app instances expire so does all
> it's memcached data.
>
> If anybody has done something similar and found different numbers
> please post the results and let me know what I did wrong.
>
> Here's the main.py I'm using in case you'd like to point out any
> errors in my methodology:
>
>
> from time import clock
> t_main_start = clock()
>
> # Standard Python imports.
> import os
> import sys
> import logging
> import settings
>
> logging.getLogger().setLevel(settings.LOGGING_LEVEL)
>
> # Fix (hack) for missing unlink
> if os.name == 'nt':
> os.unlink = lambda: None
>
> # TODO (after gathering stats): remove this since appengine_django
> does it for us
> # Import Django 1.x from a zipfile and get rid of v.96.
> t_zip_start = clock()
> django_zip_path = os.path.abspath('django.zip')
> if django_zip_path not in sys.path:
> sys.path.insert(0, django_zip_path)
> for k in [k for k in sys.modules if k.startswith('django')]:
> del sys.modules[k]
> # TODO (after gathering stats): remove this line
> from django import template # provoke django zipimport
> t_zip = (clock() - t_zip_start) * 1000
>
> # Install appengine_django. Patches Django 1.x for use with GAE
> t_patch_start = clock()
> from appengine_django import InstallAppengineHelperForDjango
> InstallAppengineHelperForDjango()
> t_patch = (clock() - t_patch_start) * 1000
>
> # Patch the appengine_django patch to fix User object creation
> # Remove this when switching to appengine-patch
> from monkeypatch import patch
> patch()
>
> # Google App Engine imports.
> from google.appengine.ext.webapp import util
>
> # Import the part of Django that we use here.
> import django.core.handlers.wsgi
>
> t_main = (clock() - t_main_start) * 1000
> logging.info('[TIME] main.py startup: %.1fms (zipimport: %.1fms,
> appengine-django: %.1fms)' % (t_main,t_zip,t_patch))
>
> def real_main():
> t_request_start = clock()
> # Create a Django application for WSGI.
> application = django.core.handlers.wsgi.WSGIHandler()
>
> # Run the WSGI CGI handler with that application.
> util.run_wsgi_app(application)
> t_request = (clock() - t_request_start) * 1000
> logging.info('[TIME] request: %.1fms' % t_request)
>
> def profile_main():
> # Rename this to main() for profiling
> import cProfile, pstats
> prof = cProfile.Profile()
> prof = prof.runctx("real_main()", globals(), locals())
> print ""
> stats = pstats.Stats(prof)
> stats.sort_stats("time") # Or cumulative
> stats.print_stats(80) # 80 = how many to print
> # The rest is optional.
> #stats.print_callees()
> #stats.print_callers()
> print ""
>
> main = real_main
>
> if __name__ == '__main__':
> main()
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Trouble importing PIL module
Oops...yes, thanks, Barry.
Audrey
On Mar 24, 9:00 am, Barry Hunter wrote:
> You are aware that even if you get PIL working in the SDK*, it will
> not work in production
>
> http://code.google.com/appengine/docs/python/runtime.html
>
> (you can only import Pure Python code)
>
> * Because you can circumvent the sandbox.
>
> On 24/03/2009, Audrey M Roy wrote:
>
>
>
>
>
> > I have already installed PIL in my dev environment, and it works
> > fine. The Images API works fine for me (my app already resizes images
> > and saves them to the datastore with no problem) - that's not the
> > issue.
>
> > The issue is that I'd like to use PIL in my app for drawing
> > operations, e.g. drawing rainbows.
>
> > Audrey
>
> > On Mar 24, 3:12 am, Tim Hoffman wrote:
> > > Hi
>
> > > You really need to read the docs.
> > http://code.google.com/appengine/docs/python/images/overview.html
> > > Not the bit about installing PIL (only for you dev environment), it
> > > also tells you how to install PIL in your dev environment.
>
> > > T
>
> > > On Mar 24, 5:16 pm, Audrey M Roy wrote:
>
> > > > Hello,
>
> > > > I'd like to use the PIL module. I know it's not supported, so I
> > > > copied the PIL source files into a directory in my app's source
> > > > directory.
> > > > At this point, "import Image, ImageDraw" didn't work.
>
> > > > I thought it might be a package name conflict and found this other
> > > >
> > thread:http://groups.google.com/group/google-appengine/browse_thread/thread/...
>
> > > > Then I tried doing as suggested in that posting. I created a file
> > > > called import_fixer.py with those contents mentioned. I removed
> > > > "import Image, ImageDraw" and then added the following to my
> > > > application file:
> > > > import import_fixer
> > > > import_fixer.FixImports('Image')
> > > > import_fixer.FixImports('ImageDraw')
>
> > > > Now I am getting the error:
> > > > : No module named Image
> > > > args = ('No module named Image',)
> > > > message = 'No module named Image'
>
> > > > Any advice? Thanks in advance,
> > > > Audrey
>
> --
> Barry
>
> -www.nearby.org.uk-www.geograph.org.uk-
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Lots of Application Error: 5 on URLFetch
I'm calling content from Twitter on a regular basis (every 5 minutes), but suddenly over the last two days URLFetch is consistently (for more than a couple of hours) returning error 5 on my URLFetch. Now, I know that Twitter appears "slower" at these times, but it's still responding when I manually fetch the URL. Is there any way I can extend the length of time URL Fetch waits before timing out? Cheers, Tim Bull --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
Starred On Mar 23, 2:55 pm, Andy Freeman wrote: > According > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ > , simplejson is significantly slower than cjson. > > I've created an issue requesting the fastest possible json > athttp://code.google.com/p/googleappengine/issues/detail?id=1174. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] cann't update value in memcache
i tried 2 method to update valuse in memcache:
1:define a expiration time when adding a key
if not memcache.add("notes", notes,60):
i suppose the value will be updated in 1 minute,but it doesn't
2: use replace or set ,even delete then add ,neither can
could you give me some help?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: cann't update value in memcache
On Tue, Mar 24, 2009 at 8:00 PM, Coonay wrote:
>
> i tried 2 method to update valuse in memcache:
> 1:define a expiration time when adding a key
>
> if not memcache.add("notes", notes,60):
>
>
> i suppose the value will be updated in 1 minute,but it doesn't
>
The value will be removed from the cache in 1 minute. This means in 1 minute
if you call add it will add it (never tried it though).
>
>
> 2: use replace or set ,even delete then add ,neither can
>
>
> could you give me some help?
>
Can you paste the code here?
>
>
>
>
> >
>
--
Alkis
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Data upload: differences between bulkload_client and bulkloader ?
Hi, The following docs describe how to upload csv data to the datastore: http://code.google.com/appengine/articles/bulkload.html http://code.google.com/appengine/docs/python/tools/uploadingdata.html But one use the script bulkload_client.py and the other bulkloader.py. In both files it is written that their purpose is to "Imports CSV data over HTTP." What is the difference?? I cannot figure out. Julian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
Update (20090324-2): According to John Millikin, the author of jsonlib, cjson is buggy and unmaintained. I will evaluate further and post a followup blog entry. My discussion with Dan Pascu, the author of cjson, corroborates these claims. I urge readers to read John Millikin's comment. On Mar 23, 10:55 am, Andy Freeman wrote: > According > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ > , simplejson is significantly slower than cjson. > > I've created an issue requesting the fastest possible json > athttp://code.google.com/p/googleappengine/issues/detail?id=1174. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
I am the author of the blog post. Update (20090324): According to Extra Cheese, cjson 1.0.5 has an incompatibility with simplejson in processing slashes. A fix is available from http://www.vazor.com/cjson.html"; rel="nofollow">Matt Billenstein. However, Dan Pascu, the author of cjson, deprecates Matt Billenstein's cjson 1.0.6 because Matt's patch parses the JSON twice, which makes it twice as slow. This will still be faster than all alternatives in certain circumstances. You will not find Matt's cjson on the cheeseshop, only on Matt's site. On Mar 23, 10:55 am, Andy Freeman wrote: > According > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ > , simplejson is significantly slower than cjson. > > I've created an issue requesting the fastest possible json > athttp://code.google.com/p/googleappengine/issues/detail?id=1174. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Indexes Building for (3) Days
Hello, About 14K entities had been loaded *before* the index was created: kind: WordOccurance properties: - name: word (string) - name: partofspeech (int) I thought that maybe I should have first created the index and then load the data, so I then created this index with has no data. It has been building for (2) days. kind: WordCategory properties: - name: word (string) - name: partofspeech (int) Any help is appreciated. thank, David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Bandwidth quota calculation and DoS attacks
This was the response from Google: "Going forward, if you see a large amount of suspicious requests coming in for a particular resource, please block the suspicious IP addresses as you did and/or temporarily move the resource, which will help to determine whether this is a true DoS attack or simply an unexpected spike in traffic from a single location or ISP." I've since tested the bandwidth quota as described in the issue below where I transfered less than 25mb of data that was reported as 5gb: http://code.google.com/p/googleappengine/issues/detail?id=1178 I'm in the process of building a reliable per-IP bandwidth limiter for static files, but I hope that Google will incorporate this functionality natively into GAE. Marc On Mar 23, 3:38 pm, boson wrote: > Some relevant references: > > Marzia Niccolai (Google) said this in Sep 2008: > "On the broader issue of denial-of-service attacks, these are an > unfortunate > reality in the web world. While we don't currently offer applications > any > specific protections against attacks of this nature, this is something > we're > interested in looking into for the future. In the near-term, when we > begin > allowing developers to purchase computing resources beyond our free > limits, > we will provide a mechanism for reimbursement in the event of a DOS > attack." > Src:http://groups.google.com/group/google-appengine/browse_thread/thread/... > > and star this issue: > > Issue 718: Denial of Service Attack - > protectionhttp://code.google.com/p/googleappengine/issues/detail?id=718 > > On Mar 20, 1:31 pm, codermarc wrote: > > > I was recently the victim of a DoS attack against an App Engine app. > > The attacker requested a 2.3mb file approximately 13k times over 1.5 > > hours, and at least 6k of the requests were successful before by > >bandwidthquota was exceeded. > > > All of the requests were coming from a single IP address in Central > > America. The successful requests were spread out over about 45 > > minutes, which would mean a constant outgoingbandwidthrate of 5.1mb/ > > s. Does this make any sense? > > > How does Google calculate outgoingbandwidthfor static file requests? > > When a file is requested, is it automatically assumed that the entire > > file will be transferred, or is the actualbandwidthused calculated? > > > Also, is anybody aware of anything I can do in the future to prevent > > such an attack? > > > Thanks for your feedback! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Index Building for (3) Days
Hello, I have (2) indexes which have been building for (3) days. The 1st was created after loading about 14K entries. The index consists of a string and int attribute. Thinking that I should have first created the index before loading the data, I created a new Model with no data, and the same string-int index. It has also now been "building" for a couple of days. The Application ID is bible-library. thank you, David --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Files not updating?
Hello all, During our development, we sometimes see that the new version of application we updated to AppEngine using appcfg.py does not show up. When we try to access it from browser, it is still the older version. Sometimes, this is occasional and the next update works. Sometimes, we will have to wait for 10 or 20mins and retry update. Now, it takes hours before our update become working. During these time, we keep trying to run appcfg.py to update, but the new change does not seem to be there. Does anyone know such problem? We are updating using appcfg.py update bin-release Is there a limit on how many times we can update the appengine application inside a day? I did not seem to see such limit. We have one .swf file that is about 900kb. That is the file that we constantly updating and testing. Is there a limit on bandwidth of the update? Inside app.yaml, we are serving .swf file this way. - url: /(.*\.(swf|html|js|xml|gif|png|jpg)) static_files: \1 upload: (.*\.(swf|html|js|xml|gif|png|jpg)) Thank you in advance for any comment or suggestions. Very much appreciated. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] key - RefrenceProperty
I am struggling with understanding how to use keys to enforce
reference between two tables.
I need to access the same set of data from table one, and table two.
Then change data within both tables. The data manipulation works in
the "person" table, but it looses the connection to itself in the
"parentperson" table.
The code below demonstrates the issue. Reload the application four+
times and you can see the 'parentperson' table/model fills up with
entries. The key I get from 'person' is different every time, but
there is only one row in 'person'.
Completely confused as to what I am missing on this one..
class person(db.Model):
firstName = db.StringProperty()
lastName = db.StringProperty()
rid = db.IntegerProperty()
class parentperson(db.Model):
childname = db.ReferenceProperty()
persontype = db.StringProperty()
updateTime = db.DateTimeProperty()
class ShowIssues(webapp.RequestHandler):
def get(self):
personLookup = person()
query = personLookup.gql("WHERE rid = :1", int(1234))
result = query.get()
if result:
tParent = db.Query(parentperson)
tParent.filter('persontype', "Daughter")
tParent.filter('childname',result.key())
tResult = tParent.get()
else:
tResult = None
if tResult:
pOutput = """
Child - Type: %s
Key Name: %s
Parent First Name is: %s
Parent Last Name is: %s
Last Update Time was: %s
""" %(tResult.persontype, str(tResult.childname),
tResult.childname.firstName, tResult.childname.lastName, str
(tResult.updateTime))
self.response.out.write(pOutput)
if result:
output = """
First Name is %s
Last Name is %s
Key is %s
""" %(result.firstName, result.lastName,
str(result.key()))
self.response.out.write(output)
result.lastName = "New Smith"
result.put()
gParent = parentperson()
gParent.childname = result.key()
gParent.persontype = "Daughter"
gParent.updateTime = datetime.now()
gParent.put()
else:
result = person()
result.firstName = "John"
result.lastName = "Smith"
result.rid = int(1234)
result.put()
qi = parentperson.gql("LIMIT 1000")
qresults = qi.fetch(1000)
self.response.out.write("")
for qires in qresults:
self.response.out.write(str(qires.childname) + " " + str
(qires.persontype) +"")
#qires.delete()
qt = person.gql("LIMIT 1000")
qtresults = qt.fetch(1000)
self.response.out.write("")
for qtires in qtresults:
self.response.out.write(str(qtires.firstName) + " " + str
(qtires.lastName) +"")
#qtires.delete()
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Access external resources with IP block from GAE
Hi all, My GAE app needs to access an external service wich uses an IP check. Is it possible to use a static IP with GAE or does anyone have an idea for a workaround? (I dont want to pay for a service that can act as a proxy) Cheers! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
That's why issue 1174 requests the fastest possible json, not cjson. 1174 actually requests multiple jsons, so folks can use what's best for their application. On Mar 23, 10:32 pm, Joseph Turian wrote: > I am the author of the blog post. > > Update (20090324): According to rel="nofollow">Extra Cheese, cjson 1.0.5 has an incompatibility > with simplejson in processing slashes. A fix is available from href="http://www.vazor.com/cjson.html"; rel="nofollow">Matt > Billenstein. However, Dan Pascu, the author of cjson, deprecates > Matt Billenstein's cjson 1.0.6 because Matt's patch parses the JSON > twice, which makes it twice as slow. This will still be faster than > all alternatives in certain circumstances. You will not find Matt's > cjson on the cheeseshop, only on Matt's site. > > On Mar 23, 10:55 am, Andy Freeman wrote: > > > > > According > > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ > > , simplejson is significantly slower than cjson. > > > I've created an issue requesting the fastest possible json > > athttp://code.google.com/p/googleappengine/issues/detail?id=1174.- Hide > > quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
Just idly wondering, Have you done any quantitative measurement of how "bad" simplejson is? I can't imagine it being a large overhead, unless your application is seriously optimized to extremes already. David 2009/3/24 Andy Freeman : > > That's why issue 1174 requests the fastest possible json, not cjson. > > 1174 actually requests multiple jsons, so folks can use what's best > for their application. > > > > > On Mar 23, 10:32 pm, Joseph Turian wrote: >> I am the author of the blog post. >> >> Update (20090324): According to > rel="nofollow">Extra Cheese, cjson 1.0.5 has an incompatibility >> with simplejson in processing slashes. A fix is available from > href="http://www.vazor.com/cjson.html"; rel="nofollow">Matt >> Billenstein. However, Dan Pascu, the author of cjson, deprecates >> Matt Billenstein's cjson 1.0.6 because Matt's patch parses the JSON >> twice, which makes it twice as slow. This will still be faster than >> all alternatives in certain circumstances. You will not find Matt's >> cjson on the cheeseshop, only on Matt's site. >> >> On Mar 23, 10:55 am, Andy Freeman wrote: >> >> >> >> > According >> > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ >> > , simplejson is significantly slower than cjson. >> >> > I've created an issue requesting the fastest possible json >> > athttp://code.google.com/p/googleappengine/issues/detail?id=1174.- Hide >> > quoted text - >> >> - Show quoted text - > > > -- It is better to be wrong than to be vague. — Freeman Dyson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: key - RefrenceProperty
your ReferenceProperty should be declared as:
childname = db.ReferenceProperty(person)
and then set as:
childname = result
i think :)
Sent from my iPhone
On 09-03-24, at 09:51, Steve W wrote:
>
> I am struggling with understanding how to use keys to enforce
> reference between two tables.
>
> I need to access the same set of data from table one, and table two.
> Then change data within both tables. The data manipulation works in
> the "person" table, but it looses the connection to itself in the
> "parentperson" table.
>
> The code below demonstrates the issue. Reload the application four+
> times and you can see the 'parentperson' table/model fills up with
> entries. The key I get from 'person' is different every time, but
> there is only one row in 'person'.
>
> Completely confused as to what I am missing on this one..
>
> class person(db.Model):
>firstName = db.StringProperty()
>lastName = db.StringProperty()
>rid = db.IntegerProperty()
>
> class parentperson(db.Model):
>childname = db.ReferenceProperty()
>persontype = db.StringProperty()
>updateTime = db.DateTimeProperty()
>
>
> class ShowIssues(webapp.RequestHandler):
>def get(self):
>personLookup = person()
>query = personLookup.gql("WHERE rid = :1", int(1234))
>result = query.get()
>
>if result:
>tParent = db.Query(parentperson)
>tParent.filter('persontype', "Daughter")
>tParent.filter('childname',result.key())
>tResult = tParent.get()
>else:
>tResult = None
>
>if tResult:
>pOutput = """
>Child - Type: %s
>
>Key Name: %s
>
>Parent First Name is: %s
>
>Parent Last Name is: %s
>
>Last Update Time was: %s
>
>""" %(tResult.persontype, str(tResult.childname),
> tResult.childname.firstName, tResult.childname.lastName, str
> (tResult.updateTime))
>self.response.out.write(pOutput)
>if result:
>output = """
>First Name is %s
>
>Last Name is %s
>
>Key is %s
>""" %(result.firstName, result.lastName,
> str(result.key()))
>self.response.out.write(output)
>result.lastName = "New Smith"
>result.put()
>gParent = parentperson()
>gParent.childname = result.key()
>gParent.persontype = "Daughter"
>gParent.updateTime = datetime.now()
>gParent.put()
>
>else:
>result = person()
>result.firstName = "John"
>result.lastName = "Smith"
>result.rid = int(1234)
>result.put()
>
>qi = parentperson.gql("LIMIT 1000")
>qresults = qi.fetch(1000)
>self.response.out.write("")
>for qires in qresults:
>self.response.out.write(str(qires.childname) + " " + str
> (qires.persontype) +"")
>#qires.delete()
>
>qt = person.gql("LIMIT 1000")
>qtresults = qt.fetch(1000)
>self.response.out.write("")
>for qtires in qtresults:
>self.response.out.write(str(qtires.firstName) + " " + str
> (qtires.lastName) +"")
>#qtires.delete()
>
> >
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: key - RefrenceProperty
I happened to catch that today while the post was being moderated.
I updated the code to include that (person) and it still doesn't work.
It seems as though childname=result ( and childname=result.key())
returns an in memory object, and not really the key id.
doing a str(result.key()) after the put() returns a string similar too
<__main__.person object at 0x036C7210>
the memory address changes each time.
I am considering storing the str(result.key()) in a
StringPropertyType, but I shouldn't have too. =)
Thanks for responding so quick.
On Mar 24, 3:56 pm, Paul Roy wrote:
> your ReferenceProperty should be declared as:
>
> childname = db.ReferenceProperty(person)
>
> and then set as:
>
> childname = result
>
> i think :)
>
> Sent from my iPhone
>
> On 09-03-24, at 09:51, Steve W wrote:
>
>
>
> > I am struggling with understanding how to use keys to enforce
> > reference between two tables.
>
> > I need to access the same set of data from table one, and table two.
> > Then change data within both tables. The data manipulation works in
> > the "person" table, but it looses the connection to itself in the
> > "parentperson" table.
>
> > The code below demonstrates the issue. Reload the application four+
> > times and you can see the 'parentperson' table/model fills up with
> > entries. The key I get from 'person' is different every time, but
> > there is only one row in 'person'.
>
> > Completely confused as to what I am missing on this one..
>
> > class person(db.Model):
> > firstName = db.StringProperty()
> > lastName = db.StringProperty()
> > rid = db.IntegerProperty()
>
> > class parentperson(db.Model):
> > childname = db.ReferenceProperty()
> > persontype = db.StringProperty()
> > updateTime = db.DateTimeProperty()
>
> > class ShowIssues(webapp.RequestHandler):
> > def get(self):
> > personLookup = person()
> > query = personLookup.gql("WHERE rid = :1", int(1234))
> > result = query.get()
>
> > if result:
> > tParent = db.Query(parentperson)
> > tParent.filter('persontype', "Daughter")
> > tParent.filter('childname',result.key())
> > tResult = tParent.get()
> > else:
> > tResult = None
>
> > if tResult:
> > pOutput = """
> > Child - Type: %s
> >
> > Key Name: %s
> >
> > Parent First Name is: %s
> >
> > Parent Last Name is: %s
> >
> > Last Update Time was: %s
> >
> > """ %(tResult.persontype, str(tResult.childname),
> > tResult.childname.firstName, tResult.childname.lastName, str
> > (tResult.updateTime))
> > self.response.out.write(pOutput)
> > if result:
> > output = """
> > First Name is %s
> >
> > Last Name is %s
> >
> > Key is %s
> > """ %(result.firstName, result.lastName,
> > str(result.key()))
> > self.response.out.write(output)
> > result.lastName = "New Smith"
> > result.put()
> > gParent = parentperson()
> > gParent.childname = result.key()
> > gParent.persontype = "Daughter"
> > gParent.updateTime = datetime.now()
> > gParent.put()
>
> > else:
> > result = person()
> > result.firstName = "John"
> > result.lastName = "Smith"
> > result.rid = int(1234)
> > result.put()
>
> > qi = parentperson.gql("LIMIT 1000")
> > qresults = qi.fetch(1000)
> > self.response.out.write("")
> > for qires in qresults:
> > self.response.out.write(str(qires.childname) + " " + str
> > (qires.persontype) +"")
> > #qires.delete()
>
> > qt = person.gql("LIMIT 1000")
> > qtresults = qt.fetch(1000)
> > self.response.out.write("")
> > for qtires in qtresults:
> > self.response.out.write(str(qtires.firstName) + " " + str
> > (qtires.lastName) +"")
> > #qtires.delete()
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: key - RefrenceProperty
i figured it out..
The issue is the sequence after the result.put() followed by
gParent.
The childname result needs to be updated. The original code was
creating a new entry every time for parentperson() instead of reusing
the original .get().
The code below contains some goofy if nests, but it was a result of
testing without care of structure. It's only a proof (if only to
myself) of how it works.
Here is a working piece of code.
class person(db.Model):
firstName = db.StringProperty()
lastName = db.StringProperty()
rid = db.IntegerProperty()
class parentperson(db.Model):
childname = db.ReferenceProperty(person)
persontype = db.StringProperty()
updateTime = db.DateTimeProperty()
class ShowIssues(webapp.RequestHandler):
def get(self):
personLookup = person()
query = personLookup.gql("WHERE rid = :1", int(1234))
result = query.get()
if result:
tParent = db.Query(parentperson)
tParent.filter('persontype', 'Daughter')
tParent.filter('childname',result.key())
tResult = tParent.get()
if tResult:
pOutput = """
Child - Type: %s
Key Name: %s
Parent First Name is: %s
Parent Last Name is: %s
Last Update Time was: %s
""" %(tResult.persontype, str(tResult.childname),
tResult.childname.firstName, tResult.childname.lastName, str
(tResult.updateTime))
self.response.out.write(pOutput)
else:
tResult = None
if result:
output = """
First Name is %s
Last Name is %s
Key is %s
""" %(result.firstName, result.lastName,
str(result.key()))
self.response.out.write(output)
result.lastName = "New Smith"
result.put()
if tResult:
# update the existing child
# datetime is used for testing a unique data
insertion
tResult.updateTime = datetime.now()
tResult.put()
else:
# its a new child
gParent = parentperson()
gParent.childname = result.key()
gParent.persontype = "Daughter"
gParent.updateTime = datetime.now()
gParent.put()
else:
result = person()
result.firstName = "John"
result.lastName = "Smith"
result.rid = int(1234)
result.put()
qi = parentperson.gql("LIMIT 1000")
qresults = qi.fetch(1000)
self.response.out.write("")
for qires in qresults:
self.response.out.write(str(qires.childname) + " " + str
(qires.persontype) +"")
#qires.delete()
qt = person.gql("LIMIT 1000")
qtresults = qt.fetch(1000)
self.response.out.write("")
for qtires in qtresults:
self.response.out.write(str(qtires.firstName) + " " + str
(qtires.lastName) +"")
#qtires.delete()
On Mar 24, 4:00 pm, Steve W wrote:
> I happened to catch that today while the post was being moderated.
>
> I updated the code to include that (person) and it still doesn't work.
> It seems as though childname=result ( and childname=result.key())
> returns an in memory object, and not really the key id.
>
> doing a str(result.key()) after the put() returns a string similar too
> <__main__.person object at 0x036C7210>
> the memory address changes each time.
>
> I am considering storing the str(result.key()) in a
> StringPropertyType, but I shouldn't have too. =)
>
> Thanks for responding so quick.
>
> On Mar 24, 3:56 pm, Paul Roy wrote:
>
> > your ReferenceProperty should be declared as:
>
> > childname = db.ReferenceProperty(person)
>
> > and then set as:
>
> > childname = result
>
> > i think :)
>
> > Sent from my iPhone
>
> > On 09-03-24, at 09:51, Steve W wrote:
>
> > > I am struggling with understanding how to use keys to enforce
> > > reference between two tables.
>
> > > I need to access the same set of data from table one, and table two.
> > > Then change data within both tables. The data manipulation works in
> > > the "person" table, but it looses the connection to itself in the
> > > "parentperson" table.
>
> > > The code below demonstrates the issue. Reload the application four+
> > > times and you can see the 'parentperson' table/model fills up with
> > > entries. The key I get from 'person' is different every time, but
> > > there is only one row in 'person'.
>
> > > Completely confused as to what I am missing on this one..
>
> > > class person(db.Model):
> > > firstName = db.StringProperty()
> > >
[google-appengine] Re: Faster server-side json?
The referenced post has examples of simplejson taking 10x more time than cjson. 10x difference in a component is noticeable for some applications. It also shows that pickle is much slower and has 2x bigger output than json for "objects". (For strings, they all should be roughly the same.) I mention pickle because it's used by memcache. Since both pickle and json should be very efficient for strings, it may make sense to memcache json output instead of objects. In addition, some people are using pickle to create datastore blobs. Perhaps json is a better choice. On Mar 24, 1:39 pm, David Wilson wrote: > Just idly wondering, > > Have you done any quantitative measurement of how "bad" simplejson is? > I can't imagine it being a large overhead, unless your application is > seriously optimized to extremes already. > > David > > 2009/3/24 Andy Freeman : > > > > > > > > > That's why issue 1174 requests the fastest possible json, not cjson. > > > 1174 actually requests multiple jsons, so folks can use what's best > > for their application. > > > On Mar 23, 10:32 pm, Joseph Turian wrote: > >> I am the author of the blog post. > > >> Update (20090324): According to >> rel="nofollow">Extra Cheese, cjson 1.0.5 has an incompatibility > >> with simplejson in processing slashes. A fix is available from >> href="http://www.vazor.com/cjson.html"; rel="nofollow">Matt > >> Billenstein. However, Dan Pascu, the author of cjson, deprecates > >> Matt Billenstein's cjson 1.0.6 because Matt's patch parses the JSON > >> twice, which makes it twice as slow. This will still be faster than > >> all alternatives in certain circumstances. You will not find Matt's > >> cjson on the cheeseshop, only on Matt's site. > > >> On Mar 23, 10:55 am, Andy Freeman wrote: > > >> > According > >> > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ > >> > , simplejson is significantly slower than cjson. > > >> > I've created an issue requesting the fastest possible json > >> > athttp://code.google.com/p/googleappengine/issues/detail?id=1174.-Hide > >> > quoted text - > > >> - Show quoted text - > > -- > It is better to be wrong than to be vague. > — Freeman Dyson- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine Patch - dumpdata then loaddata
So, I editing the django1.0/core/serializers/python.py file to have the changes mentioned in Issue 70 for Google App Engine Helper, and I'm not getting this error: Installing json fixture 'data' from absolute path. Problem installing fixture 'data.json': Traceback (most recent call last): File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/management/commands/ loaddata.py", line 116, in handle for obj in objects: File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/serializers/json.py", line 41, in Deserializer for obj in PythonDeserializer(simplejson.load(stream)): File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/serializers/python.py", line 119, in Deserializer data[field.name] = field.validate(field_value) File "/home/sadohert/development/google_ae/google_appengine/google/ appengine/ext/db/__init__.py", line 1910, in validate 'to a Blob instance (%s)' % (self.name, err)) BadValueError: Property data must be convertible to a Blob instance (Blob() argument should be str instance, not unicode) Could there be something wrong with my setup? Otherwise I would think there would have been many complaints about this issue already. On Mar 24, 12:42 pm, Big Stu wrote: > I just found out how insanely easy it is to open, edit, and save a > file within a zip file under vim. I came across it on this link: > > http://blog.vorpal.cc/category/linux/vim-can-open-zip-files.html > > On Mar 24, 12:34 pm, Big Stu wrote: > > > I'm working with the most recent GAE Patch but still seem to have the > > problem. This issue here with GAE-Helper describes what I'm seeing > > exactly: > > > Issue 70: Deserializer does not decode datetime fields > > properlyhttp://code.google.com/p/google-app-engine-django/issues/detail?id=70 > > > Is there any convenient way I can setup my development environment so > > django is used in its unzipped form? This way I can edit pieces and > > see if that solves my problem, without having to unzip django1.0, make > > the edit, and rezip? > > > Thanks > > > Stu > > > On Mar 21, 7:08 am, Waldemar Kornewald wrote: > > > > On Mar 20, 3:31 pm, Big Stu wrote: > > > > > Hi Waldemar, > > > > Thanks for following up, and thanks for all your hard work with App > > > > Engine Patch. I'm having a lot of fun learning about app engine and > > > > django in my spare time, and hope to one day move my full time career > > > > into this stuff. Django, GAE, and your stuff will have a huge hand in > > > > that. > > > > > Anyway, the property I'm tryign to dump is DateTimeProperty: > > > > date = db.DateTimeProperty() > > > > That should work with the repository version. > > > > Bye, > > > Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] open source app engine
I haven't seen any mention of appscale http://code.google.com/p/appscale/ . It may not be ready for prime time now, but it looks like there will soon be a way for other folks to sell hosting for App Engine applications and for folks to run their own app engine clusters. In other words, no lock-in. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine Patch - dumpdata then loaddata
So, I editing the django1.0/core/serializers/python.py file to have the changes mentioned in Issue 70 for Google App Engine Helper, and I'm not getting this error: Installing json fixture 'data' from absolute path. Problem installing fixture 'data.json': Traceback (most recent call last): File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/management/commands/ loaddata.py", line 116, in handle for obj in objects: File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/serializers/json.py", line 41, in Deserializer for obj in PythonDeserializer(simplejson.load(stream)): File "/home/sadohert/development/google_ae/leagr_patch/common/zip- packages/django-1.0.2.zip/django/core/serializers/python.py", line 119, in Deserializer data[field.name] = field.validate(field_value) File "/home/sadohert/development/google_ae/google_appengine/google/ appengine/ext/db/__init__.py", line 1910, in validate 'to a Blob instance (%s)' % (self.name, err)) BadValueError: Property data must be convertible to a Blob instance (Blob() argument should be str instance, not unicode) Could there be something wrong with my setup? Otherwise I would think there would have been many complaints about this issue already. On Mar 24, 12:42 pm, Big Stu wrote: > I just found out how insanely easy it is to open, edit, and save a > file within a zip file under vim. I came across it on this link: > > http://blog.vorpal.cc/category/linux/vim-can-open-zip-files.html > > On Mar 24, 12:34 pm, Big Stu wrote: > > > I'm working with the most recent GAE Patch but still seem to have the > > problem. This issue here with GAE-Helper describes what I'm seeing > > exactly: > > > Issue 70: Deserializer does not decode datetime fields > > properlyhttp://code.google.com/p/google-app-engine-django/issues/detail?id=70 > > > Is there any convenient way I can setup my development environment so > > django is used in its unzipped form? This way I can edit pieces and > > see if that solves my problem, without having to unzip django1.0, make > > the edit, and rezip? > > > Thanks > > > Stu > > > On Mar 21, 7:08 am, Waldemar Kornewald wrote: > > > > On Mar 20, 3:31 pm, Big Stu wrote: > > > > > Hi Waldemar, > > > > Thanks for following up, and thanks for all your hard work with App > > > > Engine Patch. I'm having a lot of fun learning about app engine and > > > > django in my spare time, and hope to one day move my full time career > > > > into this stuff. Django, GAE, and your stuff will have a huge hand in > > > > that. > > > > > Anyway, the property I'm tryign to dump is DateTimeProperty: > > > > date = db.DateTimeProperty() > > > > That should work with the repository version. > > > > Bye, > > > Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: App Engine Patch - dumpdata then loaddata
Looks like this BadValueError is happening when the deserializer tries to parse the "data" field of the "sessions.session" model. On Mar 24, 7:30 pm, Big Stu wrote: > So, I editing the django1.0/core/serializers/python.py file to have > the changes mentioned in Issue 70 for Google App Engine Helper, and > I'm not getting this error: > > Installing json fixture 'data' from absolute path. > Problem installing fixture 'data.json': Traceback (most recent call > last): > File "/home/sadohert/development/google_ae/leagr_patch/common/zip- > packages/django-1.0.2.zip/django/core/management/commands/ > loaddata.py", line 116, in handle > for obj in objects: > File "/home/sadohert/development/google_ae/leagr_patch/common/zip- > packages/django-1.0.2.zip/django/core/serializers/json.py", line 41, > in Deserializer > for obj in PythonDeserializer(simplejson.load(stream)): > File "/home/sadohert/development/google_ae/leagr_patch/common/zip- > packages/django-1.0.2.zip/django/core/serializers/python.py", line > 119, in Deserializer > data[field.name] = field.validate(field_value) > File "/home/sadohert/development/google_ae/google_appengine/google/ > appengine/ext/db/__init__.py", line 1910, in validate > 'to a Blob instance (%s)' % (self.name, err)) > BadValueError: Property data must be convertible to a Blob instance > (Blob() argument should be str instance, not unicode) > > Could there be something wrong with my setup? Otherwise I would think > there would have been many complaints about this issue already. > On Mar 24, 12:42 pm, Big Stu wrote: > > > I just found out how insanely easy it is to open, edit, and save a > > file within a zip file under vim. I came across it on this link: > > >http://blog.vorpal.cc/category/linux/vim-can-open-zip-files.html > > > On Mar 24, 12:34 pm, Big Stu wrote: > > > > I'm working with the most recent GAE Patch but still seem to have the > > > problem. This issue here with GAE-Helper describes what I'm seeing > > > exactly: > > > > Issue 70: Deserializer does not decode datetime fields > > > properlyhttp://code.google.com/p/google-app-engine-django/issues/detail?id=70 > > > > Is there any convenient way I can setup my development environment so > > > django is used in its unzipped form? This way I can edit pieces and > > > see if that solves my problem, without having to unzip django1.0, make > > > the edit, and rezip? > > > > Thanks > > > > Stu > > > > On Mar 21, 7:08 am, Waldemar Kornewald wrote: > > > > > On Mar 20, 3:31 pm, Big Stu wrote: > > > > > > Hi Waldemar, > > > > > Thanks for following up, and thanks for all your hard work with App > > > > > Engine Patch. I'm having a lot of fun learning about app engine and > > > > > django in my spare time, and hope to one day move my full time career > > > > > into this stuff. Django, GAE, and your stuff will have a huge hand in > > > > > that. > > > > > > Anyway, the property I'm tryign to dump is DateTimeProperty: > > > > > date = db.DateTimeProperty() > > > > > That should work with the repository version. > > > > > Bye, > > > > Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Faster server-side json?
These aren't the kind of quantitative I meant. ;) 10 times nothing is still nothing. What kind of overhead, in terms of time and CPU usage in a typical request in your application? It's just a bunch of string manipulation, it should be really cheap compared to even a single network/memcache/datastore access (seeing as CPU quota seems to be based mainly on request duration). 2009/3/24 Andy Freeman : > > The referenced post has examples of simplejson taking 10x more time > than cjson. 10x difference in a component is noticeable for some > applications. > > It also shows that pickle is much slower and has 2x bigger output than > json for "objects". (For strings, they all should be roughly the > same.) > > I mention pickle because it's used by memcache. Since both pickle and > json should be very efficient for strings, it may make sense to > memcache json output instead of objects. > > In addition, some people are using pickle to create datastore blobs. > Perhaps json is a better choice. > > On Mar 24, 1:39 pm, David Wilson wrote: >> Just idly wondering, >> >> Have you done any quantitative measurement of how "bad" simplejson is? >> I can't imagine it being a large overhead, unless your application is >> seriously optimized to extremes already. >> >> David >> >> 2009/3/24 Andy Freeman : >> >> >> >> >> >> >> >> > That's why issue 1174 requests the fastest possible json, not cjson. >> >> > 1174 actually requests multiple jsons, so folks can use what's best >> > for their application. >> >> > On Mar 23, 10:32 pm, Joseph Turian wrote: >> >> I am the author of the blog post. >> >> >> Update (20090324): According to > >> rel="nofollow">Extra Cheese, cjson 1.0.5 has an incompatibility >> >> with simplejson in processing slashes. A fix is available from > >> href="http://www.vazor.com/cjson.html"; rel="nofollow">Matt >> >> Billenstein. However, Dan Pascu, the author of cjson, deprecates >> >> Matt Billenstein's cjson 1.0.6 because Matt's patch parses the JSON >> >> twice, which makes it twice as slow. This will still be faster than >> >> all alternatives in certain circumstances. You will not find Matt's >> >> cjson on the cheeseshop, only on Matt's site. >> >> >> On Mar 23, 10:55 am, Andy Freeman wrote: >> >> >> > According >> >> > tohttp://blog.metaoptimize.com/2009/03/22/fast-deserialization-in-python/ >> >> > , simplejson is significantly slower than cjson. >> >> >> > I've created an issue requesting the fastest possible json >> >> > athttp://code.google.com/p/googleappengine/issues/detail?id=1174.-Hide >> >> > quoted text - >> >> >> - Show quoted text - >> >> -- >> It is better to be wrong than to be vague. >> — Freeman Dyson- Hide quoted text - >> >> - Show quoted text - > > > -- It is better to be wrong than to be vague. — Freeman Dyson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Files not updating?
You have a limit of 250 times update a day. See your quota page near the bottom. I had a problem too that not all the new static files where visible after the upload. That only happened when I did an update for the default version. Now I alternate between two version numbers and only update the one that is not the default. And then make that the default Now the new static files are always present. Marzia told me that it should be possible to update the default version. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] django form problems
I am using app engine patch sample and using django template forms.
Corresponding to the user form, I have created the class as given
below
class CreateUserForm(forms.Form):
wf_first_name = forms.CharField()
wf_last_name = forms.CharField()
wf_company = forms.CharField()
wf_title = forms.CharField()
wf_street_address = forms.CharField()
The template form's field are First Name [id=wf_first_name], Last Name
[wf_last_name] and so on.
My problem is that if I submit the form without editing an field on
the form then inside the function that is handling both the GET and
the POST request, as shown
existingUserForm = CreateUserForm(request.POST)
userDetails = {}
if(request.method =="POST"):
if existingUserForm.is_valid():
The last line always fails
On the other hand if i submit the form while editing some fields then
the above code works
What's going wrong here?
Thanks
Arnie
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: memcache, What types can it contain?
Odd, thanks for the heads up. Found out, as part of my bootstrapping process, i was storing the request handler in the memcache; a nono it seems. :o On Mar 24, 2009 7:22 AM, "David Wilson" wrote: Looks like you're trying to pickle a cStringIO object. That won't work. Options are: find the code using cStringIO and convert to using StringIO (I'm guessing this will work fine), or consider implementing __getstate__ [1] instead. David [1] http://www.python.org/doc/2.5.2/lib/pickle-inst.html 2009/3/24 Lee Olayvar : > Can memcache store semi-complex object instances? Or is it basically limited > to only simple data... -- It is better to be wrong than to be vague. — Freeman Dyson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to do editing??
I am using the app engine patch sample that uses django templates and if you want to get values from a submitted form then you need to create an instance of the form like this class UserProfileForm(forms.Form): wf_first_name = forms.CharField() wf_last_name = forms.CharField() wf_title = forms.CharField() wf_street_address = forms.CharField() and inside the post [request.method=='POST'] you will go like this existingUserForm = UserProfileForm(request.POST) if(request.method =="POST"): if existingUserForm.is_valid(): --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How to do editing??
Does there exists a way of accessing the submitted form fields without creating an instance of the same like we do in non django based project like self.request.get(fieldname) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Accessing the datastore remotely with remote_api
Accessing the datastore remotely with remote_api won't work under *.appspot.com. sock has no attribute 'getdefaulttimeout'. Best Regards Tom Wu --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Accessing the datastore remotely with remote_api
Sorry, *socket* has no attribute 'getdefaulttimeout'. 2009/3/25 Tom Wu > > Accessing the datastore remotely with remote_api won't work under > *.appspot.com. > > sock has no attribute 'getdefaulttimeout'. > > > Best Regards > Tom Wu > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] getlistfeed fails
Hi guys,
I'm really perplexed by this one. I added the urlfetch fix and the
run_on_appengine but am not sure if I did it write. Without the
urlfetch fix it says that the port is not defined.
I am using the following code in GAE i get a 404 not found error:
===
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/
Resources/GoogleAppEngine-default.bundle/Contents/Resources/
google_appengine/google/appengine/ext/webapp/__init__.py", line
500, in __call__
handler.post(*groups)
File "/Users/Dean/Sites/helloworld/helloworld.py", line
134, in post
a = validateUser(email,pwd)
File "/Users/Dean/Sites/helloworld/helloworld.py", line
95, in validateUser
feed = gd_client.GetListFeed(sKey,wKeyUser,query = qga)
File "/Users/Dean/Sites/helloworld/gdata/spreadsheet/
service.py", line 249, in GetListFeed
converter=gdata.spreadsheet.SpreadsheetsListFeedFromString)
File "/Users/Dean/Sites/helloworld/gdata/service.py", line
1019, in Get
'reason': server_response.reason, 'body': result_body}
RequestError: {'status': 404, 'body': '\n
\nNot Found \n\n
\nNot Found
\nError 404
\n\n\n', 'reason': ''}
===
If I do the same thing in native python with the client library it
works.
any ideas? I've changed the usernames and passwords in the pasted
code.
import os
import cgi
import string
from google.appengine.api import users
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.ext.webapp import template
from google.appengine.ext import db
import wsgiref.handlers
try:
from xml.etree import ElementTree
except ImportError:
from elementtree import ElementTree
import gdata.service
import gdata.alt.appengine
import atom.service
import gdata.spreadsheet
import gdata.spreadsheet.service
import atom
import gdata.urlfetch
# Use urlfetch instead of httplib
gdata.service.http_request_handler = gdata.urlfetch
import settings
import hashlib
from random import choice
from base64 import b64decode, b64encode
#b64hash = "OJF6H4KdxFLgLu+oTDNFodCEfMA=";
sKey = '';
wKeyUser = 'od6';
wKeyLogin = 'od4';
wKeyTut = 'od7';
wKeyTutList = 'od5';
def GenPasswd(length=8, chars=string.letters+string.digits):
return ''.join([choice(chars) for i in range(length)])
def utf16tobin(s):
return s.encode('hex')[4:].decode('hex')
def checkhash(hashval, unhashed):
b64salt = "123456789012345678901234"
binsalt = b64decode(b64salt)
password_string = unhashed.encode("utf16")
password_string = utf16tobin(password_string)
m1 = hashlib.sha1()
# Pass in salt
m1.update(binsalt + password_string)
# Pass in password
# B64 encode the binary digest
if b64encode(m1.digest()) == hashval:
return True
else:
return False
def GSheetService():
client = gdata.service.GDataService()
gdata.alt.appengine.run_on_appengine(client)
gd_client = gdata.spreadsheet.service.SpreadsheetsService
(sKey,wKeyUser);
gd_client.email = 'test...@gmail.com';
gd_client.password = '';
return gd_client;
def PrintFeed(feed):
retstr = ""
for i, entry in enumerate(feed.entry):
if isinstance(feed, gdata.spreadsheet.SpreadsheetsCellsFeed):
print '%s %s %s' % (i, entry.title.text,
entry.content.text)
elif isinstance(feed, gdata.spreadsheet.SpreadsheetsListFeed):
for key in entry.custom:
retstr += ' %s: %s' % (key,
entry.custom[key].text)
retstr += 'done',
else:
retstr += '%s %s' % (i, entry.title.text)
return retstr
def validateUser(email,pwd):
gd_client = GSheetService();
qga = gdata.spreadsheet.service.ListQuery();
qga.sq = "email==" + email;
qga.orderby = "column:lastname"
feed = gd_client.GetListFeed(sKey,wKeyUser,query = qga)
if len(feed.entry)==1:
#we have one entry now lets get the password hash to compare
and the
return checkhash(qga.entry[0].custom['password'].text,pwd)
class MainPage(webapp.RequestHandler):
def get(self):
template_values = {
'error': '',
}
path = os.path.join(os.path.dirname(__file__), 'login.html')
self.response.out.write(template.render(path, template_values))
class LoginPage(webapp.RequestHandler):
def post(self):
# Initialize a client to talk to Google Data API services.
email = str(self.request.POST['email'])
pwd = str(self.request.POST['pwd'])
if email == "" or pw
[google-appengine] Does anyone know how to implement a file upload progress bar
I am trying to add a file update functionality to my application and it works fine. However, when loading large files there is no way to tell the user about the progress of the file upload process. I have seen some postings elsewhere about how to do this using PHP and AJAX, but nothing that would seem to work with GAE. Anyone know how to do this in GAE and some sample code? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Files not updating?
thank you very much. :) that does seem to fix the problem. On Tue, Mar 24, 2009 at 5:54 PM, djidjadji wrote: > > You have a limit of 250 times update a day. See your quota page near the > bottom. > > I had a problem too that not all the new static files where visible > after the upload. > > That only happened when I did an update for the default version. > Now I alternate between two version numbers and only update the one > that is not the default. > And then make that the default > Now the new static files are always present. > > Marzia told me that it should be possible to update the default version. > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Cold app startup stats in case you're curious
Useful stuff, thanks!
Just yesterday I was looking into this same issue. I was wondering if
plain 0.96 Django use (without appengine helper or patch) would be
significantly faster, and your post really makes me want to try that.
I changed the handling of a single url in my app to be a "plain" GAE
(webapp.WSGIApplication) handler. This URL was for a cron-triggered
task that runs every minute. In the absence of other traffic this
resulted in a cold start every minute and this processing this trivial
request now only uses approx. 200 ms CPU quota. When it was part of
the main Django app, it would take over 2000 ms CPU quota.
BTW I'm fairly convinced that memcached data doesn't get flushed out.
On Mar 24, 6:27 am, cz wrote:
> I added some timing code to see how expensive a cold app startup is.
> This is a Django1.x app so YMMV.
>
> The typical numbers are:
>
> Zipimport of Django: 180ms
> Appengine-django-helper monkeypatching: 430ms
> Misc imports, app-specific patching, etc: 20ms
>
> Total app startup: 630ms
>
> This is for startup only, not the time it takes to then process the
> request, so this is a fixed expense. Also, this is real time not cpu
> quota time which is much higher.
>
> The biggest hog is the django-helper monkeypatcher. I'm eventually
> going to try appengine-patch with the hope that it will be faster.
>
> Additionally, on a cold startup my app processes a request in about
> 500ms, but on a warm app it only takes 10ms due to some aggressive
> caching. I think that when all your app instances expire so does all
> it's memcached data.
>
> If anybody has done something similar and found different numbers
> please post the results and let me know what I did wrong.
>
> Here's the main.py I'm using in case you'd like to point out any
> errors in my methodology:
>
>
> from time import clock
> t_main_start = clock()
>
> # Standard Python imports.
> import os
> import sys
> import logging
> import settings
>
> logging.getLogger().setLevel(settings.LOGGING_LEVEL)
>
> # Fix (hack) for missing unlink
> if os.name == 'nt':
> os.unlink = lambda: None
>
> # TODO (after gathering stats): remove this since appengine_django
> does it for us
> # Import Django 1.x from a zipfile and get rid of v.96.
> t_zip_start = clock()
> django_zip_path = os.path.abspath('django.zip')
> if django_zip_path not in sys.path:
> sys.path.insert(0, django_zip_path)
> for k in [k for k in sys.modules if k.startswith('django')]:
> del sys.modules[k]
> # TODO (after gathering stats): remove this line
> from django import template # provoke django zipimport
> t_zip = (clock() - t_zip_start) * 1000
>
> # Install appengine_django. Patches Django 1.x for use with GAE
> t_patch_start = clock()
> from appengine_django import InstallAppengineHelperForDjango
> InstallAppengineHelperForDjango()
> t_patch = (clock() - t_patch_start) * 1000
>
> # Patch the appengine_django patch to fix User object creation
> # Remove this when switching to appengine-patch
> from monkeypatch import patch
> patch()
>
> # Google App Engine imports.
> from google.appengine.ext.webapp import util
>
> # Import the part of Django that we use here.
> import django.core.handlers.wsgi
>
> t_main = (clock() - t_main_start) * 1000
> logging.info('[TIME] main.py startup: %.1fms (zipimport: %.1fms,
> appengine-django: %.1fms)' % (t_main,t_zip,t_patch))
>
> def real_main():
> t_request_start = clock()
> # Create a Django application for WSGI.
> application = django.core.handlers.wsgi.WSGIHandler()
>
> # Run the WSGI CGI handler with that application.
> util.run_wsgi_app(application)
> t_request = (clock() - t_request_start) * 1000
> logging.info('[TIME] request: %.1fms' % t_request)
>
> def profile_main():
> # Rename this to main() for profiling
> import cProfile, pstats
> prof = cProfile.Profile()
> prof = prof.runctx("real_main()", globals(), locals())
> print ""
> stats = pstats.Stats(prof)
> stats.sort_stats("time") # Or cumulative
> stats.print_stats(80) # 80 = how many to print
> # The rest is optional.
> #stats.print_callees()
> #stats.print_callers()
> print ""
>
> main = real_main
>
> if __name__ == '__main__':
> main()
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
