Re: [web2py] Re: web2py and gae : Delete everything in the datastore
It worked. :) On Sat, Jan 29, 2011 at 6:03 PM, devGS vitali@geniestills.com wrote: Great than. From my experience with that future, it takes time to delete in bulk and the process is is splitted into hours in order to save your CPU time. You can track the process through your datastore panels, task queue, etc. That's all I can comment about. Hopping to hear about your experience. On Jan 29, 7:16 pm, Nathan VanHoudnos nathan...@gmail.com wrote: Yes, I am doing something wrong. I needed to navigate to 'Datastore Admin' instead of 'Datastore Viewer'. The admin pages allows you to delete everything quickly. The viewer page only allows you to delete in chunks of 20. From: http://code.google.com/appengine/docs/python/datastore/entities.html Deleting Entities in Bulk via the Admin Console You can use the Datastore Admin tab of the Admin Console to delete all entities of a kind, or all entities of all kinds, in the default namespace. To enable this feature, simply include the datastore_admin builtin handler in app.yaml: builtins: - datastore_admin: on Adding this builtin enables the Datastore Admin screen in the Data section of the Admin Console. From this screen, you can select the entity kind(s) to delete individually or in bulk, and delete them using the Delete Entities button. Warning! Deleting entities in bulk happens within your application, and thus counts against your quota. This feature is currently experimental. We believe this feature is currently the fastest way to bulk-delete data, but it is not yet stable and you may encounter occasional bugs. Thanks for the help guys! Cheers, Nathan On Sat, Jan 29, 2011 at 9:23 AM, Nathan VanHoudnos nathan...@gmail.com wrote: I tried that again this morning, but it only lets me delete 20 at a time. Am I doing something wrong? (I have hundreds of entries to delete.) Cheers, Nathan On Fri, Jan 28, 2011 at 5:35 PM, howesc how...@umich.edu wrote: if i remember correctly the bulk delete was google's christmas present to us - in the decemeber release, so it's a new feature, but quite handy! -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |-http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |-http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon
Re: [web2py] Re: web2py and gae : Delete everything in the datastore
I tried that again this morning, but it only lets me delete 20 at a time. Am I doing something wrong? (I have hundreds of entries to delete.) Cheers, Nathan On Fri, Jan 28, 2011 at 5:35 PM, howesc how...@umich.edu wrote: if i remember correctly the bulk delete was google's christmas present to us - in the decemeber release, so it's a new feature, but quite handy! -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon
Re: [web2py] Re: web2py and gae : Delete everything in the datastore
Yes, I am doing something wrong. I needed to navigate to 'Datastore Admin' instead of 'Datastore Viewer'. The admin pages allows you to delete everything quickly. The viewer page only allows you to delete in chunks of 20. From: http://code.google.com/appengine/docs/python/datastore/entities.html Deleting Entities in Bulk via the Admin Console You can use the Datastore Admin tab of the Admin Console to delete all entities of a kind, or all entities of all kinds, in the default namespace. To enable this feature, simply include the datastore_admin builtin handler in app.yaml: builtins: - datastore_admin: on Adding this builtin enables the Datastore Admin screen in the Data section of the Admin Console. From this screen, you can select the entity kind(s) to delete individually or in bulk, and delete them using the Delete Entities button. Warning! Deleting entities in bulk happens within your application, and thus counts against your quota. This feature is currently experimental. We believe this feature is currently the fastest way to bulk-delete data, but it is not yet stable and you may encounter occasional bugs. Thanks for the help guys! Cheers, Nathan On Sat, Jan 29, 2011 at 9:23 AM, Nathan VanHoudnos nathan...@gmail.comwrote: I tried that again this morning, but it only lets me delete 20 at a time. Am I doing something wrong? (I have hundreds of entries to delete.) Cheers, Nathan On Fri, Jan 28, 2011 at 5:35 PM, howesc how...@umich.edu wrote: if i remember correctly the bulk delete was google's christmas present to us - in the decemeber release, so it's a new feature, but quite handy! -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program for Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon
[web2py] web2py and gae : Delete everything in the datastore
Hi,
I'm running web2py 1.91.6 on google app engine. I'd like to delete
everything in the datastore and start over.
If you have a quick solution for this, stop reading now and just tell me the
web2py way. :)
There is a StackOverflow thread that I found:
http://stackoverflow.com/questions/1062540/how-to-delete-all-datastore-in-google-app-engine
which assumes that you setup everything in Models.py which uses syntax very
different from the DAL and I can't make heads or tails of it.
I tried using the remote_api to do the deletion:
vanhoudn@gauze:~/workspace/creatia-rubrics/src$ python2.5
~/google_appengine/remote_api_shell.py creatia-rubrics
App Engine remote_api shell
Python 2.5.5 (r255:77872, Sep 14 2010, 17:16:34)
[GCC 4.4.3]
The db, users, urlfetch, and memcache modules are imported.
creatia-rubrics from google.appengine.ext.db import GqlQuery
creatia-rubrics query = GqlQuery(SELECT * FROM allRatings)
Traceback (most recent call last):
File console, line 1, in module
File
/home/vanhoudn/google_appengine.1.4.1/google/appengine/ext/db/__init__.py,
line 2296, in __init__
model_class = class_for_kind(self._proto_query._entity)
File
/home/vanhoudn/google_appengine.1.4.1/google/appengine/ext/db/__init__.py,
line 266, in class_for_kind
raise KindError('No implementation for kind \'%s\'' % kind)
KindError: No implementation for kind 'allRatings'
But it gives me this weird KindError. Before i can get to the
db.delete(query)
step. So how do you access the kinds that you create in models/db.py? I
copied the GQL statement straight from the datastore viewer on
appspot.comso it shouldn't be a syntax error.
The relevant section of my models/db.py is:
db.define_table('allRatings',
Field('timeend','datetime',requires= IS_NOT_EMPTY() ),
Field('timestart','datetime',requires= IS_NOT_EMPTY() ),
Field('assignmentId',requires= IS_NOT_EMPTY() ),
Field('experimentId',requires= IS_NOT_EMPTY() ),
Field('workerId',requires= IS_NOT_EMPTY() ),
Field('studentIds',requires= IS_NOT_EMPTY() ),
Field('rubricCode',requires= IS_NOT_EMPTY() ),
Field('ipaddress' ,default=request.client),
Field('comment',requires= IS_NOT_EMPTY() ),
Field('rawAnswers',requires= IS_NOT_EMPTY() ),
Field('subscores',requires= IS_NOT_EMPTY() )
)
Any help would be appreciated!
Cheers,
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|- http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
Re: [web2py] Re: web2py and gae : Delete everything in the datastore
Thanks for the reply.
To clarify, I want to wipe everything in the production datastore on
Google's servers. If it were only as easy as clearing the development
datastore!
Cheers,
Nathan
On Fri, Jan 28, 2011 at 3:10 PM, devGS vitali@geniestills.com wrote:
Run your web2py app with a command from your GAE library and the flag -
c.
i.e: dev_appserver.py -c C:\web2py
On Jan 28, 9:31 pm, Nathan VanHoudnos nathan...@gmail.com wrote:
Hi,
I'm running web2py 1.91.6 on google app engine. I'd like to delete
everything in the datastore and start over.
If you have a quick solution for this, stop reading now and just tell me
the
web2py way. :)
There is a StackOverflow thread that I found:
http://stackoverflow.com/questions/1062540/how-to-delete-all-datastor...
which assumes that you setup everything in Models.py which uses syntax
very
different from the DAL and I can't make heads or tails of it.
I tried using the remote_api to do the deletion:
vanhoudn@gauze:~/workspace/creatia-rubrics/src$ python2.5
~/google_appengine/remote_api_shell.py creatia-rubrics
App Engine remote_api shell
Python 2.5.5 (r255:77872, Sep 14 2010, 17:16:34)
[GCC 4.4.3]
The db, users, urlfetch, and memcache modules are imported.
creatia-rubrics from google.appengine.ext.db import GqlQuery
creatia-rubrics query = GqlQuery(SELECT * FROM allRatings)
Traceback (most recent call last):
File console, line 1, in module
File
/home/vanhoudn/google_appengine.1.4.1/google/appengine/ext/db/__init__.py
,
line 2296, in __init__
model_class = class_for_kind(self._proto_query._entity)
File
/home/vanhoudn/google_appengine.1.4.1/google/appengine/ext/db/__init__.py
,
line 266, in class_for_kind
raise KindError('No implementation for kind \'%s\'' % kind)
KindError: No implementation for kind 'allRatings'
But it gives me this weird KindError. Before i can get to the
db.delete(query)
step. So how do you access the kinds that you create in models/db.py? I
copied the GQL statement straight from the datastore viewer on
appspot.comso it shouldn't be a syntax error.
The relevant section of my models/db.py is:
db.define_table('allRatings',
Field('timeend','datetime',requires= IS_NOT_EMPTY() ),
Field('timestart','datetime',requires= IS_NOT_EMPTY() ),
Field('assignmentId',requires= IS_NOT_EMPTY() ),
Field('experimentId',requires= IS_NOT_EMPTY() ),
Field('workerId',requires= IS_NOT_EMPTY() ),
Field('studentIds',requires= IS_NOT_EMPTY() ),
Field('rubricCode',requires= IS_NOT_EMPTY() ),
Field('ipaddress' ,default=request.client),
Field('comment',requires= IS_NOT_EMPTY() ),
Field('rawAnswers',requires= IS_NOT_EMPTY() ),
Field('subscores',requires= IS_NOT_EMPTY() )
)
Any help would be appreciated!
Cheers,
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|-http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|- http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
Re: [web2py] Re: Bug? SQLFORM.factory and SQLFORM differ in handling of hidden form fields
Done: http://code.google.com/p/web2py/issues/detail?id=162
http://code.google.com/p/web2py/issues/detail?id=162Thanks!
On Tue, Jan 18, 2011 at 4:29 PM, Massimo Di Pierro
massimo.dipie...@gmail.com wrote:
This is a bug. Would you please post your bug report email on
googlecode? I will fix it asap.
On Jan 18, 12:23 pm, Nathan VanHoudnos nathan...@gmail.com wrote:
Hi,
A minimal example:
likertRightAnswer = ['Yes', 'No', 'Impossible to tell']
form = SQLFORM.factory(
Field('rightAnswer',
widget=horizontal_radios,
requires=IS_IN_SET(likertRightAnswer,
error_message=T('Please choose a
response.'))),
hidden={'timestart':request.now.strftime('%Y-%m-%d %H:%M:%S')}
)
In this case, BEAUTIFY(request.post_vars) will include, timestart and
rightAnswer, but BEAUTIFY(form.vars) will only include rightAnswer.
If we instead setup the appropriate db as db.ratings, and then do:
likertRightAnswer = ['Yes', 'No', 'Impossible to tell']
form = SQLFORM(db.ratings,
hidden={'timestart':request.now.strftime('%Y-%m-%d %H:%M:%S')}
)
In this case, both BEAUTIFY(request.post_vars) and BEAUTIFY(form.vars)
will
both include timestart and rightAnswer.
The second behavior is more expected at least to me.
Right now, I'm using request.post_vars.timestart to access the value
after
the form.accepts call, but this feels like I'm hacking something
together.
And it essentially circumvents the security feature discussed in this
thread:
http://groups.google.com/group/web2py/browse_thread/thread/ab21d9d216...
So is this a bug? Or should I be doing something different? (My
application
creates a bunch of little temporary forms and I don't want to bloat
the data-store with them.)
Cheers,
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|-http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|- http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
[web2py] Bug? SQLFORM.factory and SQLFORM differ in handling of hidden form fields
Hi,
A minimal example:
likertRightAnswer = ['Yes', 'No', 'Impossible to tell']
form = SQLFORM.factory(
Field('rightAnswer',
widget=horizontal_radios,
requires=IS_IN_SET(likertRightAnswer,
error_message=T('Please choose a
response.'))),
hidden={'timestart':request.now.strftime('%Y-%m-%d %H:%M:%S')}
)
In this case, BEAUTIFY(request.post_vars) will include, timestart and
rightAnswer, but BEAUTIFY(form.vars) will only include rightAnswer.
If we instead setup the appropriate db as db.ratings, and then do:
likertRightAnswer = ['Yes', 'No', 'Impossible to tell']
form = SQLFORM(db.ratings,
hidden={'timestart':request.now.strftime('%Y-%m-%d %H:%M:%S')}
)
In this case, both BEAUTIFY(request.post_vars) and BEAUTIFY(form.vars) will
both include timestart and rightAnswer.
The second behavior is more expected at least to me.
Right now, I'm using request.post_vars.timestart to access the value after
the form.accepts call, but this feels like I'm hacking something together.
And it essentially circumvents the security feature discussed in this
thread:
http://groups.google.com/group/web2py/browse_thread/thread/ab21d9d21693ee2e/1f6781206e956405?lnk=gstq=hidden+form+fields+not+accepted#1f6781206e956405
So is this a bug? Or should I be doing something different? (My application
creates a bunch of little temporary forms and I don't want to bloat
the data-store with them.)
Cheers,
--
Nathan VanHoudnos
|- Statistics Public Policy PhD student
|- Program for Interdisciplinary Education Research (PIER) Fellowship
|- Carnegie Mellon University
|- http://www.andrew.cmu.edu/user/nmv
Neglect of mathematics works injury to all knowledge,
since he who is ignorant of it cannot know the other
sciences or the things of this world. -- Roger Bacon
[web2py] Can't install gae remote_api using web2py
Hi, Attempting to follow the directions here: http://code.google.com/appengine/docs/python/tools/uploadingdata.html So I added to my app.yaml: builtins: - remote_api: on And then I try to to run appcfg.py update: vanho...@gauze:~/google_appengine$ python2.5 appcfg.py update ../workspace/nathanvan-creatia/src/ Error parsing yaml file: Unexpected attribute 'builtins' for object of type class 'google.appengine.api.appinfo.AppInfoExternal'. in ../workspace/nathanvan-creatia/src/app.yaml, line 58, column 1 The whole app.yaml file follows. (It is just the standard web2py file.): application: nathanvan-creatia version: 1 api_version: 1 runtime: python derived_file_type: - python_precompiled handlers: - url: /_ah/stats.* script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py login: admin - url: /(?Pa.+?)/static/(?Pb.+) static_files: applications/\1/static/\2 upload: applications/(.+?)/static/(.+) secure: optional expiration: 90d - url: /_ah/admin/.* script: $PYTHON_LIB/google/appengine/ext/admin login: admin - url: /_ah/queue/default script: gaehandler.py login: admin - url: .* script: gaehandler.py secure: optional admin_console: pages: - name: Appstats url: /_ah/stats skip_files: | ^(.*/)?( (app\.yaml)| (app\.yml)| (index\.yaml)| (index\.yml)| (#.*#)| (.*~)| (.*\.py[co])| (.*/RCS/.*)| (\..*)| ((admin|examples|welcome)\.tar)| (applications/(admin|examples)/.*)| (applications/.*?/databases/.*) | (applications/.*?/errors/.*)| (applications/.*?/cache/.*)| (applications/.*?/sessions/.*)| )$ builtins: - remote_api: on I must be missing something obvious. Do I need to manually add some files to the web2py instance for this to work? Cheers, -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program in Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon
Re: [web2py] Re: Can't install gae remote_api using web2py
I do have the most recent version of the App Engine SDK loaded and I've double checked I'm calling the correct version. I'll ask my question on the GAE forum as Richard suggested. (I had thought at first that this was a web2py issue. I'm new to both, so I'm often unsure if my question should be an App Engine or web2py question.) Thanks for the response though! Cheers, Nathan On Mon, Oct 18, 2010 at 8:24 PM, howesc how...@umich.edu wrote: perhaps silly question...but did you upgrade the google SDK to the latest version? being that it's a new feature you want to make sure you are using the latest SDK to do the upload. (perhaps you have an old version of appcfg.py on your path?) cfh On Oct 18, 11:21 am, Nathan VanHoudnos nathan...@gmail.com wrote: Hi, Attempting to follow the directions here: http://code.google.com/appengine/docs/python/tools/uploadingdata.html So I added to my app.yaml: builtins: - remote_api: on And then I try to to run appcfg.py update: vanho...@gauze:~/google_appengine$ python2.5 appcfg.py update ../workspace/nathanvan-creatia/src/ Error parsing yaml file: Unexpected attribute 'builtins' for object of type class 'google.appengine.api.appinfo.AppInfoExternal'. in ../workspace/nathanvan-creatia/src/app.yaml, line 58, column 1 The whole app.yaml file follows. (It is just the standard web2py file.): application: nathanvan-creatia version: 1 api_version: 1 runtime: python derived_file_type: - python_precompiled handlers: - url: /_ah/stats.* script: $PYTHON_LIB/google/appengine/ext/appstats/ui.py login: admin - url: /(?Pa.+?)/static/(?Pb.+) static_files: applications/\1/static/\2 upload: applications/(.+?)/static/(.+) secure: optional expiration: 90d - url: /_ah/admin/.* script: $PYTHON_LIB/google/appengine/ext/admin login: admin - url: /_ah/queue/default script: gaehandler.py login: admin - url: .* script: gaehandler.py secure: optional admin_console: pages: - name: Appstats url: /_ah/stats skip_files: | ^(.*/)?( (app\.yaml)| (app\.yml)| (index\.yaml)| (index\.yml)| (#.*#)| (.*~)| (.*\.py[co])| (.*/RCS/.*)| (\..*)| ((admin|examples|welcome)\.tar)| (applications/(admin|examples)/.*)| (applications/.*?/databases/.*) | (applications/.*?/errors/.*)| (applications/.*?/cache/.*)| (applications/.*?/sessions/.*)| )$ builtins: - remote_api: on I must be missing something obvious. Do I need to manually add some files to the web2py instance for this to work? Cheers, -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program in Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |-http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon -- Nathan VanHoudnos |- Statistics Public Policy PhD student |- Program in Interdisciplinary Education Research (PIER) Fellowship |- Carnegie Mellon University |- http://www.andrew.cmu.edu/user/nmv Neglect of mathematics works injury to all knowledge, since he who is ignorant of it cannot know the other sciences or the things of this world. -- Roger Bacon
