I'm working from Version 2.3.2 (2012-12-17 15:03:30) stable Here's the table in question (I've commented-out the custom_qualifier for the string fields):
db.define_table('t_run', Field('f_trial', type='reference t_trial', label=T('Trial')), Field('f_when', type='date', label=T('When')), Field('f_dog', type='reference t_dog', label=T('Dog')), Field('f_name', type='string',#custom_qualifier={'indexed':False}, label=T('Name')), Field('f_breed', type='string',#custom_qualifier={'indexed':False}, label=T('Breed')), Field('f_jump_height', type='integer',custom_qualifier={'indexed':False }, label=T('Jump Height')), Field('f_level', type='string',#custom_qualifier={'indexed':False}, label=T('Level')), Field('f_class', type='string',#custom_qualifier={'indexed':False}, label=T('Class')), Field('f_pref', type='boolean',custom_qualifier={'indexed':False}, label=T('Preferred')), Field('f_armband', type='string',#custom_qualifier={'indexed':False}, label=T('Armband')), Field('f_yards', type='integer',custom_qualifier={'indexed':False}, label=T('Yards')), Field('f_sct', type='integer',custom_qualifier={'indexed':False}, label=T('SCT')), Field('f_judge', type='string',#custom_qualifier={'indexed':False}, label=T('Judge')), Field('f_score', type='integer',custom_qualifier={'indexed':False}, label=T('Score')), Field('f_time', type='double',custom_qualifier={'indexed':False}, label=T('Time')), Field('f_faults', type='string',#custom_qualifier={'indexed':False}, label=T('Faults')), Field('f_handler', type='string',#custom_qualifier={'indexed':False}, label=T('Handler')), Field('f_order', type='integer', label=T('Order')), auth.signature, format='%(f_key)s', migrate=settings.migrate) P.S. Turning off the indexes DOES make a difference; with the changes above, instead of using up over 30% of my quota, I'm "only" using 23%. If I can get the strings unindexed, and take out the web2py-supplied fields (created_by & _on, modified_by & _on, and maybe is_active), that should get it down to something manageable. (Not clear on how to handle the web2py-supplied fields, as I don't know what parameters were using in making them, making it difficult to know just how to "supply my own definitions".) On Tuesday, February 26, 2013 6:24:21 PM UTC-5, howesc wrote: > > your line numbers are off from mine so i'm having trouble making sense of > this. :( > > can you send your model definition so i can see what you are working > with? i think then i can line it up with the dal version i have here > (which was trunk from HG as of saturday AM PST) > > cfh > > On Tuesday, February 26, 2013 3:18:13 PM UTC-8, Scott Hunter wrote: >> >> Sorry about that. I've fixed it, and now I get the following: >> >> In FILE: /base/data/home/apps/s~sbhweb2py/1.365574604253984974/ >> applications/ppt_demo/models/db_wizard.py >> >> >> Traceback (most recent call last): >> File >> "/base/data/home/apps/s~sbhweb2py/1.365574604253984974/gluon/restricted.py" >> , line 212, in restricted >> exec ccode in environment >> File >> "/base/data/home/apps/s~sbhweb2py/1.365574604253984974/applications/ppt_demo/models/db_wizard.py" >> , line 198, in <module> >> migrate=settings.migrate) >> File >> "/base/data/home/apps/s~sbhweb2py/1.365574604253984974/gluon/dal.py",line >> 7189, in define_table >> table = self.lazy_define_table(tablename,*fields,**args) >> File >> "/base/data/home/apps/s~sbhweb2py/1.365574604253984974/gluon/dal.py",line >> 7225, in lazy_define_table >> polymodel=polymodel) >> File >> "/base/data/home/apps/s~sbhweb2py/1.365574604253984974/gluon/dal.py",line >> 4368, in create_table >> ftype = self.types[field_type](**attr) >> TypeError: <lambda>() takes no arguments (1 given) >> >> >> This only happens if one of the fields in question is a string; I get no >> error when the unindexed fields are all integer, boolean or double. >> >> - Scott >> >> >> On Tuesday, February 26, 2013 1:04:12 AM UTC-5, howesc wrote: >>> >>> it looks like you have a typo.... "custom_qualifer" vs "custom_qualifier" >>> >>> On Monday, February 25, 2013 6:44:14 PM UTC-8, Scott Hunter wrote: >>>> >>>> I applied the patch, and added custom_qualifiers like so: >>>> >>>> Field('f_name', type='string',custom_qualifer={'indexed':False}, >>>> label=T('Name')), >>>> >>>> >>>> >>>> and this is the error I got: >>>> >>>> In FILE: /base/data/home/apps/s~sbhweb2py/1.365567821359373728/ >>>> applications/ppt_demo/models/db_wizard.py >>>> >>>> >>>> Traceback (most recent call last): >>>> File >>>> "/base/data/home/apps/s~sbhweb2py/1.365567821359373728/gluon/restricted.py" >>>> , line 212, in restricted >>>> exec ccode in environment >>>> File >>>> "/base/data/home/apps/s~sbhweb2py/1.365567821359373728/applications/ppt_demo/models/db_wizard.py" >>>> , line 165, in <module> >>>> label=T('Name')), >>>> TypeError: __init__() got an unexpected keyword argument >>>> 'custom_qualifer' >>>> >>>> >>>> >>>> >>>> On Saturday, February 23, 2013 12:30:48 PM UTC-5, howesc wrote: >>>>> >>>>> Scott, >>>>> >>>>> this is *completely* untested, but here's a proposal: >>>>> - use the (undocumented) field.custom_qualifier property in GAE field >>>>> definitions >>>>> - if you want a field to be unindexed set >>>>> custom_qualifier={'indexed':False} >>>>> - for "properties that don't get listed explicitly"....override those >>>>> default properties with what you want (there are a few techniques for not >>>>> explicitly listing fields, depending on which technique you are using the >>>>> answer here is different. >>>>> >>>>> this patch against HG trunk this AM might work (i say might cause i >>>>> have not tested it). are you willing to experiment with it and let us >>>>> know? >>>>> >>>>> thanks, >>>>> >>>>> christian >>>>> >>>>> On Friday, February 22, 2013 3:53:33 PM UTC-8, Scott Hunter wrote: >>>>>> >>>>>> If I had to guess, I'd say a patch is needed in the loop over the >>>>>> fields in create_table, that would add a new entry to the dict for the >>>>>> field definition based on a new attribute (which would be ignored for >>>>>> anything other than Google's Datastore) which gets added to sql_fields; >>>>>> then migrate_table, which seems to build the table building/altering >>>>>> commands would need to be made to recognize the new entry & add the >>>>>> disable >>>>>> index command. There's not a lot of comments in the code, and I'd be >>>>>> afraid to break something, but might give it a try. >>>>>> >>>>>> - Scott >>>>>> >>>>>> P.S. Not sure of the best way to handle this (what's describe above >>>>>> wouldn't handle it), but it would be nice to be able to turn off the >>>>>> indexes for the fields that don't get listed explicitly in models.db >>>>>> (things like created by & when, modified by & when). >>>>>> >>>>>> On Friday, February 22, 2013 6:14:28 PM UTC-5, howesc wrote: >>>>>>> >>>>>>> thanks for the link. i'll try and take a look this weekend and see >>>>>>> if there is a place for that in the DAL (feel free to open >>>>>>> gluon/dal.py >>>>>>> yourself too). >>>>>>> >>>>>>> yup, i'm paying more for writes right now then i am for instance >>>>>>> hours per day on my largest paid application. :) >>>>>>> >>>>>>> cfh >>>>>>> >>>>>>> On Thursday, February 21, 2013 5:59:06 PM UTC-8, Scott Hunter wrote: >>>>>>>> >>>>>>>> According to >>>>>>>> https://developers.google.com/appengine/docs/python/datastore/indexes#Unindexed_Properties, >>>>>>>> >>>>>>>> "You declare a property unindexed by setting indexed=False in the >>>>>>>> property >>>>>>>> constructor". >>>>>>>> >>>>>>>> One incurs the cost of a write (or 2?) for every property of a >>>>>>>> record that gets written; so, for a table with about 25 fields (once >>>>>>>> you >>>>>>>> include all of the ones web2py adds), inserting 300 records incurs the >>>>>>>> cost >>>>>>>> of 14K+ writes; as the free account has a limit of 50K writes per day, >>>>>>>> that >>>>>>>> is rather limiting. It is easy to turn such indicies back on, but you >>>>>>>> have >>>>>>>> to re-write every record in order to repopulate them. >>>>>>>> >>>>>>>> - Scott >>>>>>>> >>>>>>>> On Thursday, February 21, 2013 4:51:59 PM UTC-5, howesc wrote: >>>>>>>>> >>>>>>>>> i'm being lazy here....do you have the instructions (or link to >>>>>>>>> instructions) from GAE for disabling indexes? i don't have it handy >>>>>>>>> right >>>>>>>>> now.... >>>>>>>>> >>>>>>>>> we can check if the DAL has a secret way to handle it, and/or >>>>>>>>> create a patch to allow it. >>>>>>>>> >>>>>>>>> may i ask what advantage you are hoping to achieve by skipping >>>>>>>>> those indexes? i've left them on for fear of making a bad choice >>>>>>>>> that i >>>>>>>>> can't revert. >>>>>>>>> >>>>>>>>> christian >>>>>>>>> >>>>>>>>> On Thursday, February 21, 2013 5:56:07 AM UTC-8, Scott Hunter >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> As I understand it, the GAE datastore will automatically create >>>>>>>>>> an index for every field of every table (to facilitate queries using >>>>>>>>>> single >>>>>>>>>> fields); those requiring multiple fields also get generated, but >>>>>>>>>> will get >>>>>>>>>> added to index.yaml by the SDK when tested locally. Since these are >>>>>>>>>> created outside of the DAL, I wouldn't expect to be able to remove >>>>>>>>>> them >>>>>>>>>> using it. I also understand that there is a way, with the GAE SDK, >>>>>>>>>> to >>>>>>>>>> disable the index for any given field (by a parameter in the field >>>>>>>>>> definition); is there any way to accomplish this through the DAL, >>>>>>>>>> and if >>>>>>>>>> so, what is it? >>>>>>>>> >>>>>>>>> -- --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.