Scott,
this is *completely* untested, but here's a proposal:
- use the (undocumented) field.custom_qualifier property in GAE field
definitions
- if you want a field to be unindexed set
custom_qualifier={'indexed':False}
- for "properties that don't get listed explicitly"....override those
default properties with what you want (there are a few techniques for not
explicitly listing fields, depending on which technique you are using the
answer here is different.
this patch against HG trunk this AM might work (i say might cause i have
not tested it). are you willing to experiment with it and let us know?
thanks,
christian
On Friday, February 22, 2013 3:53:33 PM UTC-8, Scott Hunter wrote:
>
> If I had to guess, I'd say a patch is needed in the loop over the fields
> in create_table, that would add a new entry to the dict for the field
> definition based on a new attribute (which would be ignored for anything
> other than Google's Datastore) which gets added to sql_fields; then
> migrate_table, which seems to build the table building/altering commands
> would need to be made to recognize the new entry & add the disable index
> command. There's not a lot of comments in the code, and I'd be afraid to
> break something, but might give it a try.
>
> - Scott
>
> P.S. Not sure of the best way to handle this (what's describe above
> wouldn't handle it), but it would be nice to be able to turn off the
> indexes for the fields that don't get listed explicitly in models.db
> (things like created by & when, modified by & when).
>
> On Friday, February 22, 2013 6:14:28 PM UTC-5, howesc wrote:
>>
>> thanks for the link. i'll try and take a look this weekend and see if
>> there is a place for that in the DAL (feel free to open gluon/dal.py
>> yourself too).
>>
>> yup, i'm paying more for writes right now then i am for instance hours
>> per day on my largest paid application. :)
>>
>> cfh
>>
>> On Thursday, February 21, 2013 5:59:06 PM UTC-8, Scott Hunter wrote:
>>>
>>> According to
>>> https://developers.google.com/appengine/docs/python/datastore/indexes#Unindexed_Properties,
>>>
>>> "You declare a property unindexed by setting indexed=False in the property
>>> constructor".
>>>
>>> One incurs the cost of a write (or 2?) for every property of a record
>>> that gets written; so, for a table with about 25 fields (once you include
>>> all of the ones web2py adds), inserting 300 records incurs the cost of 14K+
>>> writes; as the free account has a limit of 50K writes per day, that is
>>> rather limiting. It is easy to turn such indicies back on, but you have to
>>> re-write every record in order to repopulate them.
>>>
>>> - Scott
>>>
>>> On Thursday, February 21, 2013 4:51:59 PM UTC-5, howesc wrote:
>>>>
>>>> i'm being lazy here....do you have the instructions (or link to
>>>> instructions) from GAE for disabling indexes? i don't have it handy right
>>>> now....
>>>>
>>>> we can check if the DAL has a secret way to handle it, and/or create a
>>>> patch to allow it.
>>>>
>>>> may i ask what advantage you are hoping to achieve by skipping those
>>>> indexes? i've left them on for fear of making a bad choice that i can't
>>>> revert.
>>>>
>>>> christian
>>>>
>>>> On Thursday, February 21, 2013 5:56:07 AM UTC-8, Scott Hunter wrote:
>>>>>
>>>>> As I understand it, the GAE datastore will automatically create an
>>>>> index for every field of every table (to facilitate queries using single
>>>>> fields); those requiring multiple fields also get generated, but will get
>>>>> added to index.yaml by the SDK when tested locally. Since these are
>>>>> created outside of the DAL, I wouldn't expect to be able to remove them
>>>>> using it. I also understand that there is a way, with the GAE SDK, to
>>>>> disable the index for any given field (by a parameter in the field
>>>>> definition); is there any way to accomplish this through the DAL, and if
>>>>> so, what is it?
>>>>
>>>>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to web2py+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
diff -r 87162e3fcd8c gluon/dal.py
--- a/gluon/dal.py Fri Feb 22 18:45:26 2013 -0600
+++ b/gluon/dal.py Sat Feb 23 09:26:28 2013 -0800
@@ -4501,6 +4501,9 @@
if isinstance(polymodel,Table) and field.name in
polymodel.fields():
continue
attr = {}
+ if isinstance(field.custom_qaulifier, dict):
+ #this is custom properties to add to the GAE field declartion
+ attr = field.custom_qualifier
field_type = field.type
if isinstance(field_type, SQLCustomType):
ftype = self.types[field_type.native or
field_type.type](**attr)
@@ -4517,10 +4520,10 @@
if field.notnull:
attr = dict(required=True)
referenced = field_type[10:].strip()
- ftype = self.types[field_type[:9]](referenced)
+ ftype = self.types[field_type[:9]](referenced, **attr)
elif field_type.startswith('list:reference'):
if field.notnull:
- attr = dict(required=True)
+ attr['required'] = True
referenced = field_type[15:].strip()
ftype = self.types[field_type[:14]](**attr)
elif field_type.startswith('list:'):
