Re: [web2py] Re: Suggestion - a pickled DAL field
You could monkey patch it in 0.py in models, perhaps. On Thursday, July 26, 2012 1:10:43 AM UTC-7, Omri Har-Shemesh wrote: I use the pickled field to store a dictionary. Since dictionary is not defined as a serializable type for the as_dict function, it is not returned when calling to as_dict. Adding it (dal.py line 6183) solved my problem, but I would like to keep this change when I update my web2py next. Is there a different way (without changing the dal) to achieve this? Could you add it to the dal.py? Is it without risks? Thanks, Omri On Thu, Jul 26, 2012 at 9:46 AM, Omri omr...@gmail.com javascript:wrote: So I tried it out, and it works great, except for one thing - when I try to use record.as_dict() to get the structure which I can return to a JSON-RPC call which I normally use to communicate with my client, it simply does not return the field. I tried to print it out and I see very clearly that as long as it is a Row object there is no problem, but as soon as I run .as_dict() it disappears. My definitions are as follows: from gluon.dal import SQLCustomType pickled = SQLCustomType( type = 'text', native = 'text', encoder = (lambda x: pickle.dumps(x)), decoder = (lambda x: pickle.loads(x)) ) then I use db.define_table(my_table, Field(pickled_field, type=pickled) ) but when I fetch the record it does not go to as_dict() (actually I use as_list() but both don't work). Any suggestions? Is there a way to teach as_dict to interpret this field type? Best wishes, Omri On Wed, Jul 25, 2012 at 9:20 PM, howesc how...@umich.edu javascript:wrote: For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/** books/default/chapter/29/6#**Custom-Field-types-(**experimental)http://web2py.com/books/default/chapter/29/6#Custom-Field-types-%28experimental%29 . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri -- --
Re: [web2py] Re: Suggestion - a pickled DAL field
Hi Derek, thanks! the as_dict method has already been fixed in trunk to handle my use-case. best wishes, Omri On Sat, Aug 11, 2012 at 12:26 AM, Derek sp1d...@gmail.com wrote: You could monkey patch it in 0.py in models, perhaps. On Thursday, July 26, 2012 1:10:43 AM UTC-7, Omri Har-Shemesh wrote: I use the pickled field to store a dictionary. Since dictionary is not defined as a serializable type for the as_dict function, it is not returned when calling to as_dict. Adding it (dal.py line 6183) solved my problem, but I would like to keep this change when I update my web2py next. Is there a different way (without changing the dal) to achieve this? Could you add it to the dal.py? Is it without risks? Thanks, Omri On Thu, Jul 26, 2012 at 9:46 AM, Omri omr...@gmail.com wrote: So I tried it out, and it works great, except for one thing - when I try to use record.as_dict() to get the structure which I can return to a JSON-RPC call which I normally use to communicate with my client, it simply does not return the field. I tried to print it out and I see very clearly that as long as it is a Row object there is no problem, but as soon as I run .as_dict() it disappears. My definitions are as follows: from gluon.dal import SQLCustomType pickled = SQLCustomType( type = 'text', native = 'text', encoder = (lambda x: pickle.dumps(x)), decoder = (lambda x: pickle.loads(x)) ) then I use db.define_table(my_table, Field(pickled_field, type=pickled) ) but when I fetch the record it does not go to as_dict() (actually I use as_list() but both don't work). Any suggestions? Is there a way to teach as_dict to interpret this field type? Best wishes, Omri On Wed, Jul 25, 2012 at 9:20 PM, howesc how...@umich.edu wrote: For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/**books** /default/chapter/29/6#**Custom-**Field-types-(**experimental)http://web2py.com/books/default/chapter/29/6#Custom-Field-types-%28experimental%29 . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri -- -- --
Re: [web2py] Re: Suggestion - a pickled DAL field
aww, I love to see a cute monkey patch. On Friday, August 10, 2012 3:28:03 PM UTC-7, Omri Har-Shemesh wrote: Hi Derek, thanks! the as_dict method has already been fixed in trunk to handle my use-case. best wishes, Omri On Sat, Aug 11, 2012 at 12:26 AM, Derek sp1...@gmail.com javascript:wrote: You could monkey patch it in 0.py in models, perhaps. On Thursday, July 26, 2012 1:10:43 AM UTC-7, Omri Har-Shemesh wrote: I use the pickled field to store a dictionary. Since dictionary is not defined as a serializable type for the as_dict function, it is not returned when calling to as_dict. Adding it (dal.py line 6183) solved my problem, but I would like to keep this change when I update my web2py next. Is there a different way (without changing the dal) to achieve this? Could you add it to the dal.py? Is it without risks? Thanks, Omri On Thu, Jul 26, 2012 at 9:46 AM, Omri omr...@gmail.com wrote: So I tried it out, and it works great, except for one thing - when I try to use record.as_dict() to get the structure which I can return to a JSON-RPC call which I normally use to communicate with my client, it simply does not return the field. I tried to print it out and I see very clearly that as long as it is a Row object there is no problem, but as soon as I run .as_dict() it disappears. My definitions are as follows: from gluon.dal import SQLCustomType pickled = SQLCustomType( type = 'text', native = 'text', encoder = (lambda x: pickle.dumps(x)), decoder = (lambda x: pickle.loads(x)) ) then I use db.define_table(my_table, Field(pickled_field, type=pickled) ) but when I fetch the record it does not go to as_dict() (actually I use as_list() but both don't work). Any suggestions? Is there a way to teach as_dict to interpret this field type? Best wishes, Omri On Wed, Jul 25, 2012 at 9:20 PM, howesc how...@umich.edu wrote: For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/**books** /default/chapter/29/6#**Custom-**Field-types-(**experimental)http://web2py.com/books/default/chapter/29/6#Custom-Field-types-%28experimental%29 . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri -- -- --
Re: [web2py] Re: Suggestion - a pickled DAL field
So I tried it out, and it works great, except for one thing - when I try to use record.as_dict() to get the structure which I can return to a JSON-RPC call which I normally use to communicate with my client, it simply does not return the field. I tried to print it out and I see very clearly that as long as it is a Row object there is no problem, but as soon as I run .as_dict() it disappears. My definitions are as follows: from gluon.dal import SQLCustomType pickled = SQLCustomType( type = 'text', native = 'text', encoder = (lambda x: pickle.dumps(x)), decoder = (lambda x: pickle.loads(x)) ) then I use db.define_table(my_table, Field(pickled_field, type=pickled) ) but when I fetch the record it does not go to as_dict() (actually I use as_list() but both don't work). Any suggestions? Is there a way to teach as_dict to interpret this field type? Best wishes, Omri On Wed, Jul 25, 2012 at 9:20 PM, howesc how...@umich.edu wrote: For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/** books/default/chapter/29/6#**Custom-Field-types-(**experimental)http://web2py.com/books/default/chapter/29/6#Custom-Field-types-%28experimental%29 . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri -- --
Re: [web2py] Re: Suggestion - a pickled DAL field
I use the pickled field to store a dictionary. Since dictionary is not defined as a serializable type for the as_dict function, it is not returned when calling to as_dict. Adding it (dal.py line 6183) solved my problem, but I would like to keep this change when I update my web2py next. Is there a different way (without changing the dal) to achieve this? Could you add it to the dal.py? Is it without risks? Thanks, Omri On Thu, Jul 26, 2012 at 9:46 AM, Omri omri...@gmail.com wrote: So I tried it out, and it works great, except for one thing - when I try to use record.as_dict() to get the structure which I can return to a JSON-RPC call which I normally use to communicate with my client, it simply does not return the field. I tried to print it out and I see very clearly that as long as it is a Row object there is no problem, but as soon as I run .as_dict() it disappears. My definitions are as follows: from gluon.dal import SQLCustomType pickled = SQLCustomType( type = 'text', native = 'text', encoder = (lambda x: pickle.dumps(x)), decoder = (lambda x: pickle.loads(x)) ) then I use db.define_table(my_table, Field(pickled_field, type=pickled) ) but when I fetch the record it does not go to as_dict() (actually I use as_list() but both don't work). Any suggestions? Is there a way to teach as_dict to interpret this field type? Best wishes, Omri On Wed, Jul 25, 2012 at 9:20 PM, howesc how...@umich.edu wrote: For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/** books/default/chapter/29/6#**Custom-Field-types-(**experimental)http://web2py.com/books/default/chapter/29/6#Custom-Field-types-%28experimental%29 . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri -- --
[web2py] Re: Suggestion - a pickled DAL field
Thanks! I will try it out :) On Wednesday, July 25, 2012 3:55:39 AM UTC+2, Anthony wrote: On Tuesday, July 24, 2012 5:58:29 PM UTC-4, Derek wrote: Make it a computed field? I'm not sure that would be helpful in this case. A computed field computes its value automatically based on other fields in the record, but in this case, he needs to pickle an object, which is not one of the other fields. Also, it needs to be unpickled when queried, which a computed field wouldn't handle. Anthony --
Re: [web2py] Re: Suggestion - a pickled DAL field
On 07/24/2012 06:55 PM, Anthony wrote: On Tuesday, July 24, 2012 5:58:29 PM UTC-4, Derek wrote: Make it a computed field? I'm not sure that would be helpful in this case. A computed field computes its value automatically based on other fields in the record, but in this case, he needs to pickle an object, which is not one of the other fields. Also, it needs to be unpickled when queried, which a computed field wouldn't handle. I haven't used one, but I remember coming across it in the manual. Don't you just pass a lambda when you create the field. It would seem that you could do anything you want to the value without consulting other fields. --
Re: [web2py] Re: Suggestion - a pickled DAL field
Make it a computed field? I'm not sure that would be helpful in this case. A computed field computes its value automatically based on other fields in the record, but in this case, he needs to pickle an object, which is not one of the other fields. Also, it needs to be unpickled when queried, which a computed field wouldn't handle. I haven't used one, but I remember coming across it in the manual. Don't you just pass a lambda when you create the field. It would seem that you could do anything you want to the value without consulting other fields. Are you talking about a computed field? The compute function only runs if you *don't* pass a value to the field -- it calculates a value for the field based on the values of *other* fields in the record. Also, it doesn't do anything to the value upon query, so wouldn't help with the unpickling in this case. Anthony --
[web2py] Re: Suggestion - a pickled DAL field
For what it is worth i have used custom fields for this type of thing with great success! On Tuesday, July 24, 2012 8:20:03 AM UTC-7, Anthony wrote: You could use a SQLCustomType field: http://web2py.com/books/default/chapter/29/6#Custom-Field-types-(experimental) . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri --
[web2py] Re: Suggestion - a pickled DAL field
You could use a SQLCustomType field: http://web2py.com/books/default/chapter/29/6#Custom-Field-types-(experimental) . Anthony On Tuesday, July 24, 2012 8:34:07 AM UTC-4, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri --
[web2py] Re: Suggestion - a pickled DAL field
Make it a computed field? On Tuesday, July 24, 2012 5:34:07 AM UTC-7, Omri Har-Shemesh wrote: Hi web2pyers, very often, I have a field in the table which has to hold a complicated value (most often numpy arrays). The way I implement it is that I use text as the type of field, and then simply pickle.dumps(my_value) when I save the field, and then pickle.loads(value_from_db) to access the field again. My suggestion is simple - create field that automatically pickles the values it gets and unpickles them on extraction. Is this already implemented? Do you have other suggestions on how to implement this so that I won't need to pickle every time I access the table? Best wishes, Omri --
[web2py] Re: Suggestion - a pickled DAL field
On Tuesday, July 24, 2012 5:58:29 PM UTC-4, Derek wrote: Make it a computed field? I'm not sure that would be helpful in this case. A computed field computes its value automatically based on other fields in the record, but in this case, he needs to pickle an object, which is not one of the other fields. Also, it needs to be unpickled when queried, which a computed field wouldn't handle. Anthony --
