Pickle model fields with cache...
I have insert a small cache mechanism. So pickle.loads() would only used for
the
first _get_value():
-
class Preference(models.Model):
def __init__(self, *args, **kwargs):
self._cache = {}
super(Preference, self).__init__(*args, **kwargs)
...
def _get_value(self):
if "value" in self._cache:
value = self._cache["value"]
else:
value = pickle.loads(self._value)
self._cache["value"] = value
return value
def _set_value(self, value):
self._cache["value"] = value
self._value = pickle.dumps(value)
_value = models.TextField()
value = property(_get_value, _set_value)
...
-
--
Mfg.
Jens Diemer
A django powered CMS: http://www.pylucid.org
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Pickle model fields
I would make a pickle field, too. I tried this: - class Preference(models.Model): ... def _get_value(self): return pickle.loads(self._value) def _set_value(self, value): self._value = pickle.dumps(value) _value = models.TextField() value = property(_get_value, _set_value) ... - In a short test works this seems to work... Question: Is this a good idea? -- Mfg. Jens Diemer A django powered CMS: http://www.pylucid.org --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On Mon, 2007-06-11 at 08:21 -0700, [EMAIL PROTECTED] wrote: > No worries. I updated the get_db_prep_save to > > def get_db_prep_save(self, value): > if not isinstance( value, basestring ): > return cPickle.dumps(value) > else: > return value > > So that it doesn't re-pickle data that is already a string. Then I > unpickle the field if it's a string when ever I use it. A little > clunky, but my project's getting done. I was thinking about this a bit more. As a work around, you could also add an __init__ to your model. Normally, no __init__ method is needed, but if you did something like this: class MyModel(models.Model): def __init__(self, *args, **kwargs): super(MyModel, self).__init__(*args, **kwargs) self.pickled_field = cPickle.dumps(value) it might do the job without having to modify Django's core. You can just explicitly convert the field (or fields) to the value you want. It's not particularly neat as a general solution for field sub-classing, but for one or two fields with code that works right this minute, it could be worth trying. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
No worries. I updated the get_db_prep_save to def get_db_prep_save(self, value): if not isinstance( value, basestring ): return cPickle.dumps(value) else: return value So that it doesn't re-pickle data that is already a string. Then I unpickle the field if it's a string when ever I use it. A little clunky, but my project's getting done. --Aaron On Jun 11, 4:34 am, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Sun, 2007-06-10 at 15:59 +, [EMAIL PROTECTED] wrote: > > Ok, turns out my code doesn't work. What gets called to convert the > > data as soon as you get it back from the database? I've got it > > pickling on the way in, but it doesn't unpickle on the way out unless > > I call validate manually. > > Nothing does. This is one of the missing pieces for Field sub-classing. > Django just shoves the raw database value into the class's attribute. > Calling a coercion function is something that needs be added. > > It will happen very soon now, just a little more patience. I've nearly > caught up on my backlog. > > Regards, > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
Ok, turns out my code doesn't work. What gets called to convert the data as soon as you get it back from the database? I've got it pickling on the way in, but it doesn't unpickle on the way out unless I call validate manually. --Aaron On May 16, 4:50 pm, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > On Wed, 2007-05-16 at 09:42 -0700, [EMAIL PROTECTED] wrote: > > *sigh* posted to quickly. The solution was to define get_internal_type > > to return TextField.__name__ > > > On May 16, 12:40 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > > > Turns out I decided to go with the Field class since I'll be needing > > > them in a few different models. The issue I'm running into now is that > > > the new field name isn't defined in the data_types list (from db/ > > > backends/*/creation.py). > > > > Obviously I can just add an entry for the backend I'm using, but that > > > seems wrong. I inherit from TextField, is there anyway to tell it to > > > just use that type? > > Remember how I said there were a few pieces of machinery missing to make > subclassing Fields easy? You just found one of them. :-) > > Yes, it has a solution, but it should be automatic in a perfect > solution. > > Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On Wed, 2007-05-16 at 09:42 -0700, [EMAIL PROTECTED] wrote: > *sigh* posted to quickly. The solution was to define get_internal_type > to return TextField.__name__ > > On May 16, 12:40 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > > Turns out I decided to go with the Field class since I'll be needing > > them in a few different models. The issue I'm running into now is that > > the new field name isn't defined in the data_types list (from db/ > > backends/*/creation.py). > > > > Obviously I can just add an entry for the backend I'm using, but that > > seems wrong. I inherit from TextField, is there anyway to tell it to > > just use that type? Remember how I said there were a few pieces of machinery missing to make subclassing Fields easy? You just found one of them. :-) Yes, it has a solution, but it should be automatic in a perfect solution. Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
Here's the final code that seems to be working:
from django.core import validators
from django.utils.translation import gettext
from django.db.models.fields import TextField
import cPickle
class PickleField(TextField):
def __init__(self, *args, **kwargs):
kwargs['editable'] = False
TextField.__init__(self, *args, **kwargs)
def to_python(self, value):
if isinstance( value, basestring ):
try:
return cPickle.loads(value)
except:
raise validators.ValidationError, gettext("Field must be
pickle string")
def get_db_prep_save(self, value):
return cPickle.dumps(value)
def get_db_prep_lookup(self, lookup_type, value):
raise TypeError("Field has invalid lookup: %s (Pickle does not
support lookups)" % lookup_type)
def get_internal_type(self):
return TextField.__name__
On May 16, 12:42 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
> *sigh* posted to quickly. The solution was to define get_internal_type
> to return TextField.__name__
>
> On May 16, 12:40 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
>
> > Turns out I decided to go with the Field class since I'll be needing
> > them in a few different models. The issue I'm running into now is that
> > the new field name isn't defined in the data_types list (from db/
> > backends/*/creation.py).
>
> > Obviously I can just add an entry for the backend I'm using, but that
> > seems wrong. I inherit from TextField, is there anyway to tell it to
> > just use that type?
>
> > --Lucki
>
> > On May 16, 10:54 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
>
> > > Thank you both for your help. I really should have thought of the
> > > proxy attribute myself. I use Property objects all over the place
> > > already. I'll probably go with that solution since I'm a bit time
> > > crunched and can do that one quickly. Though I think the Field class
> > > would be the 'better' solution.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Pickle model fields
*sigh* posted to quickly. The solution was to define get_internal_type to return TextField.__name__ On May 16, 12:40 pm, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Turns out I decided to go with the Field class since I'll be needing > them in a few different models. The issue I'm running into now is that > the new field name isn't defined in the data_types list (from db/ > backends/*/creation.py). > > Obviously I can just add an entry for the backend I'm using, but that > seems wrong. I inherit from TextField, is there anyway to tell it to > just use that type? > > --Lucki > > On May 16, 10:54 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > > > Thank you both for your help. I really should have thought of the > > proxy attribute myself. I use Property objects all over the place > > already. I'll probably go with that solution since I'm a bit time > > crunched and can do that one quickly. Though I think the Field class > > would be the 'better' solution. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
Turns out I decided to go with the Field class since I'll be needing them in a few different models. The issue I'm running into now is that the new field name isn't defined in the data_types list (from db/ backends/*/creation.py). Obviously I can just add an entry for the backend I'm using, but that seems wrong. I inherit from TextField, is there anyway to tell it to just use that type? --Lucki On May 16, 10:54 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > Thank you both for your help. I really should have thought of the > proxy attribute myself. I use Property objects all over the place > already. I'll probably go with that solution since I'm a bit time > crunched and can do that one quickly. Though I think the Field class > would be the 'better' solution. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
Thank you both for your help. I really should have thought of the proxy attribute myself. I use Property objects all over the place already. I'll probably go with that solution since I'm a bit time crunched and can do that one quickly. Though I think the Field class would be the 'better' solution. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On Tue, 2007-05-15 at 17:11 -0500, Jeremy Dunck wrote: > On 5/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > One day I'll have the infinite spare time I've been asking Santa Claus > > for and maybe get around to adding the missing pieces and documentation. > > Or somebody will beat me to it (even better). > > I certainly wasn't intending to prod you or anyone else. I'm sure > you've contributed more than me, and I thank you for it. I realise that, don't worry. This is actually something I *want* to do, but it's lower priorities than the other things we should get done first. Cheers, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On 5/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > One day I'll have the infinite spare time I've been asking Santa Claus > for and maybe get around to adding the missing pieces and documentation. > Or somebody will beat me to it (even better). I certainly wasn't intending to prod you or anyone else. I'm sure you've contributed more than me, and I thank you for it. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On Tue, 2007-05-15 at 17:01 -0500, Jeremy Dunck wrote: > On 5/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > > One solution would be that the model field only stores the pickled > > value. > > I think it's not well understood that model fields can be created by > subclassing django.db.models.fields.Field. True, although I wasn't actually suggesting a Field subclass (I'm aware that's not well documented yet, but that's partly because it's missing some machinery to make it truly useful). I was suggesting adding a property to the Model class that acted as a proxy for the CharField or whatever is being used to store the pickled value. > Lucki, have a look at the docstrings in that class (in file > django/db/models/fields/__init__.py). They're pretty descriptive; > look at FileField for an example of a field that does a fair bit of > non-vanilla customization. Creating a Field sub-class is certainly another option, you're right. One day I'll have the infinite spare time I've been asking Santa Claus for and maybe get around to adding the missing pieces and documentation. Or somebody will beat me to it (even better). Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On 5/15/07, Malcolm Tredinnick <[EMAIL PROTECTED]> wrote: > One solution would be that the model field only stores the pickled > value. I think it's not well understood that model fields can be created by subclassing django.db.models.fields.Field. Lucki, have a look at the docstrings in that class (in file django/db/models/fields/__init__.py). They're pretty descriptive; look at FileField for an example of a field that does a fair bit of non-vanilla customization. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickle model fields
On Tue, 2007-05-15 at 13:59 -0700, [EMAIL PROTECTED] wrote: > Someone /has/ to be doing this. I've got a dictionary, that I want to > pickle and store in a field in one of my Model classes. What is the > optimal solution for this situation? All the solutions I can think of > equate to jiggery-hacky and having to know if the field is in the > pickled or unpickled state. One solution would be that the model field only stores the pickled value. Then you have a property or method on the model that returns the unpickled value (since properties can have getters and setters, you can use this to store new values as well). You wouldn't have to unpickle the value every time it's accessed, either: you could store the unpickled value in a cache attribute on the model and invalidate or update it every time the setter changes the field value. Regards, Malcolm --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Pickle model fields
Someone /has/ to be doing this. I've got a dictionary, that I want to pickle and store in a field in one of my Model classes. What is the optimal solution for this situation? All the solutions I can think of equate to jiggery-hacky and having to know if the field is in the pickled or unpickled state. Any thoughts out there? --Lucki --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
