Re: Pickling error in Django using Apache2
Thank-you for the demonstration. Makes it much clearer, and sent
me back to my 'Python In A Nutshell' book.
The function I call to get the datetime, in turn, calls sqlalchemy
functions:
**
def getHubIndexUpdateDate(file_id):
"""
This function looks at the hubindex table for the record
containing 'file_id' and returns the update_date field of that record.
This date is used to prevent database update concurrency issues.
The date will be used to determine
if the records have been updated since a particular user read
them. If so, it will not allow the user to
update the records.
"""
(engine, hubindextable, vhubdisptable) = hubutil_conn()
result =
hubindextable.select(and_(~(hubindextable.c.filename.like('%\\_var\\_
%')),hubindextable.c.file_id == file_id)).execute()
myrec = result.fetchone()
return myrec['update_date']
I am thinking sqlalchemy must recreate the class tzinfo everytime it
is called, and this is messing up the Pickling. Not sure
how to get around this, except creat my own datetime instance of a
class that does not change and assign the results to
that instance. Going back to the sqlalchemy documentation may also
provide some clues.
The hubutil_conn() function has a firsttime flag, so it only actually
connects the first time.
What do you think?
On Apr 6, 3:54 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote:
> On 4/6/07, paceman <[EMAIL PROTECTED]> wrote:
>
>
>
> > I guess what I am beginning to realize is that everytime I call my
> > mydatetime = getHubIndexUpdateDate(0), I get a new
> > tz.FixedOffsetTimezone object at , noting the .
>
> Well, if using ctime is workable for you, it's certainly a way to
> avoid the problem.
>
> But no, what you're seeing there is that there are different tzinfo
> *instances*. Yet what pickle is complaining about is different tzinfo
> *classes* of the same name.
>
> In python, everything is an object, including types.
>
> A little demonstration:
>
> >>> i = 1 #Make i an instance of type int.
> >>> type(i) #see?
>
>
>
> >>> t=type(i) #make t be a reference to the type object.
> >>> id(t) #what is the id of t?
>
> 135359904
>
> >>> class C(object): #make a new type, class C.
>
>...: pass
>...:
>
> >>> id(C) #what is the id of C?
>
> 136773772
>
> >>> c=C() #make c be an instance of type C.
> >>> id(type(c)) is id(C) #verify that the type of c is, indeed, what
>
> we currently call C.
> True
>
> >>> class C(object): #re-define C to be some other type.
>
>...: pass
>...:
>
> >>> id(C) #note that the id of type C has changed.
>
> 136788340
>
> >>> id(type(c)) is id(C)
>
> False
>
> Under these conditions, pickle will fail to dump "c" because the type
> it refers to no longer has a name. type(c) is *not* what we call "C"
> any longer. Pickle has no way to re-construct the instance c, since
> it'd need a way to refer to the type formerly named C.
>
> See? :)
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/6/07, paceman <[EMAIL PROTECTED]> wrote: > > I guess what I am beginning to realize is that everytime I call my > mydatetime = getHubIndexUpdateDate(0), I get a new > tz.FixedOffsetTimezone object at , noting the . Well, if using ctime is workable for you, it's certainly a way to avoid the problem. But no, what you're seeing there is that there are different tzinfo *instances*. Yet what pickle is complaining about is different tzinfo *classes* of the same name. In python, everything is an object, including types. A little demonstration: >>> i = 1 #Make i an instance of type int. >>> type(i) #see? >>> t=type(i) #make t be a reference to the type object. >>> id(t) #what is the id of t? 135359904 >>> class C(object): #make a new type, class C. ...: pass ...: >>> id(C) #what is the id of C? 136773772 >>> c=C() #make c be an instance of type C. >>> id(type(c)) is id(C) #verify that the type of c is, indeed, what we currently call C. True >>> class C(object): #re-define C to be some other type. ...: pass ...: >>> id(C) #note that the id of type C has changed. 136788340 >>> id(type(c)) is id(C) False Under these conditions, pickle will fail to dump "c" because the type it refers to no longer has a name. type(c) is *not* what we call "C" any longer. Pickle has no way to re-construct the instance c, since it'd need a way to refer to the type formerly named C. See? :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
I guess what I am beginning to realize is that everytime I call my mydatetime = getHubIndexUpdateDate(0), I get a new tz.FixedOffsetTimezone object at , noting the . Maybe this is what is messing pickle up since it is complaining that its not the same object. I tried changing the getHubIndexUpdateDate(0) function to return datetime.ctime() instead of the datetime object. So, far I have not had the pickling error. I have not tried to edit a record yet, though to see if that works. I guess the dissadvantage is the result of ctime is a text string that does not contain time zone info. Thanks to all as it is slowly dawning on me what is going on. Let me know if I am on the right track. On Apr 6, 3:30 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On 4/6/07, paceman <[EMAIL PROTECTED]> wrote: > > > Next I tried it with cPickle: > ... > > >>> mydatetime > > datetime.datetime(2007, 3, 3, 10, 51, 36, > > tzinfo=) > > >>> import cPickle > > >>> cPickle.dumps(mydatetime) > > "cdatetime\ndatetime\np1\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\ > > ... > > I think that as long as the datetime's tzinfo is from psycopg2, it'll > work, since that tzinfo was specifically written to be pickle-able. > > Why don't you check your codebase (including SQLAlchemy) to see if > tzinfo is being assigned to some other type under other circumstances. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/6/07, paceman <[EMAIL PROTECTED]> wrote: > > Next I tried it with cPickle: ... > >>> mydatetime > datetime.datetime(2007, 3, 3, 10, 51, 36, > tzinfo=) > >>> import cPickle > >>> cPickle.dumps(mydatetime) > "cdatetime\ndatetime\np1\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\ ... I think that as long as the datetime's tzinfo is from psycopg2, it'll work, since that tzinfo was specifically written to be pickle-able. Why don't you check your codebase (including SQLAlchemy) to see if tzinfo is being assigned to some other type under other circumstances. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Next I tried it with cPickle:
>>> from hubDatView import *
>>> mydatetime = getHubIndexUpdateDate(0)
>>> mydatetime
datetime.datetime(2007, 3, 3, 10, 51, 36,
tzinfo=)
>>> import cPickle
>>> cPickle.dumps(mydatetime)
"cdatetime\ndatetime\np1\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\
\x00'\ncpsycopg2.tz\nFixedOffsetTimezone
\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta
\np6\n(I-1\nI68400\nI0\ntRp7\nsbtRp8\n."
>>> cPickle.loads("cdatetime\ndatetime\np1\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\\x00'\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntRp7\nsbtRp8\n.")
datetime.datetime(2007, 3, 3, 10, 51, 36,
tzinfo=)
Seemed to work as well. Am I looking at the wrong thing? Why is
Pickling working here?
Any help is appreciated.
Thanks
On Apr 6, 2:46 pm, "paceman" <[EMAIL PROTECTED]> wrote:
> To try an see if the sqlalchemy datetime was causing the problem,
> I changed this session variable to a string. When I did this, the
> problem
> appeared to be gone, indicating that this is the cuprit.
>
> Next I called the routine by hand that gives the datetime, and played
> with it a bit:
>
> >>> from hubDatView import *
> >>> mydatetime = getHubIndexUpdateDate(0)
> >>> mydatetime
>
> datetime.datetime(2007, 3, 3, 10, 51, 36,
> tzinfo=)>>> import
> pickle
> >>> pickle.dumps(mydatetime)
>
> "cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\
> \x00'\np1\ncpsycopg2.tz\nFixedOffsetTimezone
> \np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta
> \np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\n.">>>
> pickle.loads("cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\\x00'\np1\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\n.")
>
> datetime.datetime(2007, 3, 3, 10, 51, 36,
> tzinfo=)>>>
> mydatetime.tzinfo
>
> >>>
> dir(mydatetime.tzinfo)
>
> ['__class__', '__delattr__', '__dict__', '__doc__',
> '__getattribute__', '__hash__', '__init__', '__module__', '__new__',
> '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__',
> '__weakref__', '_name', '_offset', 'dst', 'fromutc', 'tzname',
> 'utcoffset']
>
> I was trying to see if I could pickle and unpickle the datatime that I
> appear to have problems with, and to see if the tzinfo object had
> and __init__ method since the documentation mentioned in this
> discussion says that it must in order to successfully pickle and un-
> pickle.
> It appears to.
>
> Not sure why this works, if this is the problem. Perhaps I will try
> with cPickle next.
>
> On Apr 5, 5:20 pm, "paceman" <[EMAIL PROTECTED]> wrote:
>
> > Jeremy,
>
> > Thank-you for some more good tips.
> > The end of the error message is:
>
> > > > Can't pickle : it's not the same object as
> > > > psycopg2.tz.FixedOffsetTimezone
>
> > I am pretty sure there is nothing after that.
>
> > I will check into the sqlalchemy datetime wrt pickling. The error
> > appears to occur 50% of the time, but maybe the session
> > only gets pickled 50% of the time, and the pickling is failing all the
> > time. If it is expecting a psycopg2 datetime, but I am providing
> > a sqlalchemy datetime - don't know why it would be expecting the
> > psycopg datetime in the first place.
>
> > Thank-you for straightening me out on my misinterpretation of the
> > trace. I see what you mean. I may temporarily remove
> > the update_date and see if the problem goes away - an confirmation
> > that this is where the problem is.
>
> > On Apr 5, 3:59 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote:
>
> > > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote:
>
> > > > One is an update_date
> > > > that comes from
> > > > a postgresql database
>
> > > Yes, the original error reported is:
>
> > > > > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py",
> > > > > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError:
> > > > > Can't pickle : it's not the same object as
> > > > > psycopg2.tz.FixedOffsetTimezone
>
> > > Which suggests that it has something to do with psycopg2. :)
> > > Unfortunately, it looks like part of your error message was eaten.
> > > There should be something just after "Can't pickle". Can you send
> > > that again?
>
> > > Anyway, the datetimes returned by psycopg2 are a subclass defined in
> > > psycopg2. The python docs actually have a specific admonition about
> > > pickling tzinfo:http://www.python.org/doc/2.3.5/lib/datetime-tzinfo.html
>
> > > psycopg2's tzinfo subclass implements this.
>
> > > Is it possible that the tzinfo attached to the datetime you're storing
> > > in the session is not picklable?
>
> > > >>> import psycopg2
> > > >>> c=psycopg2.connect("dbname=sekrit")
> > > >>> cur = c.cursor()
> > > >>> cur.execute('select current_timestamp')
> > > >>> r = cur.fet
> > > >>> r = cur.fetchone()
> > > >>> r
>
> > > (datetime.datetime(2007, 4, 5, 14, 46, 57, 754021,
> > > tzinfo=),)
>
> > > >>> im
Re: Pickling error in Django using Apache2
To try an see if the sqlalchemy datetime was causing the problem,
I changed this session variable to a string. When I did this, the
problem
appeared to be gone, indicating that this is the cuprit.
Next I called the routine by hand that gives the datetime, and played
with it a bit:
>>> from hubDatView import *
>>> mydatetime = getHubIndexUpdateDate(0)
>>> mydatetime
datetime.datetime(2007, 3, 3, 10, 51, 36,
tzinfo=)
>>> import pickle
>>> pickle.dumps(mydatetime)
"cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\
\x00'\np1\ncpsycopg2.tz\nFixedOffsetTimezone
\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta
\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\n."
>>> pickle.loads("cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x03\\x03\\n3$\\x00\\x00\\x00'\np1\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\n.")
datetime.datetime(2007, 3, 3, 10, 51, 36,
tzinfo=)
>>> mydatetime.tzinfo
>>> dir(mydatetime.tzinfo)
['__class__', '__delattr__', '__dict__', '__doc__',
'__getattribute__', '__hash__', '__init__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__str__',
'__weakref__', '_name', '_offset', 'dst', 'fromutc', 'tzname',
'utcoffset']
I was trying to see if I could pickle and unpickle the datatime that I
appear to have problems with, and to see if the tzinfo object had
and __init__ method since the documentation mentioned in this
discussion says that it must in order to successfully pickle and un-
pickle.
It appears to.
Not sure why this works, if this is the problem. Perhaps I will try
with cPickle next.
On Apr 5, 5:20 pm, "paceman" <[EMAIL PROTECTED]> wrote:
> Jeremy,
>
> Thank-you for some more good tips.
> The end of the error message is:
>
> > > Can't pickle : it's not the same object as
> > > psycopg2.tz.FixedOffsetTimezone
>
> I am pretty sure there is nothing after that.
>
> I will check into the sqlalchemy datetime wrt pickling. The error
> appears to occur 50% of the time, but maybe the session
> only gets pickled 50% of the time, and the pickling is failing all the
> time. If it is expecting a psycopg2 datetime, but I am providing
> a sqlalchemy datetime - don't know why it would be expecting the
> psycopg datetime in the first place.
>
> Thank-you for straightening me out on my misinterpretation of the
> trace. I see what you mean. I may temporarily remove
> the update_date and see if the problem goes away - an confirmation
> that this is where the problem is.
>
> On Apr 5, 3:59 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote:
>
> > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote:
>
> > > One is an update_date
> > > that comes from
> > > a postgresql database
>
> > Yes, the original error reported is:
>
> > > > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py",
> > > > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError:
> > > > Can't pickle : it's not the same object as
> > > > psycopg2.tz.FixedOffsetTimezone
>
> > Which suggests that it has something to do with psycopg2. :)
> > Unfortunately, it looks like part of your error message was eaten.
> > There should be something just after "Can't pickle". Can you send
> > that again?
>
> > Anyway, the datetimes returned by psycopg2 are a subclass defined in
> > psycopg2. The python docs actually have a specific admonition about
> > pickling tzinfo:http://www.python.org/doc/2.3.5/lib/datetime-tzinfo.html
>
> > psycopg2's tzinfo subclass implements this.
>
> > Is it possible that the tzinfo attached to the datetime you're storing
> > in the session is not picklable?
>
> > >>> import psycopg2
> > >>> c=psycopg2.connect("dbname=sekrit")
> > >>> cur = c.cursor()
> > >>> cur.execute('select current_timestamp')
> > >>> r = cur.fet
> > >>> r = cur.fetchone()
> > >>> r
>
> > (datetime.datetime(2007, 4, 5, 14, 46, 57, 754021,
> > tzinfo=),)
>
> > >>> import pickle
> > >>> pickle.dumps(r)
>
> > "(cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x04\\x05\\x0e.9\\x0b\\x81e'\np1\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\ntp11\n."
>
> > > What throws me off, is the pickling error seems to be complaining
> > > about the Session cookie name which
> > > is a datetime that django takes care of - not me.
>
> > No, that's one line in the stack trace, but it's not the line with the
> > error. That line forces pickling of the session object, which is
> > where the problem is. It has nothing to do with the cookie. :)
>
> > > I thought I was using straightforward django technique, except for
> > > using sqlalchemy to access
> > > my database tables.
>
> > Hmm, I have no idea how SQLAlchemy is constructing its datetime
> > instances. It would probably be worth looking at, since using
> > SQLAlchemy is probably fairly unusual with Django+psycopg2.
>
> > -Jeremy
--~--~-~--~~~---~--~
Re: Pickling error in Django using Apache2
Jeremy,
Thank-you for some more good tips.
The end of the error message is:
> > Can't pickle : it's not the same object as
> > psycopg2.tz.FixedOffsetTimezone
I am pretty sure there is nothing after that.
I will check into the sqlalchemy datetime wrt pickling. The error
appears to occur 50% of the time, but maybe the session
only gets pickled 50% of the time, and the pickling is failing all the
time. If it is expecting a psycopg2 datetime, but I am providing
a sqlalchemy datetime - don't know why it would be expecting the
psycopg datetime in the first place.
Thank-you for straightening me out on my misinterpretation of the
trace. I see what you mean. I may temporarily remove
the update_date and see if the problem goes away - an confirmation
that this is where the problem is.
On Apr 5, 3:59 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote:
> On 4/5/07, paceman <[EMAIL PROTECTED]> wrote:
>
> > One is an update_date
> > that comes from
> > a postgresql database
>
> Yes, the original error reported is:
>
> > > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py",
> > > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError:
> > > Can't pickle : it's not the same object as
> > > psycopg2.tz.FixedOffsetTimezone
>
> Which suggests that it has something to do with psycopg2. :)
> Unfortunately, it looks like part of your error message was eaten.
> There should be something just after "Can't pickle". Can you send
> that again?
>
> Anyway, the datetimes returned by psycopg2 are a subclass defined in
> psycopg2. The python docs actually have a specific admonition about
> pickling tzinfo:http://www.python.org/doc/2.3.5/lib/datetime-tzinfo.html
>
> psycopg2's tzinfo subclass implements this.
>
> Is it possible that the tzinfo attached to the datetime you're storing
> in the session is not picklable?
>
> >>> import psycopg2
> >>> c=psycopg2.connect("dbname=sekrit")
> >>> cur = c.cursor()
> >>> cur.execute('select current_timestamp')
> >>> r = cur.fet
> >>> r = cur.fetchone()
> >>> r
>
> (datetime.datetime(2007, 4, 5, 14, 46, 57, 754021,
> tzinfo=),)
>
> >>> import pickle
> >>> pickle.dumps(r)
>
> "(cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x04\\x05\\x0e.9\\x0b\\x81e'\np1\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\ntp11\n."
>
> > What throws me off, is the pickling error seems to be complaining
> > about the Session cookie name which
> > is a datetime that django takes care of - not me.
>
> No, that's one line in the stack trace, but it's not the line with the
> error. That line forces pickling of the session object, which is
> where the problem is. It has nothing to do with the cookie. :)
>
> > I thought I was using straightforward django technique, except for
> > using sqlalchemy to access
> > my database tables.
>
> Hmm, I have no idea how SQLAlchemy is constructing its datetime
> instances. It would probably be worth looking at, since using
> SQLAlchemy is probably fairly unusual with Django+psycopg2.
>
> -Jeremy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/5/07, paceman <[EMAIL PROTECTED]> wrote:
> One is an update_date
> that comes from
> a postgresql database
Yes, the original error reported is:
> > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py",
> > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError:
> > Can't pickle : it's not the same object as
> > psycopg2.tz.FixedOffsetTimezone
Which suggests that it has something to do with psycopg2. :)
Unfortunately, it looks like part of your error message was eaten.
There should be something just after "Can't pickle". Can you send
that again?
Anyway, the datetimes returned by psycopg2 are a subclass defined in
psycopg2. The python docs actually have a specific admonition about
pickling tzinfo:
http://www.python.org/doc/2.3.5/lib/datetime-tzinfo.html
psycopg2's tzinfo subclass implements this.
Is it possible that the tzinfo attached to the datetime you're storing
in the session is not picklable?
>>> import psycopg2
>>> c=psycopg2.connect("dbname=sekrit")
>>> cur = c.cursor()
>>> cur.execute('select current_timestamp')
>>> r = cur.fet
>>> r = cur.fetchone()
>>> r
(datetime.datetime(2007, 4, 5, 14, 46, 57, 754021,
tzinfo=),)
>>> import pickle
>>> pickle.dumps(r)
"(cdatetime\ndatetime\np0\n(S'\\x07\\xd7\\x04\\x05\\x0e.9\\x0b\\x81e'\np1\ncpsycopg2.tz\nFixedOffsetTimezone\np2\n(tRp3\n(dp4\nS'_offset'\np5\ncdatetime\ntimedelta\np6\n(I-1\nI68400\nI0\ntp7\nRp8\nsbtp9\nRp10\ntp11\n."
> What throws me off, is the pickling error seems to be complaining
> about the Session cookie name which
> is a datetime that django takes care of - not me.
No, that's one line in the stack trace, but it's not the line with the
error. That line forces pickling of the session object, which is
where the problem is. It has nothing to do with the cookie. :)
> I thought I was using straightforward django technique, except for
> using sqlalchemy to access
> my database tables.
Hmm, I have no idea how SQLAlchemy is constructing its datetime
instances. It would probably be worth looking at, since using
SQLAlchemy is probably fairly unusual with Django+psycopg2.
-Jeremy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Jeremy, the added detail you have provided caused me to go back and look at my code to see if I am doing what you are suggesting. It is not obvious to me. I believe I add two things to request.session. One is an update_date that comes from a postgresql database and the other is a status text string. Because the editing session can cross more than one web page, I wanted to remember the update_date to ensure nobody updates my record before I get to update it. I have even gone and looked at the session database records, on one occasion, but did not see anything suspicious. What throws me off, is the pickling error seems to be complaining about the Session cookie name which is a datetime that django takes care of - not me. The code from django.contrib.sessions.middleware.py in the def SessionMiddleware(object) function fails on line 89: response.set_cookie(settings.SESSION_COOKIE_NAME, session_key, max_age=max_age, expires=expires, domain=settings.SESSION_COOKIE_DOMAIN) This is just before it returns response. The line before that creates a new session: new_session = Session.objects.save(session_key, request.session._session, datetime.datetime.now() + datetime.timedelta(seconds=settings.SESSION_COOKIE_AGE)) In my code, view.py functions get called every time a webpage is refreshed or a form gets submitted. I get my problem without submitting any forms, just scrolling thru records viewing them, but this calles the view.py function everytime. As you scroll, the update_date gets changed to the one for the next record, from the database. As I am scrolling, eventually I get this pickling error. I thought I was using straightforward django technique, except for using sqlalchemy to access my database tables. I am going to look at this further, looking for the scenario you have outlined. However, I still don't understand why the code would work fine under certain conditions - ie. mpm-worker or in the django development mode. Mae, you have added a dimension to this for me. Because, I started experiencing this problem when I set up a new PC and used the svn to build the software configuration in site-packages and then added symbolic links from my svn folder. When I discovered that my original machine was using the apache2 mpm-worker and my new target machine was using apache2-prefork, and making it the same fixed the problem, I thought the problem was somehow related to apache2. However, I do not exclude the .pyc files from my svn repository. Maybe I should. I do notice that sometimes you have to reload apache2 to get your latest changes. Maybe this is a characteristic of the prefork apache2 as it only runs one thread (just guessing). I appreciate the time you guys are taking to help me with this. On Apr 5, 12:54 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On 4/5/07, Mae <[EMAIL PROTECTED]> wrote: > > > > > Wow, what luck! I've been having a sporadic "" problem, and I've just > > resigned myself to spending today to try to debug it. Searched the > > group for exact error, found nothing, refreshed, and saw this post in > > top slot! Kismet. > > (The error is "TypeError: can't pickle function objects", per a later > email. This error is issued by cPickle rather than pickle, but the > approach is basically the same.) > > In either case, the basic problem is that pickle has to be able to > re-create the objects later, and can only do that if it knows how to > refer to the type of the thing it must create. Pickle refuses to > serialize an object it can't be fairly sure of later de-serializing. > > http://docs.python.org/lib/pickle-inst.html > > Here's a quick example producing this error: > > >>> import cPickle > >>> def z(): > >...: return 1 >...: > > >>> class C(object): > >...: pass >...: > > >>> c=C() > >>> cPickle.dumps(c) #works fine > > 'ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n.' > > >>> c.x = z > >>> cPickle.dumps(c) #works fine > > "ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n(dp5\nS'x'\nc__main__\nz\np6\nsb." > > >>> def z(): #redefine z, which c.x refers to. > >...: return 2 >...: > > >>> cPickle.dumps(c) #explodes because the function referred to by c.x > > no longer has a name. > ... > TypeError: can't pickle function objects > > > > Similarly, paceman's error is something like this (starting with a new > interpreter!): > > >>> import cPickle > >>> class C(object): > >...: pass >...: > > >>> c=C() > >>> cPickle.dumps(c) #works fine > > 'ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n.' > > >>> class C(object): #redefine C > >...: pass >...: > > >>> cPickle.dumps(c) #fails because the class c is an instance of no > > longer has a name. > ... > PicklingError: Can't pickle : it's not the same > object as __main__.C > > - --~--~-~--~-
Re: Pickling error in Django using Apache2
On 4/5/07, Mae <[EMAIL PROTECTED]> wrote: ... > We also have a theory that it's adding model objects to sessions that > causes our problem because model objects are created dynamically by > their meta classes. Am I close? I don't think so, since I've successfully serialized models before. I'm pretty sure it's a common practice. The next time the problem occurs, carefully look at the context and the object passed to pickle.dump. Try to reproduce it, simplifying the problem until you're able to determine what the problem is. > Could it still be not deleting the > *.pyc files? Again, I don't think so, but what do I know? :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/5/07, Mae <[EMAIL PROTECTED]> wrote: ... > Is what I'm doing "monkey patching"? What does "monkey patching" > mean? How am I making it difficult for the pickling functions to > figure out my "type name"? What should I do to fix it? > No, but if your code is that simple and still fails, you have less work to do to figure out what's going wrong. ;-) http://en.wikipedia.org/wiki/Monkey_patching > Is it possible that running svn update on my directories and > restarting Apache without deleting all of the old compiled *.pyc files > first is the "monkey patching" that screws up my server? Err, I don't -think- so; python is generally smart enough to re-gen pyc if the py is later, but the python process has to have permissions to do so... Why not include deleting *.pyc when you do an svn up, just as a simple test? > Thanks so much, > Mae > > On Apr 5, 11:50 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote: > > ... > > > > > I wonder if apache prefork reloads modules (maybe psycopg2) that I am > > > not aware of and that causes the pickling of the object not to work? > > > > This would also occur if you were monkey patching classes, I think. > > > > Do you have anything that does something like this: > > > > import x > > > > class Y(object): --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/5/07, Mae <[EMAIL PROTECTED]> wrote: > > Wow, what luck! I've been having a sporadic "" problem, and I've just > resigned myself to spending today to try to debug it. Searched the > group for exact error, found nothing, refreshed, and saw this post in > top slot! Kismet. (The error is "TypeError: can't pickle function objects", per a later email. This error is issued by cPickle rather than pickle, but the approach is basically the same.) In either case, the basic problem is that pickle has to be able to re-create the objects later, and can only do that if it knows how to refer to the type of the thing it must create. Pickle refuses to serialize an object it can't be fairly sure of later de-serializing. http://docs.python.org/lib/pickle-inst.html Here's a quick example producing this error: >>> import cPickle >>> def z(): ...: return 1 ...: >>> class C(object): ...: pass ...: >>> c=C() >>> cPickle.dumps(c) #works fine 'ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n.' >>> c.x = z >>> cPickle.dumps(c) #works fine "ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n(dp5\nS'x'\nc__main__\nz\np6\nsb." >>> def z(): #redefine z, which c.x refers to. ...: return 2 ...: >>> cPickle.dumps(c) #explodes because the function referred to by c.x no longer has a name. ... TypeError: can't pickle function objects Similarly, paceman's error is something like this (starting with a new interpreter!): >>> import cPickle >>> class C(object): ...: pass ...: >>> c=C() >>> cPickle.dumps(c) #works fine 'ccopy_reg\n_reconstructor\np1\n(c__main__\nC\np2\nc__builtin__\nobject\np3\nNtRp4\n.' >>> class C(object): #redefine C ...: pass ...: >>> cPickle.dumps(c) #fails because the class c is an instance of no longer has a name. ... PicklingError: Can't pickle : it's not the same object as __main__.C - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Aha. I fear I'll be accused of mumbling to myself, but I think I understand now that what you meant by your x.Y = Y example. We also have a theory that it's adding model objects to sessions that causes our problem because model objects are created dynamically by their meta classes. Am I close? Could it still be not deleting the *.pyc files? Mae On Apr 5, 12:17 pm, "Mae" <[EMAIL PROTECTED]> wrote: > Er, that would be "a sporadic "TypeError: can't pickle function > objects" problem" :). Forgot to paste. > > On Apr 5, 12:08 pm, "Mae" <[EMAIL PROTECTED]> wrote: > > > Wow, what luck! I've been having a sporadic "" problem, and I've just > > resigned myself to spending today to try to debug it. Searched the > > group for exact error, found nothing, refreshed, and saw this post in > > top slot! Kismet. > > > Jeremy, I'm not sure I understood your last post well enough. Could > > you explain? I have code that goes like this: > > > file: cart_models.py > > class Cart(object): > > ... > > > file: views.py > > from myproject.cart_models import Cart > > cart = Cart() > > request.session['cart'] = cart > > ... > > > I also have several other examples where I instantiate a model object > > or another object and then add it to something that later gets added > > to the session. > > > Is what I'm doing "monkey patching"? What does "monkey patching" > > mean? How am I making it difficult for the pickling functions to > > figure out my "type name"? What should I do to fix it? > > > Is it possible that running svn update on my directories and > > restarting Apache without deleting all of the old compiled *.pyc files > > first is the "monkey patching" that screws up my server? > > > Thanks so much, > > Mae > > > On Apr 5, 11:50 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > > > > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote: > > > ... > > > > > I wonder if apache prefork reloads modules (maybe psycopg2) that I am > > > > not aware of and that causes the pickling of the object not to work? > > > > This would also occur if you were monkey patching classes, I think. > > > > Do you have anything that does something like this: > > > > import x > > > > class Y(object): > > > pass > > > > x.Y = Y > > > > ? > > > > The basic issue is that pickling works by serializing a type*name* > > > plus some state for that object. If the type changes from the time > > > the object is created until the time the serialization is attempted, > > > pickle gives up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Er, that would be "a sporadic "TypeError: can't pickle function objects" problem" :). Forgot to paste. On Apr 5, 12:08 pm, "Mae" <[EMAIL PROTECTED]> wrote: > Wow, what luck! I've been having a sporadic "" problem, and I've just > resigned myself to spending today to try to debug it. Searched the > group for exact error, found nothing, refreshed, and saw this post in > top slot! Kismet. > > Jeremy, I'm not sure I understood your last post well enough. Could > you explain? I have code that goes like this: > > file: cart_models.py > class Cart(object): > ... > > file: views.py > from myproject.cart_models import Cart > cart = Cart() > request.session['cart'] = cart > ... > > I also have several other examples where I instantiate a model object > or another object and then add it to something that later gets added > to the session. > > Is what I'm doing "monkey patching"? What does "monkey patching" > mean? How am I making it difficult for the pickling functions to > figure out my "type name"? What should I do to fix it? > > Is it possible that running svn update on my directories and > restarting Apache without deleting all of the old compiled *.pyc files > first is the "monkey patching" that screws up my server? > > Thanks so much, > Mae > > On Apr 5, 11:50 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > > > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote: > > ... > > > > I wonder if apache prefork reloads modules (maybe psycopg2) that I am > > > not aware of and that causes the pickling of the object not to work? > > > This would also occur if you were monkey patching classes, I think. > > > Do you have anything that does something like this: > > > import x > > > class Y(object): > > pass > > > x.Y = Y > > > ? > > > The basic issue is that pickling works by serializing a type*name* > > plus some state for that object. If the type changes from the time > > the object is created until the time the serialization is attempted, > > pickle gives up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Wow, what luck! I've been having a sporadic "" problem, and I've just resigned myself to spending today to try to debug it. Searched the group for exact error, found nothing, refreshed, and saw this post in top slot! Kismet. Jeremy, I'm not sure I understood your last post well enough. Could you explain? I have code that goes like this: file: cart_models.py class Cart(object): ... file: views.py from myproject.cart_models import Cart cart = Cart() request.session['cart'] = cart ... I also have several other examples where I instantiate a model object or another object and then add it to something that later gets added to the session. Is what I'm doing "monkey patching"? What does "monkey patching" mean? How am I making it difficult for the pickling functions to figure out my "type name"? What should I do to fix it? Is it possible that running svn update on my directories and restarting Apache without deleting all of the old compiled *.pyc files first is the "monkey patching" that screws up my server? Thanks so much, Mae On Apr 5, 11:50 am, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On 4/5/07, paceman <[EMAIL PROTECTED]> wrote: > ... > > > I wonder if apache prefork reloads modules (maybe psycopg2) that I am > > not aware of and that causes the pickling of the object not to work? > > This would also occur if you were monkey patching classes, I think. > > Do you have anything that does something like this: > > import x > > class Y(object): > pass > > x.Y = Y > > ? > > The basic issue is that pickling works by serializing a type*name* > plus some state for that object. If the type changes from the time > the object is created until the time the serialization is attempted, > pickle gives up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/5/07, paceman <[EMAIL PROTECTED]> wrote: ... > I wonder if apache prefork reloads modules (maybe psycopg2) that I am > not aware of and that causes the pickling of the object not to work? This would also occur if you were monkey patching classes, I think. Do you have anything that does something like this: import x class Y(object): pass x.Y = Y ? The basic issue is that pickling works by serializing a type*name* plus some state for that object. If the type changes from the time the object is created until the time the serialization is attempted, pickle gives up. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
Thank-you for your reply. It gives me more food for thought. I still do not know why it works perfectly in the django development mode, and works half the time with apache2 prefork, and works all the time with apache2 mpm-worker. I wonder if apache prefork reloads modules (maybe psycopg2) that I am not aware of and that causes the pickling of the object not to work? If anyone has any more specific encounters with this, please let me know. On Apr 4, 5:05 pm, "Jeremy Dunck" <[EMAIL PROTECTED]> wrote: > On 4/4/07, paceman <[EMAIL PROTECTED]> wrote: > ... > > > > > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py", > > > > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError: > > > > Can't pickle : it's not the same object as > > > > psycopg2.tz.FixedOffsetTimezone > > ** > > This is occurring when Django tries to pickle your session dict to > store it back in the database. > > It's puking because something you're putting in the session back is > not serializable. This is (apparently) usually caused by reloading a > module. > > See:https://sourceforge.net/tracker/?func=detail&atid=105470&aid=628925&g... > andhttps://sourceforge.net/tracker/?func=detail&atid=105470&aid=451547&g... > > In this case, it appears related to psycopg2. Are you reloading that > anywhere? Or possibly circularly importing things? > > > If I change Apache2 to the mpm-worker package (which, by the way, is > > not recommended by Django): Apache2-mpm-worker - 2.2.3-4 > > I have no idea why that would matter. :) > ... > > > Has anybody ever come across this > > No, but that's my guess. :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: Pickling error in Django using Apache2
On 4/4/07, paceman <[EMAIL PROTECTED]> wrote: ... > > > "/var/lib/python-support/python2.4/django/contrib/sessions/models.py", > > > line 10, in encode pickled = pickle.dumps(session_dict) PicklingError: > > > Can't pickle : it's not the same object as > > > psycopg2.tz.FixedOffsetTimezone > ** This is occurring when Django tries to pickle your session dict to store it back in the database. It's puking because something you're putting in the session back is not serializable. This is (apparently) usually caused by reloading a module. See: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=628925&group_id=5470 and https://sourceforge.net/tracker/?func=detail&atid=105470&aid=451547&group_id=5470 In this case, it appears related to psycopg2. Are you reloading that anywhere? Or possibly circularly importing things? > If I change Apache2 to the mpm-worker package (which, by the way, is > not recommended by Django): Apache2-mpm-worker - 2.2.3-4 I have no idea why that would matter. :) ... > Has anybody ever come across this No, but that's my guess. :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
