Interesting Observation about the trancating. This could save some
space although I think the case is rare. :/
I will continue examine the session data to see if there is anything
else that could cause this...
Best Regards,
Jason
On 10/05/2010 11:40 AM, ron_m wrote:
If you store something in session as in session.name = some_variable
then that will be pickled at the end of the current request along with
anything else in session. The result of the pickle operation is the
session file for your session containing all your session variables.
On the next request the session file is unpickled to restore the
session variables for your session. This is the only way you can
retain state between requests.
If a session variable contains a reference to a class object instance
it will likely pickle with success. However, when you hit the next
request possibly in a different controller and are missing a class
definition because of a missing import in the other controller the
unpickle will fail.
You have to look at every session.something that has an assignment to
it and then look inside the right hand side of that assignment for
what is inside. It could even be a bug where a typo in your logic is
assigning a class instance into a place in a list or dictionary which
is then placed in the session. A Google search for "insecure string
pickle" reveals many causes such as pickle on Windows and unpickle on
UNIX because line endings in Windows are \r\n but on UNIX are \n
unless binary file format is used.
However, have a look at this first, sorry for rambling a bit:
I took a look at gluon/globals.py where request, response and session
are classes. I am suspicious of something in the session saving.
If the session file already exists at the start of the request cycle
the session file is opened in mode 'rb+', the session file is locked
and then it is unpickled with cPickle.load and then the file is seeked
to offset 0 to rewind the position in preparation for the session
write.
Later at the end of the request cycle the function _try_store_on_disk
for an already existing session file has the current version of
session pickled into it with cPickle.dump, then the session file is
unlocked and closed.
What I am unsure of is if the session data on the current request to
use the session is smaller than the session data from the last request
then there will be some left over cruft on the end of the file from
the previous session. Maybe the file should be truncated at the
current position after the cPickle.dump and before the close. I don't
know the internal structure of a pickle file well enough to know if
end of file is important and whether or not extra data on the end of
the file will fool the next unpickle or load call resulting in the
error you are seeing.
To test this you could look for line 378 (Rev 1.86.2) in gluon/
globals.py which is the cPickle.dump line
if response.session_file:
cPickle.dump(dict(self), response.session_file)
and add this line of code after the cPickle.dump line
response.session_file.truncate()
at the same indent level as the cPickle.dump line. This will drop the
file size down to the current size of data from the current request
session write if the session data has shrunk and do nothing otherwise.
Massimo, what do you think of this?
Ron
On Oct 4, 11:30 pm, Jason Brower<encomp...@gmail.com> wrote:
Looked at all the data I would ever send or use in the system and I
don't see any pickles. I only use them in the messages I am sending and
then unpickle them as soon as they get to me. I can't imagine were else
it could be as I don't play with any funny objects or persistent items.
Additionally, the updates only happen every 2 seconds. I will get good,
