Carl Banks wrote:
> On Jan 17, 2:28 pm, John Nagle <[EMAIL PROTECTED]> wrote:
>
>> It's also necessary to call Pickle's "clear_memo" before each "dump"
>> call, since objects might change between successive "dump" calls.
>> "Unpickle" doesn't have a "clear_memo" function. It should, because
>> if you keep reusing the "Unpickle" object, the memo dictionary
>> fills up with old objects which can't be garbage collected.
>> This creates a memory leak in long-running programs.
>
> This is all good to know. I agree that this is a good use case for a
> clear_memo on a pickle unloader.
reader = pickle.Unpickler(self.datain) # set up reader
....
reader.memo = {} # no memory from cycle to cycle
>
>
>> Then, on Windows, there's a CR LF problem. This can be fixed by
>> launching the subprocess with
>>
>> proc = subprocess.Popen(launchargs,
>> stdin=subprocess.PIPE, stdout=subprocess.PIPE,
>> universal_newlines=True)
>>
>> Failure to do this produces the useful error message "Insecure string
>> pickle".
>> Binary "pickle" protocol modes won't work at all in this situation;
>> "universal
>> newline" translation is compatible, not transparent. On Unix/Linux, this
>> just works, but the code isn't portable.
>
> I would think a better solution would be to use the -u switch to
> launch the subprocess, or the PYTHONUNBUFFERED environment variable if
> you want to invoke the Python script directly. It opens up stdin and
> stdout in binary, unbuffered mode.
Ah. That works. I wasn't aware that "unbuffered" mode also implied
binary transparency. I did that, and now cPickle works in both text (0)
and binary (2) protocol modes. Turned off "Universal Newline" mode.
> Thanks: this was an informative post
Thanks. We have this working well now. After a while, I'll publish
the module, which is called "subprocesscall.py".
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
