[issue14436] SocketHandler sends obejcts while they cannot be unpickled on receiver's side

[Nathan Jensen]

Nathan Jensen added the comment:

I'm not sure this was fixed in an optimal way.

We have a set of processes using the SocketHandler to send log records to a 
single log process.  Some of these log records have the msg attribute as a 
dictionary that contained a variety of extra information about the event that 
was being logged.  After this change, our process receiving the events and 
logging to a file stopped working correctly because it was expecting a 
dictionary for the msg but was always receiving a string.  (I have since made 
it smarter).

The fix for this ticket makes sense when you don't have a guarantee of being 
able to unpickle the msg on the receiving end, but it also limits some of the 
adaptability to pass objects using the SocketHandler.

Some possible improvements:
1. Add a flag to SocketHandler on whether or not it should force the msg to a 
string
2. If it's it a built-in picklable type, don't force to string, else force msg 
to string

Suggestion 2 has some drawbacks in that you'd have to loop over lists, 
dictionaries, etc to verify everything inside there is also picklable.  Also it 
would prevent you from sending custom objects across.

--
nosy: +Nathan Jensen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14436] SocketHandler sends obejcts while they cannot be unpickled on receiver's side

[Roundup Robot]

Roundup Robot  added the comment:

New changeset eda0ae0d2c68 by Vinay Sajip in branch '2.7':
Closes #14436: Convert msg + args to string before pickling.
http://hg.python.org/cpython/rev/eda0ae0d2c68

New changeset cd8347e15f62 by Vinay Sajip in branch '3.2':
Closes #14436: Convert msg + args to string before pickling.
http://hg.python.org/cpython/rev/cd8347e15f62

New changeset 86a1f92c66b3 by Vinay Sajip in branch 'default':
Closes #14436: merged fix from 3.2.
http://hg.python.org/cpython/rev/86a1f92c66b3

--
nosy: +python-dev
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14436] SocketHandler sends obejcts while they cannot be unpickled on receiver's side

[R. David Murray]

Changes by R. David Murray :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14436] SocketHandler sends obejcts while they cannot be unpickled on receiver's side

[Zbigniew Kacprzak]

New submission from Zbigniew Kacprzak :

I decided to use SocketHandler in multi-processes application.
Log server, sending data, logging simple strings - works fine.

The problem is with own classes (or external libraries).
Looks like SocketHandler creates pickles that cannot be unpickled then on 
receiver's side (it does not contain my own classes in PYTHONPATH).

The issue happens only when I use recommened way of passing parameters:

class LibInfo( object ):
  """Library info taken from xml"""
  def __init__(self, libName=None, xmlName=None, packName=None):
self.libName = libName
self.xmlName = xmlName
self.packName = packName

  def __repr__(self):
return "L=%s X=%s P=%s" % (self.libName,self.xmlName,self.packName)

myObj = LibInfo("1", "2", "3")
logging.info("Simple data: %s", myObj)

Traceback (most recent call last):
  File "/opt/python2.7/logging/handlers.py", line 563, in emit
s = self.makePickle(record)
  File "/opt/python2.7/logging/handlers.py", line 533, in makePickle
s = cPickle.dumps(record.__dict__, 1)
PicklingError: Can't pickle : attribute lookup LibInfo failed


# these two lines work properly:
logging.info("Simple data: %s", str(myObj) )
logging.info("Simple data: %s" % myObj)


This would be not that critical: I could convert all passed parameters to 
strings. The issue is with external libraries. That I cannot control.

I think SocketHandler should make record with all parameters resolved to final 
string.

--
components: Library (Lib)
messages: 157019
nosy: Zbigniew.Kacprzak
priority: normal
severity: normal
status: open
title: SocketHandler sends obejcts while they cannot be unpickled on receiver's 
side
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com