[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Terry J. Reedy added the comment: This can no longer be a 2.5 issue but I am not sure how to update it. OP apparently opened it as a feature request, so I did update it to 3.2. But OP then says "I'm not sure if this is a functionality or documentation bug." and indeed subsequent messages deb

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg added the comment: Terry J. Reedy wrote: > > Terry J. Reedy added the comment: > > This can no longer be a 2.5 issue but I am not sure how to update it. > > OP apparently opened it as a feature request, so I did update it to 3.2. > > But OP then says "I'm not sure if thi

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Changes by Terry J. Reedy : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

joelpitt added the comment: Just ran into this problem using Python 2.7.3 and the issue others mention in conjunction with Django. Note the 2.7 docs still imply it's ASCII: http://docs.python.org/2/library/pickle.html#data-stream-format It has a weak caveat "(and of some other characteristics

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: Only pickle protocol 0 is ASCII. The other two are binary protocols. Protocol 2 is default in Python 2.5. This should probably be made clear in the documentation, so I'd consider this a documentation bug. -- nosy: +lemburg

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: Actually, I was wrong: protocol 0 is the default if you don't specify the protocol. This set the binary flag to false, which should result in ASCII-only data. The Unicode save routine uses the raw-unicode-escape codec, but this only esca

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Martin v. Löwis <[EMAIL PROTECTED]> added the comment: I think the documentation is fine as it stands. The format is ASCII - even though the payload might not be. -- nosy: +loewis __ Tracker <[EMAIL PROTECTED]> _

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: I can't follow you, Martin. How can a data format be printable ASCII and at the same time use non-ASCII characters ? __ Tracker <[EMAIL PROTECTED]> _

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Martin v. Löwis <[EMAIL PROTECTED]> added the comment: > How can a data format be printable ASCII and at the same time use > non-ASCII characters ? The "format" is the frame defining the structure. In the binary formatter, it's a binary format. In the standard pickle format, it's ASCII (I for in

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-05-28 00:21, Martin v. Löwis wrote: > Martin v. Löwis <[EMAIL PROTECTED]> added the comment: > >> How can a data format be printable ASCII and at the same time use >> non-ASCII characters ? > > The "format" is the frame defining

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Martin v. Löwis <[EMAIL PROTECTED]> added the comment: > Unfortunately, there's no way to fix this now, since the > bug has been around since Python 1.6. Actually, there is a way to fix that: pickle could start emitting \u escapes for characters in the range 128..256. Older pickle implementation

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Changes by Alexandre Vassalotti <[EMAIL PROTECTED]>: -- nosy: +alexandre.vassalotti ___ Python tracker <[EMAIL PROTECTED]> ___ ___ Pytho

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: We could add an extra step to also escape range(128, 256) code points, but I don't think it's worth the performance loss this would cause. Note that this was the first time anyone has ever noticed the fact that the pickle protocol 0 is no

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Dan Dibagh <[EMAIL PROTECTED]> added the comment: Your reasoning shows a lack of understanding how Python is actually used from a programmers point of view. Why do you think that "noticing" a problem is the same thing as entering as a python bug report? In practice there are several steps betwee

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-10-21 11:22, Dan Dibagh wrote: > Your reasoning shows a lack of understanding how Python is actually used > from a programmers point of view. Hmm, I've been using Python for almost 15 years now and do believe that I have an idea o

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Dan Dibagh <[EMAIL PROTECTED]> added the comment: I am well aware why my example produces an error from a technical standpoint. What I'm getting at is the decision to implement PyUnicode_EncodeRawUnicodeEscape the way it is. Probably there is nothing wrong with it, but how am I supposed to know?

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Martin v. Löwis <[EMAIL PROTECTED]> added the comment: > I read the PEP, > which serves as a specification of raw unicode escape (at least for the > decoding bit) and the reference documentation. Which PEP specifically? PEP 263 only mentions the unicode-escape encoding in its problem statement,

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg <[EMAIL PROTECTED]> added the comment: On 2008-10-22 01:34, Martin v. Löwis wrote: >> What makes you think that the problem cannot be fixed without changing >> the existing pickle format 0? > > Applications might rely on what was implemented rather than what was > specified. I

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Dan Dibagh <[EMAIL PROTECTED]> added the comment: > Which PEP specifically? PEP 263 only mentions the unicode-escape > encoding in its problem statement, i.e. as a pre-existing thing. > It doesn't specify it, nor does it give a rationale for why it behaves > the way it does. PEP 100 and PEP 263.

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Torsten Bronger added the comment: I ran into this problem today when writing python data structures into a database. Only ASCII is safe in this situation. I understood the Python docs that protocol 0 was ASCII-only. I use pickle+base64 now, however, this makes debugging more difficult. Anyw

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Torsten Bronger added the comment: "protocol 8" --> "protocol 0" of course. ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscrib

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Marc-Andre Lemburg added the comment: On 2009-01-21 16:43, Torsten Bronger wrote: > Torsten Bronger added the comment: > > I ran into this problem today when writing python data structures into a > database. Only ASCII is safe in this situation. I understood the > Python docs that protocol 0

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Torsten Bronger added the comment: Well, Django doesn't story binary data at all but wants you to store image files etc in the file system. Whether this was a good design decision, is beyond the scope of this issue. My points actually are only these: a) the docs strongly suggest that protocol

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Walter Doekes added the comment: Same issue with Django here ;-) I wouldn't mind a protocol 3 that does <128 ascii only. If only because debugging base64'd zlib'd protocol-2 data is not particularly convenient. -- nosy: +wdoekes ___ Python tracker

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Alexandre Vassalotti added the comment: > I wouldn't mind a protocol 3 that does <128 ascii only. If only because > debugging base64'd zlib'd protocol-2 data is not particularly convenient. Is there any reason that prevent you to debug your pickle using pickle disassembler tool—i.e., pickletool

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Torsten Bronger added the comment: The "problem" is the pickle result. It's not about debugging the pickler itself. -- ___ Python tracker ___ ___

[issue2980] Pickle stream for unicode object may contain non-ASCII characters.

Antoine Pitrou added the comment: If your data is simple enough, you can use JSON. It has an `ensure_ascii` flag when dumping data. -- nosy: +pitrou ___ Python tracker ___ __