[issue4769] b64decode should accept strings or bytes

[Éric Araujo]

Changes by Éric Araujo :


--
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[R. David Murray]

R. David Murray  added the comment:

After thinking about it, I'm inclined to reject this and say that quopri should 
be fixed to reject string input to decode.  On python-dev Guido opined that a 
kind of polymorphism in the stdlib was good (bytes in --> bytes out, string in 
--> string out).  string in --> bytes out and bytes in --> string out was 
considered bad, to my understanding (except for unicode encode/decode, of 
course).

As you say, all one has to do is encode the string as ascii to get the bytes to 
pass in.  It is better, I think, to maintain the clear distinction between 
bytes and strings in the programmers mind.  That's what Python3 is all about, 
really.

As for "the whole point of base64 is to safely encode binary data into text 
characters", that is not true.  The point is to encode binary data into a 
subset of *ascii*, which is *not* text, it is bytes.  The fact that this is 
also useful for transferring binary data through unicode is pretty much an 
unintended consequence of the way unicode is designed.

--
resolution:  -> rejected
stage:  -> committed/rejected
status: open -> pending
type: behavior -> feature request

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

Changes by STINNER Victor :


--
versions: +Python 3.2 -Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[Éric Araujo]

Changes by Éric Araujo :


--
nosy: +merwok

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

Changes by STINNER Victor :


Removed file: http://bugs.python.org/file17441/base64_main.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[R. David Murray]

R. David Murray  added the comment:

The patch appears to be fixing the wrong functions.  It is decode that needs to 
accept unicode.  Encode should still be restricted to bytes.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

> I commited base64_main.patch (+ tests): 3.2 (r81533) and 3.1 (r81534).

Hum, the test fails on Windows: fixed by r81535 (3.2) and r81536 (3.1).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

Accept unicode string is not "pure", but I agree that it's convinient. Here is 
a patch:
 * base64.b(16|32|64)encode and b64.encodebytes accept unicode string
 * unicode is first encoded to utf-8 to get a byte string
 * Update the docstrings and the documentation
 * Fix tests

--
Added file: http://bugs.python.org/file17463/base64_str.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

I commited base64_main.patch (+ tests): 3.2 (r81533) and 3.1 (r81534).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

This one!

--
keywords: +patch
Added file: http://bugs.python.org/file17441/base64_main.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[Dan Buch]

Dan Buch  added the comment:

@haypo - what patch? :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

Attached base64_main.patch fixes errors described in 
b64-decode-str-bytes-typeerror.txt.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[Dan Buch]

Dan Buch  added the comment:

This appears to still be an issue in py3k.  I've attached the command and 
output when running ``python3 -m base64`` with various options and inputs.  If 
there's consensus on a solution, I'd be happy to take a crack at making a patch.

--
nosy: +meatballhat
Added file: http://bugs.python.org/file17412/b64-decode-str-bytes-typeerror.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[HiroakiKawai]

Changes by HiroakiKawai :


--
nosy: +kawai

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

> If the input to b64decode is a str, just do a encode('ascii')
> operation on it and proceed.  If that fails, it wasn't valid 
> Base64 to begin with.

On unicode encode error, should we raise an UnicodeEncodeError or a 
binascii.Error?

And there is also the problem of base64.b64decode() 
alternate "characters". Should we accept non-ASCII alternate 
characters?
   base64.b64decode('01a\xfeb\xffcd', altchars=b'\xfe\xff')

For the example, the result depends on the choosen charset:
 - ASCII (strict): encode input text raise an UnicodeDecodeError
 - ISO-8859-1 (ignore): works as expected
 - UTF-8 (strict): unexpected result

The only valid choice is ASCII because ISO-8859-1 or UTF-8 will 
reintroduce bytes/character mixture.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[STINNER Victor]

STINNER Victor  added the comment:

About quoted printable, there are two implementations:
 - binascii.a2b_qp() (Modules/binascii.c): C implementation, use 
PyArg_ParseTupleAndKeywords(args, kwargs, "s*|i", ...) to parse the 
data
 - quopri.decode() (Lib/quopri.py): Python implementation
   => quopri.decodestring() uses io.BytesIO() to parse the data

But quopri.decodestring() reuses binascii.a2b_qp() if the binascii 
module is present. So quopri.decodestring behaviour depends of the 
presence of binascii module...
 - binascii present: accept bytes or unicode
 - missing binascii: accept only bytes!

binascii.a2b_qp() encodes unicode string to UTF-8 bytes string.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[James Brotchie]

Changes by James Brotchie :


--
nosy: +brotchie

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[David M. Beazley]

David M. Beazley  added the comment:

One more followup.   The quopri module (which is highly related to 
base64 in that quopri and base64 are often used together within MIME) 
does accept both unicode and byte strings when decoding.  For example, 
this works:

>>> quopri.decodestring('Hello World')
b'Hello World'
>>> quopri.decodestring(b'Hello World')
b'Hello World'
>>>

However, the quopri module, like base64, uses byte strings almost 
everywhere else.  For example, encoding a byte string with quopri still 
produces bytes (just like base64)

>>> quopri.encodestring(b'Hello World')
b'Hello World'
>>>

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[David M. Beazley]

David M. Beazley  added the comment:

Note: This problem applies to all of the other decoders/encoders in the 
base64 too (b16, b32, etc.)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4769] b64decode should accept strings or bytes

[David M. Beazley]

New submission from David M. Beazley :

The whole point of base64 encoding is to safely encode binary data into 
text characters.  Thus, the base64.b64decode() function should equally 
accept text strings or binary strings as input. For example, there is a 
reasonable expectation that something like this should work:

>>> x = 'SGVsbG8='
>>> base64.b64decode(x)
b'Hello'
>>>

In Python 3, you get this exception however:

>>> base64.b64decode(x)
Traceback (most recent call last):
  File "", line 1, in 
  File "/tmp/lib/python3.0/base64.py", line 80, in b64decode
raise TypeError("expected bytes, not %s" % s.__class__.__name__)
TypeError: expected bytes, not str
>>> 

I realize that there are encoding issues with Unicode strings, but 
base64 encodes everything into the first 127 ASCII characters.  If the 
input to b64decode is a str, just do a encode('ascii') operation on it 
and proceed.  If that fails, it wasn't valid Base64 to begin with.

I can't think of any real negative impact to making this change as long 
as the result is still always bytes.   The main benefit is just 
simplifying the decoding process for end-users.

See issue 4768.

--
components: Library (Lib)
messages: 78466
nosy: beazley
severity: normal
status: open
title: b64decode should accept strings or bytes
type: behavior
versions: Python 3.0

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com