Taking into consideration the comments from the last round:
- 'bytes.zeros' renamed to 'bytes.size', with option byte filler
(defaults to b'\x00')
- 'bytes.byte' renamed to 'fromint', add 'bchr' function
- deprecation and removal softened to deprecation/discouragement
-----------
PEP: 467
Title: Minor API improvements for binary sequences
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncogh...@gmail.com>, Ethan Furman <et...@stoneleaf.us>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2014-03-30
Python-Version: 3.6
Post-History: 2014-03-30 2014-08-15 2014-08-16 2016-06-07
Abstract
========
During the initial development of the Python 3 language specification, the
core ``bytes`` type for arbitrary binary data started as the mutable type
that is now referred to as ``bytearray``. Other aspects of operating in
the binary domain in Python have also evolved over the course of the Python
3 series.
This PEP proposes five small adjustments to the APIs of the ``bytes``,
``bytearray`` and ``memoryview`` types to make it easier to operate entirely
in the binary domain:
* Deprecate passing single integer values to ``bytes`` and ``bytearray``
* Add ``bytes.size`` and ``bytearray.size`` alternative constructors
* Add ``bytes.fromint`` and ``bytearray.fromint`` alternative constructors
* Add ``bytes.getbyte`` and ``bytearray.getbyte`` byte retrieval methods
* Add ``bytes.iterbytes``, ``bytearray.iterbytes`` and
``memoryview.iterbytes`` alternative iterators
Proposals
=========
Deprecation of current "zero-initialised sequence" behaviour without removal
----------------------------------------------------------------------------
Currently, the ``bytes`` and ``bytearray`` constructors accept an integer
argument and interpret it as meaning to create a zero-initialised sequence
of the given size::
>>> bytes(3)
b'\x00\x00\x00'
>>> bytearray(3)
bytearray(b'\x00\x00\x00')
This PEP proposes to deprecate that behaviour in Python 3.6, but to leave
it in place for at least as long as Python 2.7 is supported, possibly
indefinitely.
No other changes are proposed to the existing constructors.
Addition of explicit "count and byte initialised sequence" constructors
-----------------------------------------------------------------------
To replace the deprecated behaviour, this PEP proposes the addition of an
explicit ``size`` alternative constructor as a class method on both
``bytes`` and ``bytearray`` whose first argument is the count, and whose
second argument is the fill byte to use (defaults to ``\x00``)::
>>> bytes.size(3)
b'\x00\x00\x00'
>>> bytearray.size(3)
bytearray(b'\x00\x00\x00')
>>> bytes.size(5, b'\x0a')
b'\x0a\x0a\x0a\x0a\x0a'
>>> bytearray.size(5, b'\x0a')
bytearray(b'\x0a\x0a\x0a\x0a\x0a')
It will behave just as the current constructors behave when passed a single
integer.
Addition of "bchr" function and explicit "single byte" constructors
-------------------------------------------------------------------
As binary counterparts to the text ``chr`` function, this PEP proposes
the addition of a ``bchr`` function and an explicit ``fromint`` alternative
constructor as a class method on both ``bytes`` and ``bytearray``::
>>> bchr(ord("A"))
b'A'
>>> bchr(ord(b"A"))
b'A'
>>> bytes.fromint(65)
b'A'
>>> bytearray.fromint(65)
bytearray(b'A')
These methods will only accept integers in the range 0 to 255 (inclusive)::
>>> bytes.fromint(512)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: integer must be in range(0, 256)
>>> bytes.fromint(1.0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'float' object cannot be interpreted as an integer
The documentation of the ``ord`` builtin will be updated to explicitly note
that ``bchr`` is the primary inverse operation for binary data, while ``chr``
is the inverse operation for text data, and that ``bytes.fromint`` and
``bytearray.fromint`` also exist.
Behaviourally, ``bytes.fromint(x)`` will be equivalent to the current
``bytes([x])`` (and similarly for ``bytearray``). The new spelling is
expected to be easier to discover and easier to read (especially when used
in conjunction with indexing operations on binary sequence types).
As a separate method, the new spelling will also work better with higher
order functions like ``map``.
Addition of "getbyte" method to retrieve a single byte
------------------------------------------------------
This PEP proposes that ``bytes`` and ``bytearray`` gain the method ``getbyte``
which will always return ``bytes``::
>>> b'abc'.getbyte(0)
b'a'
If an index is asked for that doesn't exist, ``IndexError`` is raised::
>>> b'abc'.getbyte(9)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: index out of range
Addition of optimised iterator methods that produce ``bytes`` objects
---------------------------------------------------------------------
This PEP proposes that ``bytes``, ``bytearray`` and ``memoryview`` gain an
optimised ``iterbytes`` method that produces length 1 ``bytes`` objects
rather than integers::
for x in data.iterbytes():
# x is a length 1 ``bytes`` object, rather than an integer
For example::
>>> tuple(b"ABC".iterbytes())
(b'A', b'B', b'C')
The method can be used with arbitrary buffer exporting objects by wrapping
them in a ``memoryview`` instance first::
for x in memoryview(data).iterbytes():
# x is a length 1 ``bytes`` object, rather than an integer
For ``memoryview``, the semantics of ``iterbytes()`` are defined such that::
memview.tobytes() == b''.join(memview.iterbytes())
This allows the raw bytes of the memory view to be iterated over without
needing to make a copy, regardless of the defined shape and format.
The main advantage this method offers over the ``map(bytes.byte, data)``
approach is that it is guaranteed *not* to fail midstream with a
``ValueError`` or ``TypeError``. By contrast, when using the ``map`` based
approach, the type and value of the individual items in the iterable are
only checked as they are retrieved and passed through the ``bytes.byte``
constructor.
Design discussion
=================
Why not rely on sequence repetition to create zero-initialised sequences?
-------------------------------------------------------------------------
Zero-initialised sequences can be created via sequence repetition::
>>> b'\x00' * 3
b'\x00\x00\x00'
>>> bytearray(b'\x00') * 3
bytearray(b'\x00\x00\x00')
However, this was also the case when the ``bytearray`` type was originally
designed, and the decision was made to add explicit support for it in the
type constructor. The immutable ``bytes`` type then inherited that feature
when it was introduced in PEP 3137.
This PEP isn't revisiting that original design decision, just changing the
spelling as users sometimes find the current behaviour of the binary sequence
constructors surprising. In particular, there's a reasonable case to be made
that ``bytes(x)`` (where ``x`` is an integer) should behave like the
``bytes.byte(x)`` proposal in this PEP. Providing both behaviours as separate
class methods avoids that ambiguity.
References
==========
.. [1] Initial March 2014 discussion thread on python-ideas
(https://mail.python.org/pipermail/python-ideas/2014-March/027295.html)
.. [2] Guido's initial feedback in that thread
(https://mail.python.org/pipermail/python-ideas/2014-March/027376.html)
.. [3] Issue proposing moving zero-initialised sequences to a dedicated API
(http://bugs.python.org/issue20895)
.. [4] Issue proposing to use calloc() for zero-initialised binary sequences
(http://bugs.python.org/issue21644)
.. [5] August 2014 discussion thread on python-dev
(https://mail.python.org/pipermail/python-ideas/2014-March/027295.html)
.. [6] June 2016 discussion thread on python-dev
(https://mail.python.org/pipermail/python-dev/2016-June/144875.html)
Copyright
=========
This document has been placed in the public domain.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
