[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution: fixed -> rejected

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Withdrawn in favor of issue17530.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Oh, I forgot about bytes.fromhex(). This of course looks better than 
base64.b16decode((...).replace(' ', '')).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

  1) A separate function might be better.  I think this kind of output would 
 be more useful while inspecting individual byte objects, rather than having 
 it for arbitrary byte objects (that might be inside other containers).

I don't think the general hexdump() function is worth to including in the 
stdlib. It should have too many options (How many bytes display in one line? 
How group hexdigits? What replacemant character for non-printables? Whether or 
not to display addresses? Whether or not to display chars? What are delimiters 
between hexdigits and chars, address and hexdigits? What are line prefix and 
suffix? How display last incomplete line? How display first incomplete line?) 
and this makes it complicated. An application which outputs a hexdump on more 
rich device (a html file or a ANSI-colored terminal) needs advanced options.

However a simple specialized code can be used for special purposes, i.e. 
internally in the pprint module. I don't see how it can be reused and don't 
interested in a general function.

  2) I don't know if the output of pprint is supposed to be eval()uable, but I 
 don't like too much the base64.b16decode(...).replace(' ', '') in the output 
 (especially if the byte objects are short).  If a separate function is used 
 as suggested in 1) this won't be a problem.  Using the hex_codec might be 
 another option.

An alternative option is first output a bytes object as is (perhaps splitting 
it on multiple line as in issue17530) and then output a hexdamp as a comment.

[b'\x7fELF\x01\x01\x01\x00\x00\n'
 b'\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00\x01'
 # 7F 45 4C 46 01 01 01 00 00 0A 00 00 00 00 00 00 | .ELF
 # 02 00 03 00 01  | .
 ]

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here is an alternative patch which outputs a bytes literal and a hexdump as a 
comment.

--
Added file: http://bugs.python.org/file30121/pprint_bytes_hex_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le samedi 04 mai 2013 à 09:42 +, Serhiy Storchaka a écrit :
 
 However a simple specialized code can be used for special purposes,
 i.e. internally in the pprint module. I don't see how it can be reused
 and don't interested in a general function.

I don't understand how it would be useful in the pprint module if it
can't be useful as a general function. The general intent is the same:
print something in a nice way. Just the nice way is different
depending on the situations: if my bytes object is simply a bunch of
HTTP headers, I don't want to have a hexdump.

 An alternative option is first output a bytes object as is (perhaps
 splitting it on multiple line as in issue17530) and then output a
 hexdamp as a comment.
 
 [b'\x7fELF\x01\x01\x01\x00\x00\n'
  b'\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00\x01'
  # 7F 45 4C 46 01 01 01 00 00 0A 00 00 00 00 00 00 | .ELF
  # 02 00 03 00 01  | .
  ]

This won't work very nicely in smaller display widths. You'll need too
many lines to represent a bytes object.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


Removed file: http://bugs.python.org/file30121/pprint_bytes_hex_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Changes by Serhiy Storchaka storch...@gmail.com:


Added file: http://bugs.python.org/file30122/pprint_bytes_hex_2.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 I don't understand how it would be useful in the pprint module if it
 can't be useful as a general function.

How can it be used besides pprint/pformat functions?

  Just the nice way is different
 depending on the situations: if my bytes object is simply a bunch of
 HTTP headers, I don't want to have a hexdump.

Then perhaps a new parameter for pprint/pformat needed (hex=True?). I think 
printing integers in hexadecimal can sometimes be useful too.

 This won't work very nicely in smaller display widths. You'll need too
 many lines to represent a bytes object.

This is a nature of hexdumps. Every byte requires 4+ characters (or 3+ if group 
hexdigits tighter).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Antoine Pitrou]

Antoine Pitrou added the comment:

  I don't understand how it would be useful in the pprint module if it
  can't be useful as a general function.
 
 How can it be used besides pprint/pformat functions?

I don't understand your question. Do you never print some data at the
command-line prompt? Or even as part of small test programs?

 Then perhaps a new parameter for pprint/pformat needed (hex=True?). I
 think printing integers in hexadecimal can sometimes be useful too.

Passing type-specific parameters to pprint/pformat sounds like a bad
idea to me. And I don't think you'd want to print *all* integers as hex.

  This won't work very nicely in smaller display widths. You'll need too
  many lines to represent a bytes object.
 
 This is a nature of hexdumps. Every byte requires 4+ characters (or 3+
 if group hexdigits tighter).

Which is why the proposal doesn't fit well with pprint/pformat.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Ezio Melotti]

Ezio Melotti added the comment:

The idea is that the output of pprint should be something like (once #17530 is 
applied):
 pprint.pprint(b'\x7fELF\x01\x01\x01\x00\x00\n\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00\x01')
(b'\x7fELF\x01\x01\x01\x00\x00\n\x00\x00'
 b'\x00\x00\x00\x00\x02\x00\x03\x00\x01')

whereas the output of hexdump can be something like:
pprint.hexdump(b'\x7fELF\x01\x01\x01\x00\x00\n\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00\x01')
7F 45 4C 46 01 01 01 00 00 0A 00 00 00 00 00 00 | .ELF
02 00 03 00 01  | .

hexdump() could accept some additional args too if required, but otherwise I 
don't think the details are so important as long as it produces something 
readable for a human.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 I don't understand your question. Do you never print some data at the
 command-line prompt? Or even as part of small test programs?

To be honest, I very rarely even use pprint. I'm too lazy to import it. If I 
want to quickly get a hexdump, I use something like `' '.join('%02X'%i for i in 
data)`. It is shorter than `import pprint; pprint.hexdump(data)`. For a small 
program most likely the standard hexdump() will not be enough.

 Passing type-specific parameters to pprint/pformat sounds like a bad
 idea to me.

Agree. Of course it would be better to automatically determine a nice display 
(use hexdump only for large non-printable bytes).

 And I don't think you'd want to print *all* integers as hex.

If you want to print bytes in hex, why not ints and floats? ;)  In fact I don't 
want to print data as hex, so shut up.

 Which is why the proposal doesn't fit well with pprint/pformat.

Perhaps I misunderstood your wish. I'm not against consider pprint as a black 
box, which does all good magic inside by default. The use of this feature does 
not require anything from the users and does not impose obligations on the 
maintainers. But I'm not interested in a separate function.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Antoine Pitrou]

Antoine Pitrou added the comment:

Le samedi 04 mai 2013 à 15:41 +, Serhiy Storchaka a écrit :
  Which is why the proposal doesn't fit well with pprint/pformat.
 
 Perhaps I misunderstood your wish. I'm not against consider pprint as
 a black box, which does all good magic inside by default. The use of
 this feature does not require anything from the users and does not
 impose obligations on the maintainers. But I'm not interested in a
 separate function.

The problem is the good magic will depend on the situation. Really, I
don't want a hexdump of a HTTP message :-)
Which is why there should be a separate function for those *wishing* a
hexdump.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[anatoly techtonik]

anatoly techtonik added the comment:

Some issues:

1. the hex converting logic doesn't belong to base64 module - there is no 
chance a person without StackOverflow access can find it
2. i'd put issue17862 first as a dependency for this one, because proposed 
itertools.chunks() can be further optimized (chunking endless sequences with 
constant memory overhead, pypy specific speedups, etc)

I like that this is not over-engineered. In my hexdump module I got too 
involved with problems of parsing/producing full dumps in a way compatible with 
Python 2/3. So I have to postpone my own user story until finally I run out of 
time.

Probably hexdump.dump() returning string will make it a useful API for the 
primary user story.
hexdump.dumpgen() as a line generator with 16 hexadecimal bytes delimited by 
space should cover all other use cases.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Here is a patch with which pprint formats long bytes objects which contain 
non-ascii or non-printable bytes as a hexdump.

Inspired by Antoine's wish 
(http://permalink.gmane.org/gmane.comp.python.ideas/20329).

--
components: Library (Lib)
files: pprint_bytes_hex.patch
keywords: patch
messages: 188081
nosy: fdrake, pitrou, serhiy.storchaka, techtonik
priority: normal
severity: normal
stage: patch review
status: open
title: pprint long non-printable bytes as hexdump
type: behavior
versions: Python 3.4
Added file: http://bugs.python.org/file30067/pprint_bytes_hex.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Ezio Melotti]

Ezio Melotti added the comment:

A couple of comments:
 1) A separate function might be better.  I think this kind of output would be 
more useful while inspecting individual byte objects, rather than having it for 
arbitrary byte objects (that might be inside other containers).
 2) I don't know if the output of pprint is supposed to be eval()uable, but I 
don't like too much the base64.b16decode(...).replace(' ', '') in the output 
(especially if the byte objects are short).  If a separate function is used as 
suggested in 1) this won't be a problem.  Using the hex_codec might be another 
option.

--
nosy: +ezio.melotti

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Fred L. Drake, Jr.]

Changes by Fred L. Drake, Jr. fdr...@gmail.com:


--
nosy:  -fdrake

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17868] pprint long non-printable bytes as hexdump

[Antoine Pitrou]

Antoine Pitrou added the comment:

Yes, I think a separate function would be better. There's another issue for 
pprint() of bytes with line continuations:

http://bugs.python.org/issue17530

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17868
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com