[issue8776] Bytes version of sys.argv

[Nick Coghlan]

Nick Coghlan added the comment:

Makes sense to me. Assuming we eventually manage to resolve the POSIX locale 
issue, the bytes variant will become even less useful.

--
resolution: later -> rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor added the comment:

Today I regret os.environb (I added it). If I remember correctly, os.environb 
was added before the PEP 383 (surrogateescape). This PEP makes os.environb 
almost useless. In Python 3, Unicode is the natural choice, and thanks to the 
PEP 383, it's still possible to use any "raw bytes".

argvb can be computed in one line: list(map(os.fsencode, sys.argv)).

I now suggest to close this issue as wontfix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Raymond Hettinger]

Raymond Hettinger added the comment:

Without commenting on this specific proposal, I would like to make an overall 
observation that Python is impairing its usability by adding 
too-many-ways-to-it in a number of categories (file descriptor variants of file 
methods, multiple versions of time.time, byte variants of everything that is 
done with strings).  Python 3 was intended to be a cleaner, more learnable 
version of Python.  Instead, it is growing enums, multiple dispatch, and 
multiple variants of every function.   Professional programmers can be well 
served by some of the these tools, but the Python universe is much larger than 
that and the other users are not being well served by these additions (too many 
choices impairs usability and learnability).

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Nick Coghlan]

Nick Coghlan added the comment:

I'd like to revisit this after PEP 432 is in place, since having to do this 
dance for arg processing when running on Linux in the POSIX locale is somewhat 
lame:

argv = sys.argv
encoding = locale.getpreferredencoding() # Hope nobody changed the locale!
fixed_encoding = read_encoding_from("/etc/locale.conf") # For example
argvb = [arg.encode(encoding, "surrogateescape") for arg in argv]
fixed_argv = [arg.decode(fixed_encoding, "surrogateescape") for arg in 
argvb]

(For stricter parsing, leave out the second "surrogateescape")

Now, if PEP 432 resolves the system encoding issue such that we are able to use 
the right encoding even when locale.getpreferredencoding() returns the wrong 
answer, then it may not be worthwhile to also provide sys.argvb (especially 
since it won't help hybrid 2/3 code). On the other hand, like os.environb, it 
does make it easier for POSIX-only code paths that wants to handle boundary 
encoding issues directly to stick with consuming the binary data directly and 
avoid the interpreter's automatic conversion to the text domain.

Note also that os.environb is only available when os.supports_bytes_environ is 
True, so it would make sense to only provide sys.argvb in the circumstances 
where we provide os.environb.

--
assignee:  -> ncoghlan
nosy: +ncoghlan
resolution: wont fix -> later
status: closed -> open
versions: +Python 3.5 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

One year after opening the issue, I don't have any real use case. And there are 
technical issues to implement this feature, so I prefer just to close this 
issue. Reopen it if you really want it, but please give an use case ;-)

--
resolution:  -> wont fix
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[R. David Murray]

Changes by R. David Murray :


--
stage:  -> needs patch
type:  -> feature request
versions: +Python 3.3 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

Prototype (in Python) of argvb.py. Try it with: ./python -i argvb.py.

It's not possible to create sys.argvb in Python in a module loaded by 
Py_Initialize(), because sys.argv is created after Py_Initialize().

--
Added file: http://bugs.python.org/file19355/argvb.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

Since r85765 (issue #4388), always use UTF-8 to decode the command line 
arguments on Mac OS X, not the locale encoding. Which means that the 
pseudo-code becomes:

 if os.name != 'nt':
 if sys.platform == 'darwin':
encoding = 'utf-8'
 else:
encoding = locale.getpreferredencoding()
 sys.argvb = [arg.decode(encoding, 'surrogateescape') for arg in sys.argv]

sys.argvb should be synchronized with sys.argv, as os.environb with os.environ.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Marc-Andre Lemburg]

Marc-Andre Lemburg  added the comment:

STINNER Victor wrote:
> 
> STINNER Victor  added the comment:
> 
>> Using that approach would work on POSIX systems.
> 
> As os.environb, I think that sys.argv should not exist on Windows.
> 
>> Another problem I see is synchronizing the two
> 
> os.environ and os.environb are synchronized. It would be possible to do the 
> same with sys.argv and sys.argvb. The implement would be simplier because 
> it's just a list, not a dict.

+1 on adding sys.argvb for systems that use char* in main().

--
nosy: +lemburg

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

> Using that approach would work on POSIX systems.

As os.environb, I think that sys.argv should not exist on Windows.

> Another problem I see is synchronizing the two

os.environ and os.environb are synchronized. It would be possible to do the 
same with sys.argv and sys.argvb. The implement would be simplier because it's 
just a list, not a dict.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Martin v . Löwis]

Martin v. Löwis  added the comment:

Using that approach would work on POSIX systems.

Another problem I see is synchronizing the two. If some function strips 
arguments from sys.argv (because it has completed processing), sys.argvb would 
still keep the arguments. Of course, this could be fixed by having sys.argvb be 
a dynamic list (i.e. a sequence object) instead.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Ezio Melotti]

Changes by Ezio Melotti :


--
nosy: +ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

You should read .encode(), not .decode() :-/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

"no byte-oriented representation of the command line is readily available."

Why not using the following recipe?

 encoding = locale.getpreferredencoding()
 sys.argvb = [arg.decode(encoding, 'surrogateescape') for arg in argv]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Martin v . Löwis]

Martin v. Löwis  added the comment:

> As os.environb, it would be useful to have bytes version of sys.argv
> to have able to decide the encoding used to decode each argument, or
> to manipulate bytes if we don't care about the encoding.

-1. Py_Main expects wchar_t*, so no byte-oriented representation of the
command line is readily available.

--
nosy: +loewis

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

STINNER Victor  added the comment:

> The wchar_t strings themselves are built with mbstowcs(), 
> the file system encoding is not used.

Oops sorry, you are right, and it's worse :-) sys.argv is decoded using the 
locale encoding, but subprocess & cie use the file system encoding for the 
reverse operation. => it doesn't work if both encodings are different (#4388, 
#8775).

The pseudo-code to create sys.argv on Unix is:

 # argv is a bytes list
 encoding = locale.getpreferredencoding()
 sys.argv = [arg.decode(encoding, 'surrogateescape') for arg in argv]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc  added the comment:

> sys.argv is decoded with the file system encoding

IIRC this is not exact. Py_Main signature is 
Py_Main(int argc, wchar_t **argv)
then PyUnicode_FromWideChar is used, and there is no conversion (except from 
UCS4 to UCS2).
The wchar_t strings themselves are built with mbstowcs(), the file system 
encoding is not used.

--
nosy: +amaury.forgeotdarc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue8776] Bytes version of sys.argv

[STINNER Victor]

New submission from STINNER Victor :

In some situations, the encoding of the command line is incorrect or unknown. 
sys.argv is decoded with the file system encoding which can be wrong. Eg. see 
issue #4388 (ok, it's a bug, it should be fixed).

As os.environb, it would be useful to have bytes version of sys.argv to have 
able to decide the encoding used to decode each argument, or to manipulate 
bytes if we don't care about the encoding.

See also issue #8775 which propose to add a new encoding to decode sys.argv.

--
components: Interpreter Core, Unicode
messages: 106140
nosy: haypo
priority: normal
severity: normal
status: open
title: Bytes version of sys.argv
versions: Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com