[issue11193] test_subprocess error on AIX

[Michael Felt]

Change by Michael Felt :


--
keywords: +patch
pull_requests: +8412
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Michael Felt]

Michael Felt  added the comment:

Short Version:
root@x065:[/data/prj/python/python3-3.8]./python -m test test_subprocess
Run tests sequentially
0:00:00 [1/1] test_subprocess
test_subprocess passed in 2 min 18 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2 min 18 sec
Tests result: SUCCESS

...
--
Ran 285 tests in 136.443s

OK (skipped=33)
test_subprocess passed in 2 min 16 sec

== Tests result: SUCCESS ==

1 test OK.

Total duration: 2 min 16 sec
Tests result: SUCCESS

Posting a PR - tested manually on AIX 5.3, AIX 6.1 and AIX 7.1

imho: being able to remove the special condition for AIX is an improvement.

diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py
index 73b57b21db..4719773b67 100644
--- a/Lib/test/test_subprocess.py
+++ b/Lib/test/test_subprocess.py
@@ -2228,15 +2228,9 @@ class POSIXProcessTestCase(BaseTestCase):
 env = os.environ.copy()
 env[key] = value
 # Use C locale to get ASCII for the locale encoding to force
-# surrogate-escaping of \xFF in the child process; otherwise it can
-# be decoded as-is if the default locale is latin-1.
+# surrogate-escaping of \xFF in the child process
 env['LC_ALL'] = 'C'
-if sys.platform.startswith("aix"):
-# On AIX, the C locale uses the Latin1 encoding
-decoded_value = encoded_value.decode("latin1", 
"surrogateescape")
-else:
-# On other UNIXes, the C locale uses the ASCII encoding
-decoded_value = value
+decoded_value = value
 stdout = subprocess.check_output(
 [sys.executable, "-c", script],
 env=env)

--
versions: +Python 3.8 -Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Michael Felt]

Michael Felt added the comment:

Seems to be okay at least with Python 3.6 "test" version - but maybe there is 
better way to call these tests - just to be sure the one needed is not being 
skipped.

On AIX 5.3 TL7 SP0:

root@x064:[/data/prj/aixtools/python/python-3.6.0.162/Lib/test]../../python 
test_subprocess.py
...s...s.s.s...s.s.s.s
--
Ran 254 tests in 113.796s

OK (skipped=24)

On AIX 6.1 TL9 SP4:

michael@x071:[/data/prj/aixtools/python/python-3.6.0.162/Lib/test]../../python 
test_subprocess.py
...s...s.s.s...s.s.s.s
--
Ran 254 tests in 126.684s

OK (skipped=24)
michael@x071:[/data/prj/aixtools/python/python-3.6.0.162/Lib/test]

Please note that GNU libiconv is used by default (version 1.14) rather than IBM 
iconv.

This may be the key issue (will test 11190 as well)

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[David Edelsohn]

Changes by David Edelsohn dje@gmail.com:


--
components: +Interpreter Core
nosy: +David.Edelsohn
type:  - behavior

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor added the comment:

ping myself

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 I don't understand why the test pass on FreeBSD, Solaris and
 Mac OS X, but not on AIX.

Oh, the command line and the filesystem encodings may be different. 
test_undecodable_env() of test_subprocess.py uses os.environ which uses 
sys.getfilesystemencoding(), whereas test_undecodable_code() of 
test_cmd_line.py uses _Py_char2wchar() which uses mbstowcs() functions (which 
use the locale encoding).

sys.getfilesystemencoding() is initialized from nl_langinfo(CODESET).

nl_langinfo(CODESET) gives maybe ISO-8859-1 (or an alias to this encoding) 
even if LC_ALL=C.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Sébastien Sablé]

Sébastien Sablé sa...@users.sourceforge.net added the comment:

I am not sure this is what you want:

 LC_ALL=C ./python
Python 3.2rc3 (py3k:88417M, Feb 14 2011, 10:37:42) 
[GCC 4.2.0] on aix6
Type help, copyright, credits or license for more information.
 import sys
 sys.getfilesystemencoding()
'iso8859-1'
 

What information can I provide to help solve this bug?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

 sys.getfilesystemencoding()
'iso8859-1'

Ok, I expected this result.

Can you also try:

$ LC_ALL=C ./python -c import sys; print(ascii(sys.argv)) $(echo -ne 
abc\xff)
['-c', 'abc\udcff']
$ LC_ALL=C python3.1 -c import locale; 
print(locale.nl_langinfo(locale.CODESET))
ANSI_X3.4-1968
$ LC_ALL=C python3.1 -c import locale; print(locale.getpreferredencoding())
ANSI_X3.4-1968

Anyway, test_undecodable_env() is not written to support ISO-8859-1 filesystem 
encoding. I should patch the test to check the filesystem encoding.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Sébastien Sablé]

Sébastien Sablé sa...@users.sourceforge.net added the comment:

$ LANG=C ./python -Wd -E -bb -c import sys; print(ascii(sys.argv)) $(echo -ne 
abc\xff)['-c', 'abc\xff']
['-c', 'abc\xff']
$ LANG=C ./python -Wd -E -bb -c import locale; 
print(locale.nl_langinfo(locale.CODESET))
ISO8859-1
$ LANG=C ./python -Wd -E -bb -c import locale; 
print(locale.getpreferredencoding())
ISO8859-1

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Sébastien Sablé]

New submission from Sébastien Sablé sa...@users.sourceforge.net:

The following tests fail in test_subprocess on AIX:

==
FAIL: test_undecodable_env (test.test_subprocess.POSIXProcessTestCase)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_subprocess.py,
 line 1039, in test_undecodable_env
self.assertEqual(stdout.decode('ascii'), ascii(value))
AssertionError: 'abc\\xff' != 'abc\\udcff'
- 'abc\xff'
?  ^
+ 'abc\udcff'
?  ^^^


==
FAIL: test_undecodable_env (test.test_subprocess.ProcessTestCasePOSIXPurePython)
--
Traceback (most recent call last):
  File 
/san_cis/home/cis/.buildbot/python-aix6/3.x.phenix.xlc/build/Lib/test/test_subprocess.py,
 line 1039, in test_undecodable_env
self.assertEqual(stdout.decode('ascii'), ascii(value))
AssertionError: 'abc\\xff' != 'abc\\udcff'
- 'abc\xff'
?  ^
+ 'abc\udcff'
?  ^^^


--
Ran 267 tests in 366.280s

FAILED (failures=2, skipped=22)

I haven't investigated yet.

--
messages: 128402
nosy: sable
priority: normal
severity: normal
status: open
title: test_subprocess error on AIX
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Perhaps related to the test_locale failure in issue11190.

--
nosy: +haypo, pitrou

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11193] test_subprocess error on AIX

[STINNER Victor]

STINNER Victor victor.stin...@haypocalc.com added the comment:

test_undecodable_code() in test_cmd_line had a similar issue: on FreeBSD, 
Solaris and Mac OS X, even if the locale is C (and nl_langinfo(CODESET) 
announces ASCII), _Py_char2wchar() (mbstowcs) decoded b'\xff' as '\xff' (use 
ISO-8859-1 encoding). To fix the test, I just patched the test to accept both 
results: b'\xff' may be decoded as '\udcff' or '\xff'.


test_undecodable_env does something like: os.environb[b'test']=b'abc\xff', but 
os.getenv() decodes b'abc\xff' as 'abc\xff' instead of 'abc\udcff' even if 
LC_ALL=C.

I don't understand why the test pass on FreeBSD, Solaris and Mac OS X, but not 
on AIX. It would be interesting to get the locale encoding of the child process 
using LC_ALL=C.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11193
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com