[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ned Deily]

Ned Deily  added the comment:

I've chosen to unconditionally skip test_undecodable_filename on macOS; it was 
already skipped for older macOS systems and I think it would be a mistake to 
test for a specific error code that could change in later releases.  As Ronald 
points out, testing on Linux should be sufficient.  Thanks everyone for your 
input.  Fixed in 3.6.4 and 3.7.0.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ned Deily]

Ned Deily  added the comment:


New changeset d9cadc5f597e5966132c9249f6c7ff0ed1eba0cb by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
[3.6] bpo-31380: Skip test_httpservers test_undecodable_file on macOS. 
(GH-4720) (#4721)
https://github.com/python/cpython/commit/d9cadc5f597e5966132c9249f6c7ff0ed1eba0cb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4630

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ned Deily]

Ned Deily  added the comment:


New changeset b3edde8dd44c878e9f039a2165d00ff075157d4b by Ned Deily in branch 
'master':
bpo-31380: Skip test_httpservers test_undecodable_file on macOS. (#4720)
https://github.com/python/cpython/commit/b3edde8dd44c878e9f039a2165d00ff075157d4b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ned Deily]

Change by Ned Deily :


--
keywords: +patch
pull_requests: +4629
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Another option is to skip this test unconditionally on macOS when using APFS, 
the test is already skipped on win32 and tests a generic codepath where only 
testing on Linux should be fine.

BTW. It is fairly easy to detect if a path is on APFS:

```
import subprocess
  
def is_apfs(path):
lines = subprocess.check_output(['df', path]).decode('utf-8').splitlines()
mountpoint = lines[1].split(None, 1)[0]

lines = subprocess.check_output(['mount']).decode('utf-8').splitlines()
for ln in lines:
path = ln.split(None, 1)[0]
if path == mountpoint:
return '(apfs' in ln

return False
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[STINNER Victor]

STINNER Victor  added the comment:

> @vstinner, as the expert in this area, what do you suggest as the best way to 
> deal with this?  In support.__init__.py?  Or test_httpservers itself?

It's tricky when a filename is valid or not depending on the path, depending on 
the filesystem. The tempfile module checks if O_TMPFILE is supported: same 
issue, it depends on the path, and the user can configure the path. So we check 
each time if O_TMPFILE works or not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I suggest to catch an error and skip the test if OSError with the specific 
errno has been raised.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Ned Deily]

Ned Deily  added the comment:

More background: APFS became available in macOS 10.12 although not used by 
default.  But with the current macOS 10.13 (High Sierra), root file systems on 
certain devices, such as SSDs, are automatically migrated from HFS+ to APFS so 
are much more likely to be seen starting in 10.13 and, in general, there is no 
easy way of knowing what type of FS a particular file resides on.

# TMPDIR on APFS volume
$ ./bin/python3.7 -m test test_httpservers
Run tests sequentially
0:00:00 load avg: 1.43 [1/1] test_httpservers
test test_httpservers failed -- Traceback (most recent call last):
  File ".../lib/python3.7/test/support/__init__.py", line 598, in wrapper
return func(*args, **kw)
  File ".../lib/python3.7/test/test_httpservers.py", line 395, in 
test_undecodable_filename
with open(os.path.join(self.tempdir, filename), 'wb') as f:
OSError: [Errno 92] Illegal byte sequence: 
'/var/folders/sn/0m4rnbyj2z1byjs68sz83830gn/T/tmpqeg89k76/@test_64005_tmp\udce7w\udcf0.txt'

test_httpservers failed

1 test failed:
test_httpservers

Total duration: 3 sec
Tests result: FAILURE
# TMPDIR redirected to HFS+ volume
$ TMPDIR=/Volumes/euterpe/a ./bin/python3.7 -m test test_httpservers
Run tests sequentially
0:00:00 load avg: 1.28 [1/1] test_httpservers
1 test OK.

Total duration: 3 sec
Tests result: SUCCESS

@vstinner, as the expert in this area, what do you suggest as the best way to 
deal with this?  In support.__init__.py?  Or test_httpservers itself?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
components: +Tests
type: crash -> behavior
versions:  -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Matt Billenstein]

Matt Billenstein  added the comment:

Reflected on the High Sierra buildbot now:  
http://buildbot.python.org/all/#/builders/14/builds/162

--
nosy: +mattbillenstein

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31380] test_undecodable_filename() in Lib/test/test_httpservers.py broken on APFS

[Jack Howarth]

New submission from Jack Howarth:

The Python 3.x test suite produces a new regression on macOS 10.13 under the 
new APFS filesystem when executing the test_undecodable_filename() test from 
Lib/test/test_httpservers.py. The error appears as...

==
ERROR: test_undecodable_filename
(test.test_httpservers.SimpleHTTPServerTestCase)
--
Traceback (most recent call last):
  File "/Users/howarth/Python-3.6.2/Lib/test/support/__init__.py",
line 601, in wrapper
return func(*args, **kw)
  File "/Users/howarth/Python-3.6.2/Lib/test/test_httpservers.py",
line 380, in test_undecodable_filename
with open(os.path.join(self.tempdir, filename), 'wb') as f:
OSError: [Errno 92] Illegal byte sequence:
'/var/folders/7g/1x2rsy3j40n1pydq931hzlkmgn/T/tmpomp1r36b/@test_58317_tmp\udce7w\udcf0.txt'

--
Ran 59 tests in 3.207s

See 
https://developer.apple.com/library/content/documentation/FileManagement/Conceptual/APFS_Guide/FAQ/FAQ.html

"""
APFS accepts only valid UTF-8 encoded filenames for creation, and preserves 
both case and normalization of the filename on disk in all variants..

Some differences between how APFS and HFS+ handle filenames include the 
following:
...
• APFS doesn’t allow files to be created with filenames that contain 
unassigned codepoints in the Unicode 9.0 standard, whereas HFS+ does.

--
components: macOS
messages: 301580
nosy: howarthjw, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: test_undecodable_filename() in Lib/test/test_httpservers.py broken on 
APFS
type: crash
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com