[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: I can confirm that the behavior is fixed in ZFS on Linux. My test case C program now prints "Everything is okay." when run on a ZFS partition on Linux, and test_touch_common from the current tree passes every time. ZFS fixing this was the best possible out

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Irit Katriel added the comment: issue15745 has been fixed in the meantime (about 6 years ago). If nobody objects I will close this as out of date. If there is still a problem with the test is will be reported for a current python version. -- nosy: +iritkatriel resolution: -> out of

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Changes by Wouter van Heyst : -- nosy: +larstiq ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.py

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: Result: Before: 1407219004 988716892 After: 1407219004 0 Before <= After 0 This is insane! -- ___ Python tracker ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Changes by Jesús Cea Avión : -- nosy: +jcea ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: Attached is a simple C test case for the same behavior. On my Linux machine this prints "This is insane!" on a ZFS partition and "Everything is okay" on an ext4 partition. The test case won't work unmodified on FreeBSD. Try commenting out the call to "utime

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Changes by Ezio Melotti : -- nosy: +ezio.melotti ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: And to further confirm, on my Linux machine with a ZFS partition: * stat() returns nanosecond resolution. * I have utimensat(), which supports nanosecond resolution. * The test code correctly sets "utime.now = 1", so it correctly calls utimensat() with a time p

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: I talked to puppet on IRC for a while and we figured out the following about his OS: * He has utime() and utimes(), but no utimensat(). * utimes() can write with *microsecond* resolution. * stat() reads the time with *nanosecond* resolution. (He has HAVE_STAT

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: Larry: Result of that: (gdb) print *utime Structure has no component named operator*. (gdb) print tv No symbol "tv" in current context. You can also catch me on IRC with the same nickname as here -- ___ Python track

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: Yup, you've got a debug build there. I'm baffled by that gdb "no debug info" line then. At line 4801 could you also print *utime print tv -- ___ Python tracker

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: To confirm, my OPT-line is OPT=-g -O0 -Wall -Wstrict-prototypes -- ___ Python tracker ___ _

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: I used configure with --with-pydebug, will look closer on it when I get to the office -- ___ Python tracker ___ ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: The "no debug info" makes me think you're using a release build. Please recompile making sure that "-O0" and "-g" are both set. By default the makefile has "-O3" and "-g", so normally you just change the "-O3" to "-O0", then "make clean" and "make". ---

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: I'm no expert at all at GDB, but I will give it a go and if this is wrong I can try a lot more. Breakpoint 1, posix_utime (self=0xf00918ed8, args=0xf014459f8, kwargs=0x0) at ./Modules/posixmodule.c:4838 4838PyObject *times = NULL; Current language: a

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: And was the "time" argument passed in to utimes() set to NULL? -- ___ Python tracker ___ ___ Python-

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: @Larry - we did that yesterday and on the FreeBSD-machine the regular utimes was used and not utimensat. utimensat is not available on FreeBSD 10 so it fall backs to regular utimes to be used in os.utime. But when a file is created it is the file system that s

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: A cursory read of the source code suggests everybody is doing the right thing. Can you run this with a debug build of CPython, put a breakpoint in posix_utime, and check that when it's called for the second p.touch that it's correctly calling the underlying f

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: @Larry - the problem arrives after the second touch, which does os.utime(FILE, None) if the file exists. The os.utime that is in the test-code doesn't really break anything, what breaks the test is the second touch. -- __

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: So why can't you use the "ns" parameter for os.utime()? -- ___ Python tracker ___ ___ Python-bugs-li

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: @Larry The issue is the following: In the test after the first p.touch (Pdb) old_mtime 1407128672.4133856 (Pdb) old_mtime_ns 1407128672413385711 After second p.touch (Pdb) st.st_mtime_ns 14071286890 (Pdb) st.st_mtime 1407128689.0 So the issue is that

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Francis MB added the comment: > What we have found out so far is that during file creation the > resolution of the timestamp is higher then at the touch attempt > when a file exists. Could it help to create 2 files (file 1, wait a bit, file 2) and then do the checks only with file 1? -

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: os.stat can return something more precise than nanosecond resolution? -- ___ Python tracker ___ ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: So me and @larstiq have been sitting with this during the sprint here and we have figured out some things but haven't had the time to put together a patch. If you put two p.touch() before starting the testing, it will not fail on BSD. What we have found out s

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: #15745 is probably related to this issue -- ___ Python tracker ___ ___ Python-bugs-list mailing lis

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Claudiu Popa added the comment: I agree. The patch was merely a temporary solution to the real problem. -- ___ Python tracker ___ ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: After thinking and looking a bit on it, this patch isn't the best to put into the code what I can see. If any of these things really break, because of something completely different - we won't notice it. A more accurate solution is to maybe look on a differe

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Daniel Eriksson added the comment: @Antoine: We have investigated a bit here and it seems that ZFS just keeps a timestamp in seconds in the memory. See function uberblock_update in uberblock.c ( http://people.freebsd.org/~gibbs/zfs_doxygenation/html/d3/d65/uberblock_8c_source.html - Row 57)

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Antoine Pitrou added the comment: So, I'm not against the patch, but it would be nice to diagnose where exactly the issue comes from. -- stage: -> patch review ___ Python tracker _

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: (By "this approach" I meant "the approach employed in the first patch posted". Sorry for the ambiguity.) -- ___ Python tracker ___ ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: Sorry to have such an awful configuration, but this approach won't fix the problem for me. I use a Linux "encrypted home directory", which uses crazy loopback mount logic to create an on-the-fly decrypted representation of my home directory. So my home direc

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Larry Hastings added the comment: This test also fails on Linux when using ZFS. -- nosy: +larry ___ Python tracker ___ ___ Python-bugs

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Claudiu.Popa added the comment: Here's a patch that marks those tests as expected failures if the source checkout is inside a zfs container. It uses `df -t zfs`, it was the easiest way I could find to detect that we are running from a zfs container. -- keywords: +patch Added file: http

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

koobs added the comment: I'd like to put the buildbot slave instances back onto ZFS for broader Disk/IO test coverage for Python and other projects as well as to gain some administrative disk utilisation flexibility. These two issues have unfortunately precluded that, and there's much more val

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Claudiu.Popa added the comment: Since issue15745 hasn't been fixed yet, would be okay to skip these tests when the test suite runs from a ZFS container? Currently, these failures are a nuissance when running the test suite. -- ___ Python tracker

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Claudiu.Popa added the comment: I believe it's similar, both test_os and test_pathlib fails when executed from within a ZFS container. I checked, I did a fresh checkout of Python inside a normal directory and run the tests there, they ran without problems. -- _

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

koobs added the comment: Sorry Claudiu I missed the issue reference in your comment -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

koobs added the comment: Is this similar/related to #15745? I took both of my buildbots (koobs-freebsd9, koobs-freebsd10) off ZFS until it could be resolved -- nosy: +koobs ___ Python tracker

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Antoine Pitrou added the comment: As a data point, if not for ZFS, test_pathlib passes on FreeBSD 6.4: http://buildbot.python.org/all/builders/x86%20FreeBSD%206.4%203.x/builds/4261/steps/test/logs/stdio and FreeBSD 7.2: http://buildbot.python.org/all/builders/x86%20FreeBSD%207.2%203.x/builds/4731

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Antoine Pitrou added the comment: I don't really know what to do with this. I think you'll have to investigate a bit and find out exactly what happens during the test. -- ___ Python tracker ___

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

Changes by Ned Deily : -- nosy: +pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org

[issue19838] test.test_pathlib.PosixPathTest.test_touch_common fails on FreeBSD with ZFS

New submission from Claudiu.Popa: Hi! test_touch_common fails when using 8.3-STABLE FreeBSD 8.3-STABLE and Python 3.4.0b1 (default:a0ec33efa743+, Nov 30 2013, 10:36:58). Here are the tracebacks: == FAIL: test_touch_common (