[issue12556] Disable size checks in mmap.mmap()
Josh Triplett added the comment: This rejection is indeed problematic. If mmap.mmap receives an explicit size, checking that size against stat will break on devices or special files. It's perfectly reasonable that mmap.mmap's automatic logic when passed length=0 (to map the entire file) won't work if stat says the file has length 0, but the mmap syscall works fine on special files with length 0 (or devices), and mmap.mmap should work in that case as well. -- nosy: +joshtriplett ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Charles-François Natali added the comment: From a cursory inspection, we only check the size for regular files. What exception are you seeing exactly? What does stat return on this file? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Charles-François Natali added the comment: Scratch that. I'll try to work on a patch that doesn't break guarantees for regular files (we try to limit the possibilities of segfault). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Ricky Ng-Adam rnga...@lophilo.com added the comment: I find this resolution to rejected somewhat questionable. It has been pointed that the mmap behavior in Python differs from the behavior of the underlying mmap as defined by the system documentation. I think the incorrect assumption here is that mmap is used to only map real files. In the case of virtual files such those found in sysfs and debugfs in Linux, mmap is a general operation that lets the underlying driver arbitrarily map to other memory regions. As an example, we are developing a new hardware platform. To maximize performance, instead of creating a file-based interface to the Linux driver, we are experimenting with mmap'ing the hardware memory address to the userspace by implementing the mmap operation in our Linux kernel module. This mmap operation works in C and in Node.JS, but not in Python where it can't get past the underlying debugfs (or sysfs) file having a size of 0... -- nosy: +rngadam ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Ross Lagerwall rosslagerw...@gmail.com added the comment: I don't like the idea of adding an argument which doesn't have a counterpart in the POSIX version (especially to address such corner cases). Indeed, it seems rather messy for a corner case that may well not exist. If there are definite cases where this usage is needed, a solution may then have to be considered. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Sergei Lebedev superbo...@gmail.com added the comment: Do you have an example of a /proc entry with st_size == 0 that can be mmapped (mapping /proc/sys/debug/exception-trace fails with EACCESS)? Yes, I've ran into the issue, while trying to mmap /proc/xen/xsd_kva, which is an interface to XenBus [1]. Unfortunately, I'm not aware of any other cases. By the way, I've checked mmap(2) manpage -- it looks like the C-version has nothing against mmaping 0-sized files, Why does Python's `mmap` still checks file size? [1] http://wiki.xensource.com/xenwiki/XenBus -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Charles-François Natali neolo...@free.fr added the comment: By the way, I've checked mmap(2) manpage -- it looks like the C-version has nothing against mmaping 0-sized files, Why does Python's `mmap` still checks file size? It doesn't check explicitely that the size is non-0, but rather that the offset is is less than the file size. Passing mmap(2) a 0 length doesn't make much sense anyway - for example, Linux returns EINVAL: http://lxr.free-electrons.com/source/mm/mmap.c?a=avr32#L979 unsigned long do_mmap_pgoff(...) [...] if (!len) return -EINVAL; Do you have an example of a /proc entry with st_size == 0 that can be mmapped (mapping /proc/sys/debug/exception-trace fails with EACCESS)? Yes, I've ran into the issue, while trying to mmap /proc/xen/xsd_kva, which is an interface to XenBus [1]. Unfortunately, I'm not aware of any other cases. That's what I thought, it's really uncommon: in that case, I'm reluctant to making such a change, for the reason explained above. Ross, Victor? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
STINNER Victor victor.stin...@haypocalc.com added the comment: That's what I thought, it's really uncommon: in that case, I'm reluctant to making such a change, for the reason explained above. Ross, Victor? Why do you want a mmap? Why not using a file? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Changes by Ralf Schmitt python-b...@systemexit.de: -- nosy: +schmir ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Ross Lagerwall rosslagerw...@gmail.com added the comment: That's what I thought, it's really uncommon: in that case, I'm reluctant to making such a change, for the reason explained above. Ross, Victor? Even if Linux did accept a a length of 0 (which it doesn't since 2.6.12), what would be the point? Wouldn't you be mapping *nothing*? Yeah, I'm against this change. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Sergei Lebedev superbo...@gmail.com added the comment: Passing mmap(2) a 0 length doesn't make much sense anyway - for example, Linux returns EINVAL. Yup, but passing mmap(2) a zero-sized file and a non-zero length works just fine. Why do you want a mmap? Why not using a file? Because Xen requires me to mmap it. Anyway, you're right, the use-case is too uncommon. -- resolution: - rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Charles-François Natali neolo...@free.fr added the comment: A couple remarks: - if st_size == 0, then you have to specify an explicit length (your example passes 0, which would fail with EINVAL without the check) - most enties in /proc don't support mmap file operation, so it's likely to fail anyway (with ENODEV, EACCESS...). Do you have an example of a /proc entry with st_size == 0 that can be mmapped (mapping /proc/sys/debug/exception-trace fails with EACCESS)? How about adding a keyword argument to `mmap.mmap()`, which disables fstat-based size checks? I don't like the idea of adding an argument which doesn't have a counterpart in the POSIX version (especially to address such corner cases). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
New submission from Sergei Lebedev superbo...@gmail.com: Current `mmap` implementation raises a ValueError if a sum of offset and length exceeds file size, as reported by `fstat`. While perfectly valid for most use-cases, it doesn't work for special files, for example: with open(/proc/sys/debug/exception-trace, r+b) as f: ... mmap.mmap(f.fileno(), 0) ... Traceback (most recent call last): File stdin, line 2, in module ValueError: mmap offset is greater than file size Same goes for almost any other /proc file, because most of them have S_ISREG() == True and st_size = 0. How about adding a keyword argument to `mmap.mmap()`, which disables fstat-based size checks? -- components: Library (Lib) messages: 140330 nosy: superbobry priority: normal severity: normal status: open title: Disable size checks in mmap.mmap() type: feature request versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12556] Disable size checks in mmap.mmap()
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +neologix, rosslagerwall versions: +Python 3.3 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12556 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
