[issue41625] Add splice() to the os module

Michael Felt added the comment: Sorry Victor - family matters - so I was not watching for several days. -- ___ Python tracker ___

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: New changeset dedc2cd5f02a2e2fc2c995a36a3dccf9d93856bb by Pablo Galindo in branch 'master': bpo-41625: Do not add os.splice on AIX due to compatibility issues (GH-23608) https://github.com/python/cpython/commit/dedc2cd5f02a2e2fc2c995a36a3dccf9d93856bb

[issue41625] Add splice() to the os module

STINNER Victor added the comment: > Seems that adding #include does not work so I am going to skip > adding this function on AIX. I'm fine with not implementing the function on AIX for now. -- ___ Python tracker

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: Seems that adding #include does not work so I am going to skip adding this function on AIX. If someone is interested in fixing it, they can remove the #ifdef and figure out what's going on with that buildbot --

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: Started custom build of PR 23608 in https://buildbot.python.org/all/#/buildrequests/84365 -- ___ Python tracker ___

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: https://buildbot.python.org/all/#/builders/526/builds/3 -- ___ Python tracker ___ ___

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- pull_requests: +22476 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/23608 ___ Python tracker ___

[issue41625] Add splice() to the os module

Change by STINNER Victor : -- resolution: fixed -> status: closed -> open ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue41625] Add splice() to the os module

STINNER Victor added the comment: > ./Modules/posixmodule.c:10429:15: error: implicit declaration of function > 'splice'; did you mean 'plock'? [-Werror=implicit-function-declaration] Is it possible that posixmodule.c lacks an #include to get the function on AIX? On AIX 7.1, man splice

[issue41625] Add splice() to the os module

Michael Felt added the comment: This is still broken. Since this was included in master - the AIX buildbot is failing to compile (https://buildbot.python.org/all/#/builders/438/builds/391 and https://buildbot.python.org/all/#/builders/302/builds/377) Strangely enough - the first bot

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: Thanks a lot Victor -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue41625] Add splice() to the os module

STINNER Victor added the comment: FYI I checked and AIX is fixed. All tests pass again on POWER6 AIX 3.x buildbot. -- ___ Python tracker ___

[issue41625] Add splice() to the os module

STINNER Victor added the comment: New changeset 1de61d3923840b29e847d311f0c7d4c5821d98e6 by Victor Stinner in branch 'master': bpo-41625: Skip os.splice() tests on AIX (GH-23354) https://github.com/python/cpython/commit/1de61d3923840b29e847d311f0c7d4c5821d98e6 --

[issue41625] Add splice() to the os module

STINNER Victor added the comment: Nice, AIX can build again Python. But now the 3 tests fail since the test uses a pipe and a file, whereas on AIX, it seems like splice() requires one end to be a socket. I wrote attached PR 23354 to skip the 3 tests on AIX.

[issue41625] Add splice() to the os module

Change by STINNER Victor : -- pull_requests: +22247 pull_request: https://github.com/python/cpython/pull/23354 ___ Python tracker ___

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue41625] Add splice() to the os module

miss-islington added the comment: New changeset 2a9eddf070f72060f62db1856a0af2e08729a46c by Pablo Galindo in branch 'master': bpo-41625: Add a guard for Linux for splice() constants in the os module (GH-23350) https://github.com/python/cpython/commit/2a9eddf070f72060f62db1856a0af2e08729a46c

[issue41625] Add splice() to the os module

miss-islington added the comment: New changeset e59958f8b6815f51f6c33b6a613cf8467ca18a11 by Pablo Galindo in branch 'master': bpo-41625: Specify that Linux >= 2.6.17 *and* glibc >= 2.5 are requir… (GH-23351) https://github.com/python/cpython/commit/e59958f8b6815f51f6c33b6a613cf8467ca18a11

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- pull_requests: +22245 pull_request: https://github.com/python/cpython/pull/23351 ___ Python tracker ___

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- pull_requests: +22244 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/23350 ___ Python tracker ___

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: New changeset fa96608513b6eafe48777f1a5504134939dcbebc by Pablo Galindo in branch 'master': bpo-41625: Add versionadded to os.splice() constants (GH-23340) https://github.com/python/cpython/commit/fa96608513b6eafe48777f1a5504134939dcbebc --

[issue41625] Add splice() to the os module

STINNER Victor added the comment: > The splice() system call first appeared in Linux 2.6.17; > library support was added to glibc in version 2.5. There is no emulation. It's just a function which wraps the syscall:

[issue41625] Add splice() to the os module

STINNER Victor added the comment: I reopen the issue. This issue broke Python compilation on AIX. https://buildbot.python.org/all/#/builders/302/builds/377 configure: "checking for splice... yes" "./Modules/posixmodule.c", line 15146.53: 1506-045 (S) Undeclared identifier SPLICE_F_MOVE.

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- pull_requests: +22231 pull_request: https://github.com/python/cpython/pull/23340 ___ Python tracker ___

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: > Do you mean "Linux kernel >= 2.6.17 and glibc >= 2.5" ? My understanding is that glibc provides emulation for glibc >= 2.5 The section from the manpage says: The splice() system call first appeared in Linux 2.6.17; library support was

[issue41625] Add splice() to the os module

STINNER Victor added the comment: > .. availability:: Linux kernel >= 2.6.17 or glibc >= 2.5 Do you mean "Linux kernel >= 2.6.17 and glibc >= 2.5" ? > .. data:: SPLICE_F_MOVE Maybe also add ".. versionadded:: 3.10" on these constants. --

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: New changeset a57b3d30f66c90f42da751bf82256b9b22961ed0 by Pablo Galindo in branch 'master': bpo-41625: Expose the splice() system call in the os module (GH-21947) https://github.com/python/cpython/commit/a57b3d30f66c90f42da751bf82256b9b22961ed0

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: Heads up: I plant to land this next week in case someone could to do a review or has something against -- ___ Python tracker ___

[issue41625] Add splice() to the os module

Change by hai shi : -- nosy: +shihai1991 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: > Since it is Linux-specific API, would not be better to add a purposed module > linux? This is an interesting point, but I think that at this particular point it would be more confusing for users than not (normally people go to the os module for

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: > OpenBSD uses a different API: The semantics are considerably different (splice() is about pipes while sosplice() talks about general sockets). Also, the point of splice() is to skip copying from kernel buffers, but sosplice() does not mention that

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: > The API of splice() looks complicated. How would you use it in Python? It has the same API as copy_file_range and other similar system calls that we already expose, so we just need to do the same thing we do there. > Are off_in and off_out adjusted

[issue41625] Add splice() to the os module

STINNER Victor added the comment: OpenBSD uses a different API: https://man.openbsd.org/sosplice.9 int sosplice(struct socket *so, int fd, off_t max, struct timeval *tv); int somove(struct socket *so, int wait); "The function sosplice() is used to splice together a source and a drain

[issue41625] Add splice() to the os module

STINNER Victor added the comment: > Are you going to add vmsplice() and tee() too? Since it is Linux-specific > API, would not be better to add a purposed module linux? It's not uncommon that a syscall added to the Linux kernel is later added to other platforms. Example: getrandom() exists

[issue41625] Add splice() to the os module

Serhiy Storchaka added the comment: The API of splice() looks complicated. How would you use it in Python? Are off_in and off_out adjusted as in copy_file_range() and sendfile()? It is not clear from the man page. If they are, how would you return updated values? Are you going to add

[issue41625] Add splice() to the os module

Change by Dong-hee Na : -- nosy: +corona10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue41625] Add splice() to the os module

Pablo Galindo Salgado added the comment: > I don't recall the subtle differences between sendfile() and splice(). Basically, splice() is specialized for pipes: splice() only works if one of the file descriptors refer to a pipe. So you can use for e.g. socket-to-pipe or pipe-to-file without

[issue41625] Add splice() to the os module

STINNER Victor added the comment: I don't recall the subtle differences between sendfile() and splice(). I recall that in early Linux versions, one was limited to sockets, and only on one side. But later, it became possible to pass two sockets, or one file on disk and one socket, etc.

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- keywords: +patch pull_requests: +21057 stage: -> patch review pull_request: https://github.com/python/cpython/pull/21947 ___ Python tracker

[issue41625] Add splice() to the os module

Change by Pablo Galindo Salgado : -- assignee: -> pablogsal components: +Library (Lib) versions: +Python 3.10 ___ Python tracker ___

[issue41625] Add splice() to the os module

New submission from Pablo Galindo Salgado : The splice system call moves data between two file descriptors without copying between kernel address space and user address space. This can be a very useful addition for libraries implementing low-level file management. -- messages: