On Tue, Mar 12, 2019 at 3:01 AM Glenn Linderman <v+pyt...@g.nevcal.com> wrote:
> On 3/11/2019 4:35 PM, Giampaolo Rodola' wrote: > > Hello, > some time ago I contributed a couple of patches to speedup shutil.copy*() > functions: > https://bugs.python.org/issue33671 > https://bugs.python.org/issue33695 > I would like to backport both functionalities so that they can be used on > Python 2.7 and <3.8 and put it on PYPI. In order to do so I will basically > have to copy some parts of shutil module (copytree() function + the > unit-tests I added in BPO-33671 and a couple of other things). Are there > constraints regarding this in terms of license? Am I supposed to use GPL? > (I was thinking about using MIT) > > Note: in this package called "zerocopy" I will probably want to expose > other functionalities such as tee(), splice() and CopyFileEx and > TransmitFile on Windows, so it's probably gonna be half a backport and half > a brand new project. > > Thanks. > > > Thanks for the contributions. I don't know about the licensing. > > I wonder if you should make two packages, though... one just exactly a > backport of the shutil speedups, and the second containing the new > functionalities. > That was my initial thought as well (a "backports.shutil_copy" module targeting copy* functions only), but (especially after playing with this today) I think I have something a bit more ambitious in mind. I'm currently experimenting with different things which could be baked in a third-party lib and possibly contributed back to Python later on: 1) on OSX we could use f/copyfile() syscall to copy file attrs/metadata; that may be useful to speedup shutil.copystat() and shutil.copymode() 2) copytree() on OSX could take advantage of f/copyfile() + COPYFILE_RECURSIVE (which is probably too platform-specific for inclusion) 3) on Linux we could use copy_file_range() as a replacement for os.sendfile() in shutil.copyfile() (it's supposed to be faster) 4) on Linux ioctl() + FICLONE could be used to implement CoW (copy on write) instantaneous copies, and could be added as shutil.cow_copyfile() (haven't look into Windows yet) 5) I was thinking about backporting socket.socket.sendfile() as well, which uses os.sendfile() on POSIX but not TransmitFile on Windows (asyncio does though) 6) another idea (but I'm not sure if it's possible, as I still have to dig into that) is a new socket recvfile() function boosted up by tee() / splice() on Linux, which maybe could be contributed back as socket.socket.recvfile() -- Giampaolo - http://grodola.blogspot.com
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com