Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Steve Dower
On 22Oct2018 1047, Steve Dower wrote: On 22Oct2018 1007, Serhiy Storchaka wrote: 22.10.18 16:24, Steve Dower пише: Yes, that's true. But "should reduce ... footprint" is also an optimisation that deserves a benchmark by that standard. Also, I'm proposing keeping the 'kind' as UCS-2 when the st

Re: [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

2018-10-22 Thread Sean Harrington
On Mon, Oct 22, 2018 at 2:01 PM Michael Selik wrote: > This thread seems more appropriate for python-ideas than python-dev. > > > On Mon, Oct 22, 2018 at 5:28 AM Sean Harrington > wrote: > >> Michael - the initializer/globals pattern still might be necessary if you >> need to create an object A

Re: [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

2018-10-22 Thread Michael Selik
This thread seems more appropriate for python-ideas than python-dev. On Mon, Oct 22, 2018 at 5:28 AM Sean Harrington wrote: > Michael - the initializer/globals pattern still might be necessary if you > need to create an object AFTER a worker process has been instantiated (i.e. > a database conn

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Steve Dower
On 22Oct2018 1007, Serhiy Storchaka wrote: 22.10.18 16:24, Steve Dower пише: Yes, that's true. But "should reduce ... footprint" is also an optimisation that deserves a benchmark by that standard. Also, I'm proposing keeping the 'kind' as UCS-2 when the string is created from UCS-2 data that i

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Serhiy Storchaka
22.10.18 16:24, Steve Dower пише: Yes, that's true. But "should reduce ... footprint" is also an optimisation that deserves a benchmark by that standard. Also, I'm proposing keeping the 'kind' as UCS-2 when the string is created from UCS-2 data that is likely to be used as UCS-2. We would not c

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Serhiy Storchaka
22.10.18 11:09, Victor Stinner пише: +1 to remove wchar_t cache. IMHO it wastes memory for no real performance gain. By the way, can we start to schedule the *removal* of the Py_UNICODE API? For example, decide when Py_DEPRECATED is used in the C API? Should we start to deprecate when Python 2 r

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Steve Dower
On 22Oct2018 0928, Victor Stinner wrote: Also, I'm proposing keeping the 'kind' as UCS-2 when the string is created from UCS-2 data that is likely to be used as UCS-2. Oh. That's a major change in the PEP 393 design. You would have to modify many functions in CPython. Currently, the PEP 393 req

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Serhiy Storchaka
20.10.18 16:01, Stefan Behnel пише: But regarding the use under Windows, I wonder if there's interest in keeping it as a special Windows-only feature, e.g. to speed up the data exchange with the Win32 APIs. I guess it would have to provide a visible (performance?) advantage to justify such specia

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le lun. 22 oct. 2018 à 15:24, Steve Dower a écrit : > Yes, that's true. But "should reduce ... footprint" is also an > optimisation that deserves a benchmark by that standard. pyperformance has a mode to mesure the memory usage (mostly the memory peak) if someone wants to have a look. > Also, I'

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Steve Dower
On 22Oct2018 0913, Victor Stinner wrote: Le lun. 22 oct. 2018 à 15:08, Steve Dower a écrit : Agreed the cache is useless here, but since the listdir() result came in as wchar_t we could keep it that way (assuming we'd only be changing it to char), and then there wouldn't have to be a conversion

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le lun. 22 oct. 2018 à 15:08, Steve Dower a écrit : > Agreed the cache is useless here, but since the listdir() result came in > as wchar_t we could keep it that way (assuming we'd only be changing it > to char), and then there wouldn't have to be a conversion when we > immediately pass it back to

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Steve Dower
On 22Oct2018 0413, Victor Stinner wrote: For code like "for name in os.listdir(): open(name): " (replace listdir with scandir if you want to get file metadata), the cache is useless, since the fresh string has to be converted to wchar_t* anyway, and the cache is destroyed at the end of the lo

Re: [Python-Dev] bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals

2018-10-22 Thread Sean Harrington
Michael - the initializer/globals pattern still might be necessary if you need to create an object AFTER a worker process has been instantiated (i.e. a database connection). Further, the user may want to access all of the niceties of Pool, like imap, imap_unordered, etc. The goal (IMO) would be to

Re: [Python-Dev] wininst-*.exe files in Lib/distutils/command

2018-10-22 Thread Petr Viktorin
On 10/18/18 7:44 PM, VanL wrote: Primarily for non-windows platforms, but I also think for Windows users without any compilers or similar tools installed. There is also some discussion of removing some of the older toolchain-specific versions (leaving only -14), but that is a subject for anothe

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le sam. 20 oct. 2018 à 18:02, Steve Dower a écrit : > I don't have numbers, but my instinct says the most impacted operations > would be retrieving collections of strings from the OS (avoiding a > scan/conversion for each one), comparisons against these collections > (in-memory handling for hash/c

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Hi Serhiy, +1 to remove wchar_t cache. IMHO it wastes memory for no real performance gain. By the way, can we start to schedule the *removal* of the Py_UNICODE API? For example, decide when Py_DEPRECATED is used in the C API? Should we start to deprecate when Python 2 reachs its end of life? Or c

Re: [Python-Dev] Some PRs to merge?

2018-10-22 Thread Victor Stinner
Le sam. 20 oct. 2018 à 13:15, Serhiy Storchaka a écrit : > Thank you Victor! I prefer to merge my PRs and PRs assigned to me > myself, but I am not sure that I would merge all PRs that can be merged > in the nearest future. ;) Some PRs were blocked by me because I was nitpicking on something. I d