Re: [Python-Dev] Make re.compile faster

03.10.17 06:29, INADA Naoki пише: More optimization can be done with implementing sre_parse and sre_compile in C. But I have no time for it in this year. And please don't do this! This would make maintaining the re module hard. The performance of the compiler is less important than

Re: [Python-Dev] Make re.compile faster

03.10.17 06:29, INADA Naoki пише: Before deferring re.compile, can we make it faster? I profiled `import string` and small optimization can make it 2x faster! (but it's not backward compatible) Please open an issue for this. I found: * RegexFlag.__and__ and __new__ is called very often. *

[Python-Dev] Make re.compile faster

Before deferring re.compile, can we make it faster? I profiled `import string` and small optimization can make it 2x faster! (but it's not backward compatible) Before optimize: import time: self [us] | cumulative | imported package import time: 2339 | 9623 | string string module

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

On Wed, Sep 27, 2017 at 1:26 AM, Nick Coghlan wrote: > It's also the case that unlike Go channels, which were designed from > scratch on the basis of implementing pure CSP, FWIW, Go's channels (and goroutines) don't implement pure CSP. They provide a variant that the Go

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

On Mon, Sep 25, 2017 at 8:42 PM, Nathaniel Smith wrote: > It's fairly reasonable to implement a mutex using a CSP-style > unbuffered channel (send = acquire, receive = release). And the same > trick turns a channel with a fixed-size buffer into a bounded > semaphore. It won't be

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

On Mon, Oct 2, 2017 at 9:31 PM, Eric Snow wrote: > On DECREF there shouldn't be a problem except possibly with a small > race between decrementing the refcount and checking for a refcount of > 0. We could address that several different ways, including allowing > the

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

After having looked it over, I'm leaning toward supporting buffering, as well as not blocking by default. Neither adds much complexity to the implementation. On Sat, Sep 23, 2017 at 5:45 AM, Antoine Pitrou wrote: > On Fri, 22 Sep 2017 19:09:01 -0600 > Eric Snow

Re: [Python-Dev] PEP 554 v3 (new interpreters module)

On Thu, Sep 14, 2017 at 8:44 PM, Nick Coghlan wrote: > Not really, because the only way to ensure object separation (i.e no > refcounted objects accessible from multiple interpreters at once) with > a bytes-based API would be to either: > > 1. Always copy (eliminating most of

Re: [Python-Dev] Investigating time for `import requests`

Op 3 okt. 2017 om 04:29 heeft Barry Warsaw het volgende geschreven: > On Oct 2, 2017, at 14:56, Brett Cannon wrote: > >> So Mercurial specifically is an odd duck because they already do lazy >> importing (in fact they are using the lazy loading support

Re: [Python-Dev] PEP 553

On Oct 2, 2017, at 18:43, Guido van Rossum wrote: > > OK. That then concludes the review of your PEP. It is now accepted! Congrats. > I am looking forward to using the backport. :-) Yay, thanks! We’ll see if I can sneak that backport past Ned. :) -Barry signature.asc

Re: [Python-Dev] PEP 553

On Mon, Oct 2, 2017 at 3:03 PM, Barry Warsaw wrote: > On Oct 2, 2017, at 17:36, Guido van Rossum wrote: > > > I've seen your updates and it is now acceptable, except for *one* nit: > in builtins.breakpoint() the pseudo code raises RuntimeError if >

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

On Oct 3, 2017 01:11, "Koos Zevenhoven" wrote: On Oct 3, 2017 01:00, "Guido van Rossum" wrote: Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven wrote I don't mind this (or Nathaniel ;-) being academic. The backwards > incompatibility issue

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

On Oct 3, 2017 01:00, "Guido van Rossum" wrote: Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven wrote I don't mind this (or Nathaniel ;-) being academic. The backwards > incompatibility issue I've just described applies to any extension via > composition, if

Re: [Python-Dev] PEP 553

On Oct 2, 2017, at 17:36, Guido van Rossum wrote: > I've seen your updates and it is now acceptable, except for *one* nit: in > builtins.breakpoint() the pseudo code raises RuntimeError if > sys.breakpointhook is missing or None. OTOH sys.breakpointhook() just issues > a

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven wrote > I don't mind this (or Nathaniel ;-) being academic. The backwards > incompatibility issue I've just described applies to any extension via > composition, if the underlying type/protocol grows new members (like the CM >

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

On Oct 3, 2017 00:02, "Guido van Rossum" wrote: On Mon, Oct 2, 2017 at 10:13 AM, Koos Zevenhoven wrote: > Hi all, It was suggested that I start a new thread, because the other > thread drifted away from its original topic. So here, in case someone is >

Re: [Python-Dev] PEP 553

On Mon, Oct 2, 2017 at 11:02 AM, Barry Warsaw wrote: > Thanks for the review Guido! The PEP and implementation have been updated > to address your comments, but let me briefly respond here. > > > On Oct 2, 2017, at 00:44, Guido van Rossum wrote: > > > -

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

On Mon, Oct 2, 2017 at 10:13 AM, Koos Zevenhoven wrote: > Hi all, It was suggested that I start a new thread, because the other > thread drifted away from its original topic. So here, in case someone is > interested: > > On Oct 2, 2017 17:03, "Koos Zevenhoven

Re: [Python-Dev] Investigating time for `import requests`

On Mon, 2 Oct 2017 11:15:35 -0400 Barry Warsaw wrote: > > I think there are opportunities for an explicit API for lazy compilation of > regular expressions, but I’m skeptical of the adoption curve making it > worthwhile. But maybe I’m wrong! We already have two caching

Re: [Python-Dev] Investigating time for `import requests`

On Mon, 02 Oct 2017 18:56:15 + Brett Cannon wrote: > > So Mercurial specifically is an odd duck because they already do lazy > importing (in fact they are using the lazy loading support from importlib). Do they? I was under the impression they had their own home-baked,

Re: [Python-Dev] PEP 553

On 10/2/2017 10:45 AM, Guido van Rossum wrote: On Sun, Oct 1, 2017 at 11:15 PM, Terry Reedy > wrote: On 10/2/2017 12:44 AM, Guido van Rossum wrote: - There's no rationale for the *args, **kwds part of the breakpoint() signature.

Re: [Python-Dev] Investigating time for `import requests`

On Oct 2, 2017, at 14:56, Brett Cannon wrote: > So Mercurial specifically is an odd duck because they already do lazy > importing (in fact they are using the lazy loading support from importlib). > In terms of all of this discussion of tweaking import to be lazy, I think the

Re: [Python-Dev] Intention to accept PEP 552 soon (deterministic pyc files)

On Sat, 30 Sep 2017 at 18:46 Benjamin Peterson wrote: > What do you mean by bytecode-specific APIs? The internal importlib ones? > There's that, but more specifically py_compile and compileall. -Brett > > On Fri, Sep 29, 2017, at 09:38, Brett Cannon wrote: > > BTW, if

Re: [Python-Dev] Python startup optimization: script vs. service

On Mon, 2 Oct 2017 at 11:19 Christian Heimes wrote: > On 2017-10-02 19:29, Brett Cannon wrote: > > My current design for an opt-in lazy importing setup includes an > > explicit function for importlib that's mainly targeted for the stdlib > > and it's startup module needs,

Re: [Python-Dev] Investigating time for `import requests`

On Mon, 2 Oct 2017 at 02:43 Raymond Hettinger wrote: > > > On Oct 2, 2017, at 12:39 AM, Nick Coghlan wrote: > > > > "What requests uses" can identify a useful set of > > avoidable imports. A Flask "Hello world" app could likely provide > >

Re: [Python-Dev] Python startup optimization: script vs. service

On 2017-10-02 19:29, Brett Cannon wrote: > My current design for an opt-in lazy importing setup includes an > explicit function for importlib that's mainly targeted for the stdlib > and it's startup module needs, but could be used by others: >

Re: [Python-Dev] PEP 553

Thanks for the review Guido! The PEP and implementation have been updated to address your comments, but let me briefly respond here. > On Oct 2, 2017, at 00:44, Guido van Rossum wrote: > - There's a comma instead of a period at the end of the 4th bullet in the > Rationale:

Re: [Python-Dev] Investigating time for `import requests`

On 10/2/2017 4:57 AM, Paul Moore wrote: In practice, I don't think the fact that re.search() et al cache the compiled expressions is that well known (it's mentioned in the re.compile docs, but not in the re.search docs) We could add redundant mentions in the functions ;-). and so people

Re: [Python-Dev] Python startup optimization: script vs. service

On Mon, 2 Oct 2017 at 08:00 Christian Heimes wrote: > On 2017-10-02 15:26, Victor Stinner wrote: > > 2017-10-02 13:10 GMT+02:00 INADA Naoki : > >> https://github.com/python/cpython/pull/3796 > >> In this PR, lazy loading only happens when uuid1 is

[Python-Dev] Inheritance vs composition in backcompat (PEP521)

Hi all, It was suggested that I start a new thread, because the other thread drifted away from its original topic. So here, in case someone is interested: On Oct 2, 2017 17:03, "Koos Zevenhoven wrote: On Mon, Oct 2, 2017 at 6:42 AM, Guido van Rossum wrote:

Re: [Python-Dev] Python startup optimization: script vs. service

On 2017-10-02 16:59, Barry Warsaw wrote: > On Oct 2, 2017, at 10:48, Christian Heimes wrote: >> >> That approach could work, but I think that it is the wrong approach. I'd >> rather keep Python optimized for long-running processes and introduce a >> new mode / option to

Re: [Python-Dev] Python startup optimization: script vs. service

On Mon, Oct 2, 2017 at 8:03 AM Victor Stinner wrote: > 2017-10-02 16:48 GMT+02:00 Christian Heimes : > > That approach could work, but I think that it is the wrong approach. I'd > > rather keep Python optimized for long-running processes and

Re: [Python-Dev] Python startup optimization: script vs. service

02.10.17 16:26, Victor Stinner пише: While "import module" is fast, maybe we should use sometimes a global variable to cache the import. module = None def func(): global module if module is None: import module ... I optimized "import module", and I think it can be optimized even

Re: [Python-Dev] Investigating time for `import requests`

On Oct 1, 2017, at 22:34, Nathaniel Smith wrote: > > In principle re.compile() itself could be made lazy -- return a > regular exception object that just holds the string, and then compiles > and caches it the first time it's used. Might be tricky to do in a > backwards

Re: [Python-Dev] Python startup optimization: script vs. service

Fair, but can you justify your preference? From my perspective, I write many small command line scripts, and all of them would benefit from faster load time. Am I going to have to stick mode-setting incantations at the top of every single one? I occasionally write simple servers, and none of

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 16:48 GMT+02:00 Christian Heimes : > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. "Filling caches on

Re: [Python-Dev] Python startup optimization: script vs. service

On 2017-10-02 15:26, Victor Stinner wrote: > 2017-10-02 13:10 GMT+02:00 INADA Naoki : >> https://github.com/python/cpython/pull/3796 >> In this PR, lazy loading only happens when uuid1 is used. >> But uuid1 is very uncommon for nowdays. > > Antoine Pitrou added a new C

Re: [Python-Dev] Python startup optimization: script vs. service

On Oct 2, 2017, at 10:48, Christian Heimes wrote: > > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. What would

Re: [Python-Dev] Python startup optimization: script vs. service

I’m new to this issue, but curious: could the long-running server mitigate lazy loading problems simply by explicitly importing the deferred modules, e.g. at the top of __main__.py? It would require some performance tracing or other analysis to figure out what needed to be imported, but this

Re: [Python-Dev] Intention to accept PEP 552 soon (deterministic pyc files)

Please start a new thread on python-dev. It's unrelated to "deterministic pyc files". Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

Re: [Python-Dev] Python startup optimization: script vs. service

On 2017-10-02 14:05, George King wrote: > I’m new to this issue, but curious: could the long-running server > mitigate lazy loading problems simply by explicitly importing the > deferred modules, e.g. at the top of __main__.py? It would require some > performance tracing or other analysis to

Re: [Python-Dev] PEP 553

On Sun, Oct 1, 2017 at 11:15 PM, Terry Reedy wrote: > On 10/2/2017 12:44 AM, Guido van Rossum wrote: > > - There's no rationale for the *args, **kwds part of the breakpoint() >> signature. (I vaguely recall someone on the mailing list asking for it but >> it seemed far-fetched

Re: [Python-Dev] Intention to accept PEP 552 soon (deterministic pyc files)

On Mon, Oct 2, 2017 at 6:42 AM, Guido van Rossum wrote: > On Sun, Oct 1, 2017 at 1:52 PM, Koos Zevenhoven wrote: > >> On Oct 1, 2017 19:26, "Guido van Rossum" wrote: >> >> Your PEP is currently incomplete. If you don't finish it, it is not

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 13:10 GMT+02:00 INADA Naoki : > https://github.com/python/cpython/pull/3796 > In this PR, lazy loading only happens when uuid1 is used. > But uuid1 is very uncommon for nowdays. Antoine Pitrou added a new C extension _uuid which is imported as soon as uuid(.py)

Re: [Python-Dev] Python startup optimization: script vs. service

Hi. My company is using Python for web service. So I understand what you're worrying. I'm against fine grained, massive lazy loading too. But I think we're careful enough for lazy importing. https://github.com/python/cpython/pull/3849 In this PR, I stop using textwrap entirely, instead of lazy

[Python-Dev] Python startup optimization: script vs. service

Hello python-dev, it's great to see that so many developers are working on speeding up Python's startup. The improvements are going to make Python more suitable for command line scripts. However I'm worried that some approaches are going to make other use cases slower and less efficient. I'm

Re: [Python-Dev] Investigating time for `import requests`

> On Oct 2, 2017, at 12:39 AM, Nick Coghlan wrote: > > "What requests uses" can identify a useful set of > avoidable imports. A Flask "Hello world" app could likely provide > another such sample, as could some example data analysis notebooks). Right. It is probably

Re: [Python-Dev] Investigating time for `import requests`

On 2017-10-02 04:04, INADA Naoki wrote: > *3. ssl* > > import time:      2007 |       2007 |                     ipaddress > import time:      2386 |       2386 |                     textwrap > import time:      2723 |       2723 |                     _ssl > ... > import time:       306 |       

Re: [Python-Dev] Investigating time for `import requests`

On 2 October 2017 at 06:13, Raymond Hettinger wrote: > >> On Oct 1, 2017, at 7:34 PM, Nathaniel Smith wrote: >> >> In principle re.compile() itself could be made lazy -- return a >> regular exception object that just holds the string, and then

Re: [Python-Dev] Investigating time for `import requests`

On 2 October 2017 at 15:13, Raymond Hettinger wrote: > >> On Oct 1, 2017, at 7:34 PM, Nathaniel Smith wrote: >> >> In principle re.compile() itself could be made lazy -- return a >> regular exception object that just holds the string, and then

Re: [Python-Dev] PEP 553

On 10/2/2017 12:44 AM, Guido van Rossum wrote: - There's no rationale for the *args, **kwds part of the breakpoint() signature. (I vaguely recall someone on the mailing list asking for it but it seemed far-fetched at best.) If IDLE's event-driven GUI debugger were rewritten to run in the