Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Guido van Rossum
Let's kick this part of the discussion back to python-ideas. On Tue, Oct 17, 2017 at 1:36 PM, Michel Desmoulin wrote: > Maybe it's time to bring back the debate on the "lazy" keyword then ? > Rendering any statement arbitrarily lazy could help with perfs. It would >

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Michel Desmoulin
Maybe it's time to bring back the debate on the "lazy" keyword then ? Rendering any statement arbitrarily lazy could help with perfs. It would also make hacks like ugettext_lazy in Django useless. And would render moot the extensions of f-strings for lazily rendered ones. And bring lazy imports in

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-17 Thread Neil Schemenauer
Christian Heimes wrote: > That approach could work, but I think that it is the wrong > approach. I'd rather keep Python optimized for long-running > processes and introduce a new mode / option to optimize for > short-running scripts. Another idea is to run a fake

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-03 Thread Nick Coghlan
On 3 October 2017 at 03:02, Christian Heimes wrote: > On 2017-10-02 16:59, Barry Warsaw wrote: >> On Oct 2, 2017, at 10:48, Christian Heimes wrote: >>> >>> That approach could work, but I think that it is the wrong approach. I'd >>> rather keep Python

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Brett Cannon
On Mon, 2 Oct 2017 at 11:19 Christian Heimes wrote: > On 2017-10-02 19:29, Brett Cannon wrote: > > My current design for an opt-in lazy importing setup includes an > > explicit function for importlib that's mainly targeted for the stdlib > > and it's startup module needs,

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 19:29, Brett Cannon wrote: > My current design for an opt-in lazy importing setup includes an > explicit function for importlib that's mainly targeted for the stdlib > and it's startup module needs, but could be used by others: >

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Brett Cannon
On Mon, 2 Oct 2017 at 08:00 Christian Heimes wrote: > On 2017-10-02 15:26, Victor Stinner wrote: > > 2017-10-02 13:10 GMT+02:00 INADA Naoki : > >> https://github.com/python/cpython/pull/3796 > >> In this PR, lazy loading only happens when uuid1 is

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 16:59, Barry Warsaw wrote: > On Oct 2, 2017, at 10:48, Christian Heimes wrote: >> >> That approach could work, but I think that it is the wrong approach. I'd >> rather keep Python optimized for long-running processes and introduce a >> new mode / option to

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Gregory P. Smith
On Mon, Oct 2, 2017 at 8:03 AM Victor Stinner wrote: > 2017-10-02 16:48 GMT+02:00 Christian Heimes : > > That approach could work, but I think that it is the wrong approach. I'd > > rather keep Python optimized for long-running processes and

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Serhiy Storchaka
02.10.17 16:26, Victor Stinner пише: While "import module" is fast, maybe we should use sometimes a global variable to cache the import. module = None def func(): global module if module is None: import module ... I optimized "import module", and I think it can be optimized even

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread George King
Fair, but can you justify your preference? From my perspective, I write many small command line scripts, and all of them would benefit from faster load time. Am I going to have to stick mode-setting incantations at the top of every single one? I occasionally write simple servers, and none of

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Victor Stinner
2017-10-02 16:48 GMT+02:00 Christian Heimes : > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. "Filling caches on

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 15:26, Victor Stinner wrote: > 2017-10-02 13:10 GMT+02:00 INADA Naoki : >> https://github.com/python/cpython/pull/3796 >> In this PR, lazy loading only happens when uuid1 is used. >> But uuid1 is very uncommon for nowdays. > > Antoine Pitrou added a new C

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Barry Warsaw
On Oct 2, 2017, at 10:48, Christian Heimes wrote: > > That approach could work, but I think that it is the wrong approach. I'd > rather keep Python optimized for long-running processes and introduce a > new mode / option to optimize for short-running scripts. What would

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread George King
I’m new to this issue, but curious: could the long-running server mitigate lazy loading problems simply by explicitly importing the deferred modules, e.g. at the top of __main__.py? It would require some performance tracing or other analysis to figure out what needed to be imported, but this

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
On 2017-10-02 14:05, George King wrote: > I’m new to this issue, but curious: could the long-running server > mitigate lazy loading problems simply by explicitly importing the > deferred modules, e.g. at the top of __main__.py? It would require some > performance tracing or other analysis to

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Victor Stinner
2017-10-02 13:10 GMT+02:00 INADA Naoki : > https://github.com/python/cpython/pull/3796 > In this PR, lazy loading only happens when uuid1 is used. > But uuid1 is very uncommon for nowdays. Antoine Pitrou added a new C extension _uuid which is imported as soon as uuid(.py)

Re: [Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread INADA Naoki
Hi. My company is using Python for web service. So I understand what you're worrying. I'm against fine grained, massive lazy loading too. But I think we're careful enough for lazy importing. https://github.com/python/cpython/pull/3849 In this PR, I stop using textwrap entirely, instead of lazy

[Python-Dev] Python startup optimization: script vs. service

2017-10-02 Thread Christian Heimes
Hello python-dev, it's great to see that so many developers are working on speeding up Python's startup. The improvements are going to make Python more suitable for command line scripts. However I'm worried that some approaches are going to make other use cases slower and less efficient. I'm