[Python-ideas] Re: Top Level Await in Python like in Deno
Yes I can, but I am taking about to use it without `asyncio.run` Whenever Python in Top-Level faces with await it will wrap calling all top level statement in async function (for example) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OV5EU3LLWRK5IDZS7V6YZUJI2HYM36FU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Top Level Await in Python like in Deno
It is not the same, it will work in interactive mode But I want to run application without interactive mode ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/O4ME7K7ZVXULFBLAVTP3CTKLXI6O4R65/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Top Level Await in Python like in Deno
I mean to be able to do something like this: ```python import asyncio await asyncio.sleep(1); ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ED75ZXM5WSLZMIW6SXGLWBUIFJRINNQ2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Top Level Await in Python like in Deno
Hi all, Deno is JavaScript runtime that has very nice feature like Top Level Await, I think it would be also nice to have such feature in Python, it will make using async/await more convenient What do you think ? Share your ideas lets discuss ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/N2GA3CIY75TZNQ5RGKDMH7DDZNWLJNSH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python JIT Compilation Thoughts
For me it easier to do with C++, I know C, but with C++ it is more maintainable and easier to add new feature and also easier to refactor ... I am not sure if CPython maintainers team will appropriate that I add C++ in code base ... But I can try, I will try to find time to create small proof of concept in C++ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/B6UM6566HMUP4O3WKXCP3GMT4RDPHCJJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python JIT Compilation Thoughts
I have found very nice attempt of JIT for CPython https://github.com/tonybaloney/Pyjion It is working under .NET Runtime, but if it is possible in this project I believe it is possible in CPython as well Also to integrate JIT compilcation now it not so hard because we can use llvm ability for JIT Also I will be less hard if CPython started switching it code-base to modern C++ at least C++11 that will make more maintainable code than raw C ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JKJH7WFW73AVXR4KR7O5M6E7OX7SEWEF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
Good design should follow open-close principle from SOLID ... You provide template (standard way to do something) for community and if package can work through this interface that is required all goes good ... In such way community would have the standard way to do something instead of lots of different way that even not compatible with each other Also this design is exactly according Zen Python: "There should be one-- and preferably only one --obvious way to do it." ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EMLYJWF4UGQK6VHTYTEIXILOZ2QIJDST/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
Yes, it is not bad, it is open-close principle from SOLID for good design system ... You provide template (standard way to do something) for community and if package can work through this interface that is required all goes good ... In such way community would have the standard way to do something instead of lots of different way that even not compatible with each other Also this design is exactly according Zen Python: "There should be one-- and preferably only one --obvious way to do it." ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/U2DPBD7PVZKINBDGE6MTGVBENLX6URAW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
It is not to get something in stdlib, it is simple option in cli )) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5STITZ4AALYTB4AHJSLL54BK7BSJY2CD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
It is also about convenience ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QFCFWE7RRIYIO4YZNHU5PRZLRYR6RNB3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
No, it is not good design choose, because then all type-checker will have different optional and it will be a mess ... It should be done in one place like python ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ULAV7ZXI3NRQ3LTK6MONRC5FJLI7NTIG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
I do not want to make type-checker a part of CPython, I just what options that will allow to run python with first some type-checker ... For example, you would be able to set default python module for type-checking and if there is not module set for type checking, verify if `mypy` installed and if installed run it first ... The issue with separate running type-checker first and then script is that it is separate commands and failing run script is console dependent: in bash `set -e` and etc. in other consoles Better to have terminal agnostic option that will allow to do the same thing ... Also I think that maybe it would be also good to have `*.ty` (Typed pYthon) that would mean if python understand extension `ty` then it will run default type checker if it is available and then script ... Anyway as you can see there is obvious reason to add CPython flag and it is not enforce to CPython somehow to integrate type checker in its source code ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/6YGHZ462MITYIC7HDI2VPEQ4KDYLVZO5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
Steven D'Aprano wrote: > On Sun, Dec 06, 2020 at 10:22:53AM -, redrad...@gmail.com wrote: > > It would be nice to have "Typed Python" mode that > > will look like this: > > #!/usr/bin/env bash > > > > set -e > > python -m mypy $1 > > python $1 > > > > What does that mean? Why is it a bash script? It is only example how I use what I call "Typed Python" The same can happens under-hood, python in this "typed" mode can first run type-checker and then if there are no errors run script itself > > It could be achieved by adding special flag like > > -t (typed) > > What does "typed" mode do? > Why would it be "nice"? If all you want is to run mypy, you can run > mypy. Or any alternative type-checker, such as Pytype, Pyright, Pyre or > Jedi: > https://google.github.io/pytype/ > https://github.com/Microsoft/pyright > https://pyre-check.org/ > https://pypi.org/project/jedi > (Jedi does a lot more than just type checking.) It would be good to have direct support from python to run script in typed mode or not ... Python can support other type checker with some option `--type-checker= or ` of module that will type-check ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7MXWASAAEBCIQ2TIB6HA6ZDAZCIG6X3F/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Typed Python execution mode
Also if such mode would exsist, it would be nice to have special keywords for such mode like `protocol` Instead of writing: ```python class MathType(Protocol): def reduce(self, *args) -> int: ... ``` it woulb be nicer to have special syntax: ```python-mypy protocol MathType: def reduce(self, *args) -> int: ... def sum(self, *args) -> int: ... ``` It will make code more readable as for me ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NYVCXKF7Y62F443OXIYMEPF2GZS3NOG6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Typed Python execution mode
It would be nice to have "Typed Python" mode that will look like this: ```bash #!/usr/bin/env bash set -e python -m mypy $1 python $1 ``` https://gist.github.com/redradist/dd7253a55081a4dc13fdf3f1549f43b5 It could be achieved by adding special flag like `-t` (typed) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2F2HAI63XHMSIZYJ7CKKRQDVVQQZVRZ4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Experimental syntax proposal
I like this literal syntax !! Can't wait to use it !! ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/22U4Z7ELTZ5X5S5LVXH3VOXICOCPJCHZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Custom keywords (from: Decorators for class non function properties)
Actually, I would like to write a proposal ... It would be nice if someone support me with it because I will do it at first time ... ) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HBLPKZUXSCSJUN4O3LK3JCGQAHFTJ35M/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
No it is not possible to have something like this: ```python def function(cls): # Where is cls is Neuron class object pass class Neuron: activation = function(Neuron) ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/X4GVYC6GDNJHN4HUORJ5WUNXHZZLC2PW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
Also instead of making all class as dataclass, it would be possible to make
only some properties as instance properties:
```python
class Client:
bank = Bank()
@instance
name = Name()
print(f'Client.bank is {Client.bank}')
client = Client()
print(f'client.name is {client.name}')
```
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/LG6AQY2EFZ5MJRAPS7HKEW7LYKFAAEQH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
Actually in example: ```python class MyClass: @my_property name = arg class MyClass: def name(self): ... def see_name(self): ... ``` I have done mistake ... of course it will not be like this ... What I wanted to show that @my_property could add more complex behavior than just small wrapper in one line For example: ```python # Module a.py class SomeFactory: def __init__(self, cls): self.cls = cls def build_context(self): return context def factory(name, value, self, cls): return SomeFactory(cls) # Module b.py class MyClass: @factory name = arg ``` And so one ... also could be added optionally logging to track each time when somebody read property: ```python # Module b.py class MyClass: @logging # Help to track each time when somebody access property @factory name = arg ``` I see lots of use-cases for property decorators ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FY57MAJYDSKZ4DQVRAKG2K6XA7BQ6PC4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
I think a property decorator can be useful, because you consider the simplest case with: ```python class MyClass: @my_property name = arg ``` but consider it can generate the following code: ```python class MyClass: def name(self): ... def see_name(self): ... ``` Or consider the following example: ```python class MyClass: @factory strategy= arg ``` that can will generate: ```python class MyClass: class MyClassStrategy: ... strategy= MyClassStrategy(arg) ``` All it will be possible if attribute decorator will have the following signature: ```python def factory(name, value, self, cls): # Where name - name of the property #value - initial value of the property #self - instance of created object before call __init__ #cls - class object (MyClass) ... ``` of course some of properties could be omitted like this: ```python def factory(name, value, cls): # Where name - name of the property #value - initial value of the property #cls - class object (MyClass) ... ``` As you can see property decorators could be very powerful thing if it will done right ;) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MCQBK7GRYIUTA5AQAOVMEQNZANXTIDIY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
But I can do the same thing with class methods ... but anyway it was introduced method decorators to simplify development and add extra power ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/W54PL2MHR45B7UXC726VZUA22GGH263W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
It could work if we extend syntax like this: ```python class Neuron: activation # By default creates activation with None value activation = linear_activation(activation) ``` and then we could apply as may decorators as needed: ```python class Neuron: activation # By default creates activation with None value activation = linear_activation(activation) activation = softmax_activation(weights=["w0", "w1"])(linear_activation(activation)) ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/I7LHY3X3MV4U36VHKHK3B3L2QNHWG42T/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
Also there is maybe some addition parameter like self: ```python class Neuron: @instance_property activation def __init__(self): # automatically created pass ... def instance_property(name, property, self, *args): # Create property on instance ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LAQOOARQKIBAO2EVH6SC36S6X3JWNNKW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
Decorator will do the same thing as general decorator For example it could be implemented like this: ```python class linear_activation: def __init(self, name, property): ... def linear_activation(name, property): ... ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WXXNZHRVYRBRZBGFWLLIM7IQ3FSQORYF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Decorators for class non function properties
Disagree, because for example what if I want custom property with two or three decorators ? Like this: ```python class Neuron: @softmax_activation(weights=["w0", "w1"]) @linear_activation activation def __init__(self): self.w0 = [...] self.w1 = [...] ... ``` or for example I just want to apply to my initializer some decorator: ```python class Neuron: @softmax_activation(weights=["w0", "w1"]) activation = LinearActivation(...) def __init__(self): self.w0 = [...] self.w1 = [...] ... ``` There exist use-cases for this feature ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RQPQSHPTRYVZHPYAR4WR4HEFTQE7LUL7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Decorators for class non function properties
Hi all, Seems like this topic was previously raised, but what if we add possibility to decorate non function properties in class: ```python class Neuron: @linear_activation activation ``` ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/P6NRLZCZX3KKODT74CYH4A6ABXYMDUB7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thoughts about Fast Python Engine and Python EveryWhere
I know, I know that PyPy is fast as V8 but PyPy implement the whole library inside and it is not easy to embed it somewhere ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BPUJ4YW2B4OTBKO7LFBGKC4YBIM3D65V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thoughts about Fast Python Engine and Python EveryWhere
My main point is that it would be nice to have just very fast execute engine and all library as integration layer ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ISY56W7ZXSIOI4SOFOUNNRLLE7RM57YQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Thoughts about Fast Python Engine and Python EveryWhere
CPython is portable but due to integrated standard library (builtin functionality) it is hard to evolve it, for examle to add JIT, anyway it is just my thoughts ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TX3XGC2THCJ3YL3X5LTL4YEDYZMH3JBH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Thoughts about Fast Python Engine and Python EveryWhere
Recently I have been thinking about why `JavaScript` with it's horrible type system and lots of edge cases has supported so many platform and is very fast ... First answer is simple, because big companies such as Google, Facebook and so on evolve this language and run-time for it ... But it is only part of story, all this platforms would not be possible without some good architecture that was done by this companies What is this architecture decision ? It is a splitting run-time and infrastructure V8 is JavaScript engine that could very fast execute JavaScript and that is all !! It has a reach API using which you can embed this run-time almost every where ... What if we in Python will accomplish the same solution ? What if Python would have some very fast engine that could be integrate in every environment ? Engine will be able only execute Python code and call some registered callback (hooks) for `import` some module, for calling some registered C-extension and so on If this engine will be very portable then Python would be embedded everywhere !! ;) Engine will not suport all builtin functions and it will not have any library embedded in it To add some buiiltin functionality you will need register the callbacks in engine, the same with libraries Engine will just call callback to import code and callback optionally will return Python script which will be executed To be continue ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/D7HOOBORH2FJOJO4PWBIRHNOOVIIFBXG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] [Feature] Body closure
Today I think about lambda in Python and what if we introduce the new syntax: ```python def lock(*args, closure): # Do some stuff closure() # Call closure # Finish stuff if __name__ == '__main__': lock(): # Do some things here is thread safe ``` This feature could be very similar as in `Kotlin` inline functions https://kotlinlang.org/docs/reference/inline-functions.html ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HAIFGFE5YUCGLEWFPT3K2N3H6WBK4KQ7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
Brett Cannon wrote: > It's a discussion issue. PEP 554 is trying to focus on the API of > subinterpreters and doesn't want to distract from that by bringing the GIL > into it. > That being said, the general expectation from everyone involved is there > will be a perl-interpreter GIL. > On Sat, Jun 27, 2020 at 10:31 AM Denis Kotov redrad...@gmail.com wrote: > > Reading PEP554 > > https://www.python.org/dev/peps/pep-0554/#a-disclaimer-about-the-gil > > seems like at the current implementation of subinterpretters there will be > > no separate GIL … > > But I am wondering, why ? > > Each subinterpreter has it own object management and Garbage Collection … > > Why subinterpreters should share one GIL ? > > Is there any technical issues ? > > Sent from Mail https://go.microsoft.com/fwlink/?LinkId=550986 > > for > > Windows 10 > > > > Python-ideas mailing list -- python-ideas@python.org > > To unsubscribe send an email to python-ideas-le...@python.org > > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > > Message archived at > > https://mail.python.org/archives/list/python-ideas@python.org/message/ZC3WVJ... > > Code of Conduct: http://python.org/psf/codeofconduct/ > > Okay, where is discussions of PEP 554, because I wanted to propose to add `async` API for sub-interpreters in such way using it not required to create new thread, just use event_loop ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/K6PUB3L2VT62HNOEM3HWXFROEJNDXJFO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Steven D'Aprano wrote: > On Mon, Jun 29, 2020 at 10:20:40AM -, redrad...@gmail.com wrote: > > Why I want that ? > > Okay, here are the reasons: > > 1) Security issue, should be fixed as soon as possible without waiting > > 2 months or 1 year for next CPython release > > That is an excellent argument, but is that our responsibility? > There are many third party distributors that bundle Python and can > provide a much faster bug fix schedule, e.g. Anaconda, Red Hat, other > Linux distributions. (Apologies if I have missed anyone.) > Some of them have more resources in time, money and available manpower > than we have. > If you want security fixes faster than the Python-Devs are capable of > releasing them, perhaps you ought to pay a third-party? > > 2) Modules could be evolved independently that will > > allow to use some > > features of package earlier ... (crucial in our "Fast" World) > > You say "crucial", I say "terrible". Our "fast world" is not something > we should be encouraging. It is bad for people and bad for technology. > Libraries in the std lib should not be evolving fast. Stability is more > important than rapid development, and if a library is so experimental > that it needs rapid development, then it is too experimental to be in > the std lib. > Third-party libraries can evolve as fast or as slow as they want; the > Python std lib is under tension between people who want faster evolution > and people who want stability, and we have to balance those two desires. > As a compromise between "change once a month" and "change once a > decade", I think Python's release cycle is not too bad. > > 3) If library modularized I can remove parts of it on > > constrained > > environments (microcontroller) or in environments where we try to same > > each byte of disk memory Interfaces between modules would be thinner > > and visible that will allow to download as many packages as need for > > this module or library > > You can already do that. There are at least two currently maintained > Pythons for small systems, MicroPython and CircuitPython. There may be > others. > The question is not whether Python's standard library can be split up, > but whether we should force it to be split up for everyone, making > everyone's life more complicated in order to simplify the needs of a > minority of developers. > I have written a lot of code that has to run on older versions or > installations without third-party libraries, so I have lots of > feature-detection code: > try: > min([1], key=lambda x: None) > except TypeError: > # Current system is too old to support key functions. > # Create our own basic version. > ... > > At the beginning, it's lots of fun to come up with clever ways to detect > features which might be missing, and then find a work around. But it > gets tiresome and painful very quickly. > It is much better to work with a known environment: if I am running in > Python 3.9, then all these libraries and features come in a bundle. If > everything could change independently, then we would need feature- > detection and version checks everywhere. That is not enjoyable, and it > increases the complexity for everyone even when they get no benefit. Seems like you did not get my point ... ( I do not ask to remove or replace Python standard library, but to split it on two versions (aka standard, like .NET Standard, Rust libcore and libstd) Modularization would be also nice, but it requires time ... Claim that all Python should at least support libcore (Core Library) make whole infrastructure more uniform libcore could be all packages written in python and that do not access os resource and essential for ecosystem libstd - all other standard module libcore and libstd it is small step to modularization and standartization whole Python ecosystem ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AFTLO4LWL2RNQZCQEOUILZDSNUBZB3TL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python __main__ function
Chris Angelico wrote:
> On Fri, May 29, 2020 at 5:25 AM Alex Hall alex.moj...@gmail.com wrote:
> >
> > On Thu, May 28, 2020 at 12:57 PM Paul Sokolovsky pmis...@gmail.com wrote:
> > >
> > And in all fairness, all good ideas already came
> > to somebody else years
> > ago. There's https://www.python.org/dev/peps/pep-0299/
> > , successfully
> > rejected yet back in 2002. (So, feel free to use it in your own
> > environment/Python dialect.)
> > Thanks for the link. Pity it was rejected. I can think of one reason I'd
> > quite
> > like this beyond it being DRY and concise: I often see code written by
> > others like
> > this:
> > def foo(bar):
> > print(bar)
> >
> > if __name__ == '__main__':
> > bar = 3
> > foo(bar)
> >
> > Now bar is both a local and global variable, which leads to both annoying
> > IDE warnings and actual bugs. I think I encountered a bug related to this
> > when someone
> > used eval() (which was the right thing to do in this case) and didn't
> > specify
> > the namespaces correctly, but name shadowing made it seem like their code
> > was
> > correct.
> > Point is, I'd like something that encourages people to put all their
> > __main__ logic and variables into a function, rather than a module level
> > conditional.
> > People can already put all their main logic into a function. If you
> want to unit-test your main function, that's the best way to do it.
> The trouble is, how much goes into main() and how much into if
> __name__ == '__main__'? For example: Does your main function accept a
> list of arguments, or does it look in sys.argv? Neither answer is
> wrong. And that means the best way is to NOT force everyone across all
> of Python to choose the same way - let people write their code their
> way.
> ChrisA
It is not enforcing user it just to provide more choice:
```python
if __name__ == '__main__':
... main logic ...
```
or this way:
```python
def __main__();
sys.argv[0]
... main logic ...
```
`__main__` will be called only once at the end of script if it was called as
script
Also on additional benefit, it will help people that came from other languages:
```rust
fn main() {
...
}
```
or
```c
void main() {
...
}
```
or
```java
class MyClass {
public void main(...) {
...
}
}
```
and in Python it could look like this:
```python
def __main__();
... main logic ...
```
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/DFDNB74ZQZPZ5MJDUHAJ6C23FA7VQUOR/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Steven D'Aprano wrote: > On Tue, Jun 16, 2020 at 12:41:58PM -, redrad...@gmail.com wrote: > > As long as I cannot update version of standard > > library package > > separately from CPython version - No, they are not separate creatures > > ;) > > Why would you want to? That just sounds like adding extra complexity and > pain for no benefit. > Instead of requirements: > > requires Python 3.5 or better > > you have requirements: > > Python 3.5 or better > math 2.7 or better > sys 2.1 or better > glob 5.9 or better > etc > > This does not seem like an improvement to me. I like going to > StackOverflow, and if I read a solution or recipe that says "tested with > Python 3.8" I know it will run in 3.8, without having to guess what the > minimum requirements for each module are. > Some of the Linux distros already split the stdlib into pieces. This is > a real pain, especially for beginners. The process changes from: > $ dnf install python3 > # or apt-get or whatever package manager you use > > and everything documented at python.org Just Works straight out of the > box, to a much more annoying process: > $ dnf install python3 > > and then you have mysterious ImportErrors because some modules aren't > installed, and you have to try to work out how to install them, and > that's not an easy task: > $ dnf search python3 | wc -l > 3511 Why I want that ? Okay, here are the reasons: 1) Security issue, should be fixed as soon as possible without waiting 2 months or 1 year for next CPython release 2) Modules could be evolved independently that will allow to use some features of package earlier ... (crucial in our "Fast" World) 3) If library modularized I can remove parts of it on constrained environments (microcontroller) or in environments where we try to same each byte of disk memory Interfaces between modules would be thinner and visible that will allow to download as many packages as need for this module or library Modularized library will have two versions (run-time and itself version) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5ONQNXZHOHRVKMZBQWXYUPRVDDUCZ7JY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Edwin Zimmerman wrote: > This is still true. There are some of us that will scream very very loud if > the std > lib disappears from Python installers. However, I think there could easily > be a way to > satisfy both parties here. How difficult would it be to release both full > and minimal > installers for every release? That way everyone could have it their way. > --Edwin > On 6/16/2020 6:51 AM, Stéfane Fermigier wrote: > > It has also been discussed that some people can't > > rely on anything outside the standard packages, due to the constraints of > > their company or > > their field. > > This attempt will at least make some standardization of libraries for different Python implementations: CPython, PyPy, Brython, RustPython, MicroPython, Pycopy For example if user writes application in RustPython for WASM which will be executed in browser it could use only libcore api for base logic packages and some specific platform api within implementation of Python This attempt was done previously by .NET with .NET Standard that allows to write libraries that could be run on any implementation of .NET (.NET Framework or Mono) Since in Python we have a lot of different implementations it would be nice to know that if I use the libcore functionality it will run everywhere and if I use libstd it will run only where requirements of the system allow to use full package or where author of implementation implemented those packages ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/4EKU3MZRWOG5JKCLEATUKTMUGD2TLMTC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Paul Moore wrote: > On Tue, 16 Jun 2020 at 11:53, Stéfane Fermigier s...@fermigier.com wrote: > > the "batteries included" argument was a huge selling > > points years ago (when Aaron Watters wrote "Internet Programming With > > Python", for > > instance) but I think the situation has changed quite a bit since then. > > For some people/situations, yes. For others, definitely not. > "Batteries included" is still an extremely significant selling point > for some users of Python. > Anyone contributing to this discussion should be very careful not to > assume that everyone using Python works in a similar environment to > them. For example, many people work in environments where Internet > access is tightly controlled one way or another. Not all countries or > regions have reliable or widespread internet access. And some people > simply need scripts to be easy to run, so "install Python and run this > script" is as complex as it can get. > Paul Modularization that I suggest is just to separate modules and clean interfaces between them in such way it would be possible to use standard provided module when you download CPython or update this module from PyPi ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SEEMA22QWXN7RD2QXDACMFRIC22R7FNS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Stephen J. Turnbull wrote: > Edwin Zimmerman writes: > > This is still true. There are some of us that will > > scream very > > very loud if the std lib disappears from Python installers. > > However, I think there could easily be a way to satisfy both > > parties here. How difficult would it be to release both full and > > minimal installers for every release? That way everyone could have > > it their way. > > Not too hard, but that's a distro problem. Python is a development > organization. It's traditionally been considered that producing a > series of "official" installers for Windows and Mac is useful, but it > will be a heavy lift to go further. All .NET Core library modularized and this mean I could update any package without updating underling run-time and it is cool It will allow develop any Python package independently and receive update sooner (for example security issue should be fixed as soon as possible in our "Fast" World ;) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZH4TGCHL4VN7RXG6QC5EIHDSBMLYCWXS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python JIT Compilation Thoughts
Ned Batchelder wrote: > On 5/25/20 6:02 AM, redrad...@gmail.com wrote: > > Hi all, > > I do not know maybe it was already discussed ... > > It's been extensively discussed and attempted. > > but the toolchain like LLVM is very mature and it can > > provide the simpler JIT compilation to machine code functionality and it > > will improve > > performance of the Python a lot !! > > Be careful with "will": nothing is simple in this arena. > --Ned. And what was the result of discussions ? There are also Cranelift toolchain that is written in Rust and it provides as for very good API ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AJJWKG2OWZNMWHD7GUL3RXB25QBCBFPJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Bringing the print statement back
Guido van Rossum wrote:
> In Python 3.10 we will no longer be burdened by the old parser (though 3rd
> party tooling needs to catch up).
> One thing that the PEG parser makes possible in about 20 lines of code is
> something not entirely different from the old print statement. I have a
> prototype:
> Python 3.10.0a0 (heads/print-statement-dirty:5ed19fcc1a, Jun 9 2020,
> 16:31:17)
> [Clang 11.0.0 (clang-1100.0.33.8)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> Cannot read termcap database;
> using dumb terminal settings.
> > print 2+2
> > 4
> > print "hello world"
> > hello world
> > print "hello", input("Name:")
> > Name:Guido
> > hello Guido
> > print 1, 2, 3, sep=", "
> > 1, 2, 3
> > But wait, there's more! The same syntax will make it possible to call
> any
> function:
> > len "abc"
> > 3
> > Or any method:
> > import sys
> > sys.getrefcount "abc"
> > 24
> > Really, any method:
> > class C:
> > ... def foo(self, arg): print arg
> > ...
> > C().foo 2+2
> > 4
> > There are downsides too, though. For example, you can't call a
> method
> without arguments:
> > print
> > Worse, the first argument cannot start
> with a parenthesis or bracket:
> > print (1, 2, 3)
> > 1 2 3
> > C().foo (1, 2, 3)
> > Traceback (most recent call last):
> > File "", line 1, in
> > TypeError: C.foo() takes 2 positional arguments but 4 were given
> > print (2+2), 42
> > 4
> > (None, 42)
> > C().foo [0]
> > Traceback (most recent call last):
> > File "", line 1, in
> > TypeError: 'method' object is not subscriptable
> > No, it's not April 1st. I am seriously proposing this (but I'll
> withdraw it
> if the response is a resounding "boo, hiss"). After all, we currently have
> a bunch of complexity in the parser just to give a helpful error message to
> people used to Python 2's print statement:
> > print 1, 2, 3
> > File "", line 1
> > print 1, 2, 3
> > ^
> > SyntaxError: Missing parentheses in call to 'print'. Did you mean print(1,
> > 2, 3)?
> > And IIRC there have been a number of aborted attempts at syntactic
> hacks to
> allow people to call functions (like print) without parentheses, although
> (I think) none of them made it into a PEP. The PEG parser makes this much
> simpler, because it can simply backtrack -- by placing the grammar rule for
> this syntax (tentatively called "call statement") last in the list of
> alternatives for "small statement" we ensure that everything that's a valid
> expression statement (including print() calls) is still an expression
> statement with exactly the same meaning, while still allowing
> parameter-less function calls, without lexical hacks. (There is no code in
> my prototype that checks for a space after 'print' -- it just checks that
> there's a name, number or string following a name, which is never legal
> syntax.)
> One possible extension I didn't pursue (yet -- dare me!) is to allow
> parameter-less calls inside other expressions. For example, my prototype
> does not support things like this:
> > a = (len "abc")
> > File "", line 1
> > a = (len "abc")
> > ^
> > SyntaxError: invalid syntax
> > I think that strikes a reasonable balance between usability and
> reduced
> detection of common errors.
> I could also dial it back a bit, e.g. maybe it's too much to allow 'C().foo
> x' and we should only allow dotted names (sufficient to access functions in
> imported modules and method calls on variables). Or maybe we should only
> allow simple names (allowing 'len x' but disallowing 'sys.getrefcount x'.
> Or maybe we should really only bring back print statements.
> I believe there are some other languages that support a similar grammar
> (Ruby? R? Raku?) but I haven't investigated.
> Thoughts?
One additional drawback ... if anytime sooner or later you would wanted to
introduce scoped variables like in `Rust` with keyword `var` or `let` it could
make some problems ...
```python
let x = 3 # Parser will think that it is the function let
```
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/YBJZDOW27R6V635BKXUWE2YWN2WHW6B2/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Rhodri James wrote: > On 16/06/2020 10:23, redrad...@gmail.com wrote: > > I think it would be desired to modularize Python > > library and then to > > provide part of standard library through PyPi It will add possibility > > to evolve separately run-time and standard library > > Uh, aren't the runtime and standard library already separate creatures? As long as I cannot update separately `CPython` standard library and run-time itself - No they are not separate creatures ;) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FMDOWIXWLRPYHM3S3E4LMURE54BGA4TI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
Rhodri James wrote: > On 16/06/2020 10:23, redrad...@gmail.com wrote: > > I think it would be desired to modularize Python > > library and then to > > provide part of standard library through PyPi It will add possibility > > to evolve separately run-time and standard library > > Uh, aren't the runtime and standard library already separate creatures? As long as I cannot update version of standard library package separately from CPython version - No, they are not separate creatures ;) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QJM5KFSC4W64GWXLMG3UT5IJAQDKI3ZQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Modularize Python library
I like what they `Pycopy` maintainers did !! I allow to evolve separately runt-time and standard library ;) Library maintainer could provide fixes to library faster than waiting one year release cycle of CPython ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AXB5K4GIXHAUTSDDOSYCQW3CZUMRR276/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Modularize Python library
Hi all, me again ... ) I think it would be desired to modularize Python library and then to provide part of standard library through PyPi It will add possibility to evolve separately run-time and standard library I think standard library should be as small as needed and all other functionality should be provided through PyPi It will be similar as `Rust` did ... they have pretty small `standard library` and `core library` If Python would have small `standard library` and `core library` it would be add standardization for basic functional that should have some Python implementation ... For example: 1) MicroPython (`libpythoncore` - should be provided entirely, `libpythonstd` - could be provided partially) 2) Brython (`libpythoncore` - should be provided entirely, `libpythonstd` - could be provided partially) 3) RustPython (`libpythoncore` - should be provided entirely, `libpythonstd` - should be provided entirely), if `libpythonstd` would be small enough than it would be possible to implement it 4) PyPy (`libpythoncore` - should be provided entirely, `libpythonstd` - should be provided entirely) All other modules and packages could be provided as separate packages (json, xml and etc.) Also it is just guy's thoughts that loves programming and try to bring best things from other ecosystem because they proves by time ;) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CFUPD7K6A3WFTJVHHVJFIKEMDFOIXJ7Z/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: String module name
Yeah, but it still desire to be solved ... One of the use-cases is if I want to use `Brython`, it will be usable to load the module or entire package from the internet and then to use it in my application ... Issue with the right to access some resources could be solve with flags as `Deno` did ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RXITTI3MWQU6L5QKFGUSL7K2MXI4IYVM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: String module name
You cannot trust PyPi either ... I think user should decide if it allows code from arbitrary URL to access filesystem, network or anything else as `wasmtime` and `deno` did ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MYFCWPCDBUY27X2F53L5T5ER6YWGLGPC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] String module name
What if we introduce the string module: ```python from "https://python.org/some_module.py; import name ... ``` It will add possibility to run code with complex name of module that cannot be presented as set of lexical items and also will allow to load the module from external location ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MLP5N3QIP2II6QFHTGOQYMJ5LGDTJBIA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python WebAssembly Support
The question is why not to apply all this patches to CPython to be able to compile CPython on the Web ? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TBIQNVL5LZ6RC3IXUGYQIQTPWE3XTUJ5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python WebAssembly Support
Thanks, but I know about this implementation ... But my question is why not to apply all this patches to `CPython` to be able to compile `CPython` on the Web ? https://github.com/iodide-project/pyodide https://github.com/dgym/cpython-emscripten Patches to CPython seems pretty straightforward ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GIQ7RREZAAHO2S6XWVEPN6SDIVXP7GMB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Python WebAssembly Support
Hi all, I love Python, but as soon as I need to do something in browser I have to use ugly JavaScript !! Is there any future plans support for compiling CPython to WebAssembly using Emscripten ? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XV7U4OHKLRJ5TC3LZUBKAHVKSWTDZOTV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
I've just share information, relax ;) And also it seems like they have very low overhead with atomic variables Arc ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NCVZDAZKKHQA5NB66AJEGXP2ZGO6BSHF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
Yesterday RustPython team finished threading without GIL: https://github.com/RustPython/RustPython/issues/1831 ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2X3P7Z2LKOXZ6NBM6AASOJY4N3TJLBQK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python __main__ function
Cool !! But it disappointed that this proposal was reject ( ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2WF26WMQIGSTYEPPW7R7P7BVB6XRHBR7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Python __main__ function
Hi all, In Python we often use the following syntax to call the main logic of script when it was ran: ```python def main(): pass # whatever should be done for `python ./script.py` if __name__ == '__main__': main() ``` Maybe it is a time to introduce the new module level function like __main__ ? Consider the following code: ```python def __main__(): pass # whatever should be done for `python ./script.py` ``` It is much easy an less code to write ;) Under-hood `Python` just will generate the following code: ```python def __main__(): pass # whatever should be done for `python ./script.py` # Below generated code if __name__ == '__main__': __main__() ``` If there are two `if __name__ == '__main__':` it is also not an issue: ```python def __main__(): pass # whatever should be done for `python ./script.py` def main(): pass # whatever should be done for `python ./script.py` if __name__ == '__main__': main() # Below generated code if __name__ == '__main__': __main__() ``` Or we could require user to have only one `if __name__ == '__main__':` ... What do you think, please share your opinion ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FBT5BT7KYFVQCZYVAY6HSSWNKAVCXA5T/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python JIT Compilation Thoughts
Thanks Guido !! I have not known about `pyston` ... It is pretty good like for me I will take a close look at this project ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PLC4LASBVE7TBQLDGAKY2HANSB5E4JYR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
Edwin Zimmerman wrote: > On Monday, May 25, 2020 redrad...@gmail.com [mailto:redrad...@gmail.com] > wrote > > Edwin Zimmerman wrote: > > On 5/25/2020 5:56 AM, redrad...@gmail.com > > wrote: > > Edwin Zimmerman wrote: > > Sub-interpreters are a work in progress. The API is not anywhere near > > being finalized. > > True parallel execution will occur only if the GIL is changed to a per > > interpreter lock, > > and that requires some rather large changes to Python that haven't happened > > yet. > > Why ? True parallel execution is possible with sub-interpreters !! > > Sub-interpretter should be run in separate thread not controlled by GIL > > and sub-interpretter will notify main interpreter with > > atomic variable that it is finished (like worker) !! > > We just need to provide two APIs: > > 1) Synchronized: run, run_string, that will wait until thread > > notify with setting atomic variable in true that it finished > > 2) Asynchronized (based on async): run_async, run_string_async, > > event_loop will wait on atomic variable reading it periodically > > All sub-interpreters and all threads are subject to the GIL, that is why it > > is > called the GLOBAL Interpreter Lock. There is no such thing as a "separate > thread not > controlled by GIL." This is not a problem with the sub-interpreter API. It > is a design > decision embedded very deeply through the entire CPython code. It is not > trivial to > change this. However, there is work being done on this. Search the > python-dev mailing > list for the "PoC: Subinterpreters 4x faster than sequential execution or > threads on > CPU-bound workaround" thread if you want to see more. > --Edwin Maybe I did not know something ... but the reason that sub-interpreter was added, because of separate GIL in each sub-interpreter ? Am I right ? We should not bind all interpreters with main interpreter ... only with atomic or condition variable ... otherwise sub-interpreters is not needed at all !! ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TCLX5B2GQNMF277M2AT2NUA2CCFGYUTE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
redradist＠gmail.com wrote:
> Chris Angelico wrote:
> > On Mon, May 25, 2020 at 7:58 PM redrad...@gmail.com
> > wrote:
> > >
> > Edwin Zimmerman wrote:
> > Only if your workload is CPU bound. Python optimizes IO bound workload
> > performance by
> > releasing the GIL while doing IO. Green threads generally do not offer
> > this option.
> > Real threads is not needed in Python:
> > 1) Real threads do not work parallel
> > They do when they do any sort of blocking operation.
> > 2) Real threads only consume resources of OS
> > 3) Real threads have also as trade off context switching between threads
> > (small but
> > ...)
> > Not enough to be a concern.
> > I think considering this trade offs we should switch
> > Real Thread -> Green Thread internally or provide alternative functionality
> > ...
> > I think asyncio already provide functionality of GreenThread ...
> > Okay maybe we should leave Real Threads as it is ... I need to think about
> > it ...
> > Yes, leave real threads as is, thanks. They're pretty important actually.
> > If you don't want to use threads, that's fine, but don't change the
> > threading module to not be threads. Not really a lot of point breaking
> > backward compatibility for, uhh, zero gain.
> > ChrisA
> > I like the topic on
> > https://www.reddit.com/r/Python/comments/bpp9hg/has_the_python_gil_been_slai...
> Answer written by CSI_Tech_Dept:
> ...
> What probably needs to be done is to have a compile option that causes Python
> to work
> without GIL but at the cost of breaking some of API (this would especially
> broke C API),
> then developers would have an option to chose:
>
> compatibility with GIL
>
> no GIL, full multithreading, but broken compatibility
>
> If user could decide which python version to use and there was a clear
> benefit, existing
> applications would be migrated, people would also make sure their code works
> with both
> versions.
> ...
>
> Interesting thoughts ...
Also I have tried to add in CPython adaptive reference counter ...
What is it ...
It is reference count that is based on knowledge that there are some some
additional thread ...
Take a look at this implementation:
```
inline void add_reference_couter(atomic_int * acnt)
{
if (num_other_threads > 0) {
atomic_fetch_add_explicit(acnt, 1, memory_order_acq_rel);
} else {
++(*acnt);
}
}
inline void sub_reference_couter(atomic_int * acnt)
{
if (num_other_threads > 0) {
atomic_fetch_sub_explicit(acnt, 1, memory_order_acq_rel);
} else {
--(*acnt);
}
}
```
Or implementation with complex RefCounter:
```
struct RefCounter {
atomic_uint acnt;
uint cnt;
};
inline void add_reference_couter(RefCounter * ref_counter)
{
if (num_other_threads > 0) {
atomic_fetch_add_explicit(&(*ref_counter).acnt, 1, memory_order_acq_rel);
} else {
++(*ref_counter).cnt;
}
}
inline void sub_reference_couter(RefCounter * ref_counter)
{
if (num_other_threads > 0) {
atomic_fetch_sub_explicit(&(*ref_counter).acnt, 1, memory_order_acq_rel);
} else {
--(*ref_counter).cnt;
if (0 == (*ref_counter).cnt) {
atomic_fetch_sub_explicit(&(*ref_counter).acnt, 1,
memory_order_acq_rel);
}
}
}
```
I have also prepared test branch on my Ubuntu machine for this implementation
...
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/TY3OD4N5X4K7V3V44RZKGE4GYK4PJGCH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
Chris Angelico wrote: > On Mon, May 25, 2020 at 7:58 PM redrad...@gmail.com > wrote: > > > > Edwin Zimmerman wrote: > > Only if your workload is CPU bound. Python optimizes IO bound workload > > performance by > > releasing the GIL while doing IO. Green threads generally do not offer > > this option. > > Real threads is not needed in Python: > > 1) Real threads do not work parallel > > They do when they do any sort of blocking operation. > > 2) Real threads only consume resources of OS > > 3) Real threads have also as trade off context switching between threads > > (small but > > ...) > > Not enough to be a concern. > > I think considering this trade offs we should switch > > Real Thread -> Green Thread internally or provide alternative functionality > > ... > > I think asyncio already provide functionality of GreenThread ... > > Okay maybe we should leave Real Threads as it is ... I need to think about > > it ... > > Yes, leave real threads as is, thanks. They're pretty important actually. > If you don't want to use threads, that's fine, but don't change the > threading module to not be threads. Not really a lot of point breaking > backward compatibility for, uhh, zero gain. > ChrisA I like the topic on https://www.reddit.com/r/Python/comments/bpp9hg/has_the_python_gil_been_slain_subinterpreters_in/ Answer written by CSI_Tech_Dept: ``` ... What probably needs to be done is to have a compile option that causes Python to work without GIL but at the cost of breaking some of API (this would especially broke C API), then developers would have an option to chose: compatibility with GIL no GIL, full multithreading, but broken compatibility If user could decide which python version to use and there was a clear benefit, existing applications would be migrated, people would also make sure their code works with both versions. ... ``` Interesting thoughts ... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NIRAP5KUE2C7N5TI52IX44QNDJVQUTSC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
Edwin Zimmerman wrote: > On 5/25/2020 5:56 AM, redrad...@gmail.com wrote: > > Edwin Zimmerman wrote: > > Only if your workload is CPU bound. Python optimizes IO bound workload > > performance by > > releasing the GIL while doing IO. Green threads generally do not offer > > this option. > > Real threads is not needed in Python: > > 1) Real threads do not work parallel > > 2) Real threads only consume resources of OS > > As I said above, threads in Python provide limited parallelization when do > > IO. For > > example, if I have a thread that reads a 1GB file into memory, that thread > > releases the > > GIL while reading the file, allowing other threads to run in parallel. > > 3) Real threads have also as trade off context switching between threads > > (small but > > ...) > > I think considering this trade offs we should switch Real Thread -> Green > > Thread > > internally or provide alternative functionality ... > > I think asyncio already provide functionality of GreenThread ... > > Okay maybe we should leave Real Threads as it is ... I need to think about > > it ... > > But also in my email I have mentioned about interpreters.run_string > > @Edwin Zimmerman Do you know if it is blocking call ? And if yes than I do > > not > > understand how with such API we will have TRUE Parallel execution, because > > in this case > > only one Thread at the same time could be executed, but those threads could > > be executed > > only one at the time !! > > Also there are drawback that we need additional thread for run > > sub-interpretter !! > > Better would be the something like this in addition > > interpreters.run_string_async, interpreters.run_async in > > addition to interpreters.run_string, interpreters.run to remove > > trad offs > > Sub-interpreters are a work in progress. The API is not anywhere near being > finalized. > True parallel execution will occur only if the GIL is changed to a per > interpreter lock, > and that requires some rather large changes to Python that haven't happened > yet. > Why ? True parallel execution is possible with sub-interpreters !! `Sub-interpretter` should be run in separate thread not controlled by GIL and `sub-interpretter` will notify `main interpreter` with `atomic variable` that it is finished (like worker) !! We just need to provide two APIs: 1) Synchronized: `run`, `run_string`, that will wait until thread notify with setting `atomic variable` in true that it finished 2) Asynchronized (based on async): `run_async`, `run_string_async`, event_loop will wait on `atomic variable` reading it periodically ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DUE7XDHDPATUPPWYOOWCGP7PT6ECY76C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Python JIT Compilation Thoughts
Hi all, I do not know maybe it was already discussed ... but the toolchain like LLVM is very mature and it can provide the simpler JIT compilation to machine code functionality and it will improve performance of the Python a lot !! ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/A3VF6DJ3H2IIUAYYEKSS3T3N7X443OO7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python GIL Thoughts
>> Edwin Zimmerman wrote: Only if your workload is CPU bound. Python optimizes IO bound workload performance by releasing the GIL while doing IO. Green threads generally do not offer this option. Real threads is not needed in Python: 1) Real threads do not work parallel 2) Real threads only consume resources of OS 3) Real threads have also as trade off context switching between threads (small but ...) I think considering this trade offs we should switch Real Thread -> Green Thread internally or provide alternative functionality ... I think asyncio already provide functionality of GreenThread ... Okay maybe we should leave Real Threads as it is ... I need to think about it ... But also in my email I have mentioned about `interpreters.run_string` @Edwin Zimmerman Do you know if it is blocking call ? And if yes than I do not understand how with such API we will have TRUE Parallel execution, because in this case only one Thread at the same time could be executed, but those threads could be executed only one at the time !! Also there are drawback that we need additional thread for run sub-interpretter !! Better would be the something like this in addition `interpreters.run_string_async`, `interpreters.run_async` in addition to `interpreters.run_string`, `interpreters.run` to remove trad offs ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SZKLGCO6XYDTGQKXGQLLKJAEE7XMM3NW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Python GIL Thoughts
Hi all,
I am very exciting about the sub-interpreters ... but I do not like some parts
...
Consider the following code:
```python
import _xxsubinterpreters as interpreters
import threading
import textwrap as tw
import marshal
if __name__ == '__main__':
# Create a sub-interpreter
interpid = interpreters.create()
# If you had a function that generated some data
arry = list(range(0,100))
# Create a channel
channel_id = interpreters.channel_create()
# Pre-populate the interpreter with a module
interpreters.run_string(interpid, "import marshal; import
_xxsubinterpreters as interpreters")
# Define a
def run(interpid, channel_id):
# Still is run in thread which is
use GIL
interpreters.run_string(interpid,
tw.dedent("""
arry_raw = interpreters.channel_recv(channel_id)
arry = marshal.loads(arry_raw)
result = [1,2,3,4,5] # where you would do some calculating
result_raw = marshal.dumps(result)
interpreters.channel_send(channel_id, result_raw)
"""),
shared=dict(
channel_id=channel_id
),
)
inp = marshal.dumps(arry)
interpreters.channel_send(channel_id, inp)
# Run inside a thread
t = threading.Thread(target=run, args=(interpid, channel_id))
t.start()
# Sub interpreter will process. Feel free to do anything else now.
output = interpreters.channel_recv(channel_id)
interpreters.channel_release(channel_id)
output_arry = marshal.loads(output)
print(output_arry)
```
Add some `async` execution of `interpreters.run_string_async` and other async
functions !!
Also regarding the threads ... I think it is better to make `threading.Thread`
as green `Thread` !!
I know, I know, `Thread` is not possible due to `sync io` ... But we can to
change underling `api` from `sync io` to `async io` !!
User will notice nothing, but performance will be increased mush more !!
Consider the following execution of `threading.Thread` in Python:
```
gil_lock.acquire()
for green_thread in green_threads:
thread.execute()
gil_lock.release()
```
If we implement in such way we will be able to reduce resource used in system,
but performance would be the same !!
Sub-Interpreter would be the something like Workers in JavaScript, but much
more flexible !!
What we will have at the end:
1) `threading.Thread` is a green thread that do not consume a lot of resources
but still useful !!
2) Real parallelizm is done within sub-interpreters like workers in JavaScript,
but much more flexible
Say what do you think ?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NYKC4UCRQXT6K7BXACBTIU57MIAK2MPF/
Code of Conduct: http://python.org/psf/codeofconduct/
