Re: [Python-ideas] Add hooks to asyncio lifecycle
On 10 June 2018 at 07:59, Michel Desmoulin wrote: > What I'm proposing is to make that easy to implement by just letting > anyone put a check in there. > > Overriding policy, loops or tasks factories are usually down for > critical parts of the system. The errors emerging from a bug in there > are very cryptic. > > Asyncio design made the choice to expose very low level things. You > literally don't have this problem in languages like JS because nobody > can change those. > > Now it's here, it's a footgun, and it would be nice to provide a way to > put it in a holster. > With the API need framed that way, perhaps all that asyncio is currently missing is an "asyncio.lock_policy(unlock_token, err_callback)" API such that your application can declare that initialisation is completed and no further event loop policy changes should be allowed? (The "unlock_token" would be an arbitrary app-provided object that must also be passed to the corresponding "unlock_policy" call - that way libraries couldn't unlock the policy after the application locks it, since they won't have a reference to the app-specific unlock token). Adding further callback hooks for more events seems like it will just push the problem back another level, and you'll have the potential for conflicts between callbacks registered with the new hooks, and an even harder to understand overall system. By contrast, the above would be amenable to doing something like: 1. Per-process setup code establishes a particular event loop policy, and then locks it 2. Until the policy gets unlocked again, attempts to change it will call the err_callback (so the app can raise a custom access denied exception) 3. get_event_loop(), set_event_loop(), and new_event_loop() are all already managed by the event loop policy, so shouldn't need new hooks 4. stop(), close(), set_debug(), set_task_factory(), etc are all already managed by the event loop (and hence by the event loop policy), so shouldn't need new hooks Right now, the weak link in that chain is that there's no way for the application to keep a library from switching out the event policy with a new one, and subsequently bypassing all of the app's control over how it expects event loops to be managed. Given a way to close that loophole, the application should already have the ability to enforce everything else that it wants to enforce via the existing event loop and event loop policy APIs. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A PEP on introducing variables on 'if' and 'while'
On Sat, Jun 09, 2018 at 09:17:37AM -0400, Juancarlo Añez wrote: > > Do you mean the context manager semantics of with statements? As in, > > calling the __enter__ and __exit__ method? > > > > No. Just the scope of the variables introduced, which is different in `with > as` and `except as`. They aren't. They are the same scope: both `with` and `except` bind to a local variable. The only difference is that the `except` block implicitly unbinds the variable when the block ends. py> err = "something" py> try: ... None + 1 ... except TypeError as err: ... pass ... py> err Traceback (most recent call last): File "", line 1, in NameError: name 'err' is not defined > > Please make sure you are very familiar with PEP 572 before you do, and > > expect to have your PEP compared to it. > > My intention would be to make the to proposals orthogonal, if possible, so > both/any can be accepted or rejected in their own timeline. > > I'm certain that both can live together. Seems redundant... while (condition := expression) as flag: ... Accepting "while/if as name" would remove much (but not all) of the motivation for assignment expressions, while accepting assignment expressions would make a dedicated while/if as name syntax unnecessary. Like it or not, I expect that they will be seen as competing PEPs, not independent ones. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add hooks to asyncio lifecycle
>
> IMHO, it is not any framework's job to check for this. It is a
> programmer error.
Not clearing the memory is a programmer error either but a gc helps.
Not closing a file either but using `with` help.
Not passing the proper type is a programmer error but we have type hints.
We even have assert that we can be disabled to check that the arguments
of a function match a test when debugging.
You don't need to babysit programmers so much.
Helping to debug is not babysitting. It's a important part of the user
experience, and it's especially needed on asyncio, one of the hardest
part of the stdlib to code with.
Not if the cost is adding new APIs.
The cost of bugs is greater than the cost of API. The current API gives
no sane way to prevent the bug. You can't expect anyone to check if the
policy / loop / task factory has changed every single time in the code
you depend on it.
>
> I agree with Andrew, if we open this precedent, next thing we know,
> Python has to provide callbacks for any internal state changes.
It's not internal, it's a public API.
Why not
> callbacks for when modules are imported?
Good example.
We have import hooks to run code every time a module is imported :
https://www.python.org/dev/peps/pep-0302/#specification-part-2-registering-hooks
Etc. etc. It leads to API
> noise.
Every features is either useful or noise. I argue that this one is useful.
Doesn't seem to be justified in this case, since I would guess
> almost all applications only change event loop policy once, during
> startup, and never again.
Yes, but when is start up ?
E.G: I'm currently working on a large QT api on an old 2.7 code base. It
has threads to avoid blocking the UI, qt events loops of course and the
tornado events loop providing a web API. The orchestration of that very
complex app requires careful initialization for a start up time of about
7s, and we had to code some wrapper and document it as the only entry
point to make sure that if a junior breaks things the error message is
very clear.
RuntimeError('The event loop should not be set when...') is way easier
to debug than an obscure down the road error on some data structure that
is incorrectly accessed.
What I'm proposing is to make that easy to implement by just letting
anyone put a check in there.
Overriding policy, loops or tasks factories are usually down for
critical parts of the system. The errors emerging from a bug in there
are very cryptic.
Asyncio design made the choice to expose very low level things. You
literally don't have this problem in languages like JS because nobody
can change those.
Now it's here, it's a footgun, and it would be nice to provide a way to
put it in a holster.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow callable in slices
On 9 June 2018 at 19:20, Michel Desmoulin wrote: > Given 2 callables checking when a condition arises and returning True: > > def starting_when(element): > ... > > def ending_when(element: > ... > Allow: > > a_list[starting_when:] > > To be equivalent to: > > from itertools import dropwhile > > list(dropwhile(lambda x: not starting_when(x), a_list)) > Custom container implementations can already do this if they're so inclined, as slice objects don't type check their inputs: >>> class MyContainer: ... def __getitem__(self, key): ... return key ... >>> mc = MyContainer() >>> mc[:bool] slice(None, , None) >>> mc[bool:] slice(, None, None) >>> mc[list:tuple:range] slice(, , ) It's only slice.indices() that needs start/stop/step to adhere to the Optional[int] type hint. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Trigonometry in degrees
On Sat, Jun 9, 2018 at 2:22 AM Adam Bartoš wrote: > The idea was that the functions could handle the PiMultiple instances in a > special way and fall back to float only when a special value is not detected. > It would be like the proposed dsin functionality, but with a magic class > instead of a new set of functions, and without a particular choice of > granularity (360 degrees). > > But maybe it isn't worth it. Also what about acos(0)? Should it return > PiMultiple(1, 2) and confuse people or just 1.5707963267948966 and loose > exactness? > > That'd be the only module in the standard library with such a specialized class and behavior. You could argue that pathlib creates a sort of Path preference with str fallback, but the Path type has a large collection of methods. In contrast, this PiMultiple type would be only used as an input. That's very unusual style for Python. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow callables in slices
On Sat, Jun 9, 2018 at 6:28 AM Michel Desmoulin
wrote:
> Example, open this files, load all lines in memory, skip the first line,
> then get all the line until the first comment:
>
> import itertools
>
> def is_commented(line):
> return lines.startwith('#')
>
> def lines():
> with open('/etc/fstab'):
> lines = f.readlines()[1:]
> return list(itertools.dropwhile(lines, is_commented)
>
> Becomes:
>
> def is_commented(line):
> return lines.startwith('#')
>
> def lines():
> with open('/etc/fstab'):
> return f.readlines()[1:is_commented]
>
> It's not about how much shorter is is, but it is very nice to read.
>
If you're going to put it in a function anyway, why something like this?
def lines():
with open('/etc/fstab'):
f.readline()
for line in f:
if line.startswith('#'):
break
yield line
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add hooks to asyncio lifecycle
Le 09/06/2018 à 12:33, Andrew Svetlov a écrit :
> If we consistently apply the idea of hook for internal python structure
> modification too many things should be changed. Import
> machinery, tracemalloc, profilers/tracers, name it.
> If your code (I still don't see the real-life example) wants to check a
> policy change -- just do it.
>
> # on initialization
> policy = asyncio.get_event_loop_policy()
>
> # somewhere in code
> if policy is not asyncio.get_event_loop_policy():
> raise RuntimeError("Policy was changed")
You need to repeat this check everywhere because nothing guaranty a code
hasn't change it after the check.
While a call back allow you to catch it every time.
You can set either raise, warn, monkey patch, disable features, etc.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow callables in slices
Le 09/06/2018 à 11:47, Steven D'Aprano a écrit :
> On Sat, Jun 09, 2018 at 11:17:05AM +0200, Michel Desmoulin wrote:
>> Such as that:
>>
>> def starting_when(element):
>> ...
>>
>> a_list[starting_when:]
>
>> Is equivalent to:
> [...]
>> list(dropwhile(lambda x: not starting_when(x), a_list))
>>
>
> That looks like a slice from an index to the end of the list. Things
> which are similar should look similar, but things which are different
> should NOT look similar.
>
The semantic is [start_condition:stop_condition:step].
Here condition can be an index or something more complex.
Just like you can do dictionary[key], but key can be a complex object
with a custom __hash__ executing weird computation.
Just like you can sorted() on natural values or pass a callable as a key.
Just like you can re.replace() with a string or a function.
> What would:
>
> alist[callable:callable:callable]
>
> do? How about this one?
>
> alist[7:callable:-1]
ValueError("Step cannot be used when callables are part of a slice")
However:
alist[7:callable]
Would be:
list(dropwhile(lambda x: not starting_when(x), islice(alist, 7, None)))
Example, open this files, load all lines in memory, skip the first line,
then get all the line until the first comment:
import itertools
def is_commented(line):
return lines.startwith('#')
def lines():
with open('/etc/fstab'):
lines = f.readlines()[1:]
return list(itertools.dropwhile(lines, is_commented)
Becomes:
def is_commented(line):
return lines.startwith('#')
def lines():
with open('/etc/fstab'):
return f.readlines()[1:is_commented]
It's not about how much shorter is is, but it is very nice to read.
Of course I'd prefer to have slicing on generators, since here we load
the entire file in memory. But I already suggested it several times on
python-idea and the dictator killed it. For files like fstab it's ok
since they are small.
>
> If there are not meaningful interpretations of callables as part of
> general slice notation, then we shouldn't use slice notation as a
> shortcut for dropwhile.
>
> Rather than give this syntactic support, I'd rather add a new function
> to itertools that composes takewhile and dropwhile:
It's not syntaxic support. You can already pass callables and the
interpreter accept it fine.
But the underlying types raise TypeError because they don't know how to
use that. Just like you can pass tuples, but CPython can't use them,
while numpy can.
>
>
> def between(iterable, startcondition, endcondition):
> it = iter(iterable)
> return takewhile(lambda x: not endcondition(x),
>dropwhile(lambda x: not startcondition(x), it)
>)
Or accept callables in islice.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A PEP on introducing variables on 'if' and 'while'
> Do you mean the context manager semantics of with statements? As in, > calling the __enter__ and __exit__ method? > No. Just the scope of the variables introduced, which is different in `with as` and `except as`. > Please make sure you are very familiar with PEP 572 before you do, and > expect to have your PEP compared to it. > My intention would be to make the to proposals orthogonal, if possible, so both/any can be accepted or rejected in their own timeline. I'm certain that both can live together. -- Juancarlo *Añez* ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A PEP on introducing variables on 'if' and 'while'
On Sat, Jun 09, 2018 at 08:43:47AM -0400, Juancarlo Añez wrote: > Hello @here, > > Is there a guide about writing (and publishing) PEPs? https://www.python.org/dev/peps/pep-0001/ > I'd like to write one on `while expre as v: ...` using the context > semantics of `with expr as v` (not `except E as e`). Do you mean the context manager semantics of with statements? As in, calling the __enter__ and __exit__ method? Please make sure you are very familiar with PEP 572 before you do, and expect to have your PEP compared to it. Be especially prepared to be challenged with the question what is so special about while and if that they, and they alone, are permitted to use assignment expressions. (You don't have to answer that now provided it is answered in the PEP.) -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A PEP on introducing variables on 'if' and 'while'
On 2018-06-09 14:43, Juancarlo Añez wrote: Is there a guide about writing (and publishing) PEPs? https://www.python.org/dev/peps/pep-0001/ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] A PEP on introducing variables on 'if' and 'while'
Hello @here, Is there a guide about writing (and publishing) PEPs? I'd like to write one on `while expre as v: ...` using the context semantics of `with expr as v` (not `except E as e`). Cheers, ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Making Path() a built in.
On Wed, Jun 06, 2018 at 07:05:35PM +0100, Barry Scott wrote: > I assume the the idea is that everybody has Path available without the need > to do the import dance first. > > If its for personal convenience you can always do this trick, that is used by > gettext to make _ a builtin. > > import pathlib > import builtings > > builtins.__dict__['Path'] = pathlib.Path The public API for getting the namespace of an object is vars(): vars(builtins)['Path'] but since builtins is just a module, the best way to add a new attribute to it is: builtins.Path = pathlib.Path -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Allow callable in slices
Given 2 callables checking when a condition arises and returning True: def starting_when(element): ... def ending_when(element: ... Allow: a_list[starting_when:] To be equivalent to: from itertools import dropwhile list(dropwhile(lambda x: not starting_when(x), a_list)) And: a_list[:ending_when] To: from itertools import takewhile list(takewhile(lambda x: not ending_when(x), a_list)) ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "within" keyword
On Sat, Jun 09, 2018 at 03:07:39PM +1000, Nick Coghlan wrote: > It's doable without code generation hacks by using class statements instead > of with statements. > > The withdrawn PEP 422 shows how to use a custom metaclass to support a > "namespace" keyword argument in the class header that redirects all writes > in the body to the given dict: > https://www.python.org/dev/peps/pep-0422/#new-ways-of-using-classes > > https://www.python.org/dev/peps/pep-0422/#extending-a-class even shows how > to further use that to extend existing classes with new attributes. That's awesome! I've been poking away at similar ideas for a long time. -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "within" keyword
Classes Provide already some features of a namespace :
class cool_namespace:
A = 8
@staticmethod
def f():
return "yo"
@staticmethod
def g():
return (1 + cool_namespace.A) * cool_namespace.f()
And if you're tired of writing @staticmethod, you can write a class
decorator "namespace" :
@namespace
class cool_namespace:
A = 8
def f():
return "yo"
def g():
return (1 + cool_namespace.A) * cool_namespace.f()
And I think this decorator already exists somewhere.
Le sam. 9 juin 2018 à 10:21, Steven D'Aprano a écrit :
> On Fri, Jun 08, 2018 at 03:07:28PM -0700, Michael Selik wrote:
>
> > You can use ``eval`` to run an expression, swapping in a different
> globals
> > and/or locals namespace. Will this serve your purpose?
> >
> > In [1]: import types
> > In [2]: ns = types.SimpleNamespace(a=1)
> > In [3]: eval('a', ns.__dict__)
> > Out[3]: 1
>
> The public API for getting an object namespace is vars(ns).
>
> But why would we write eval('a', vars(ns)) instead of getattr(ns, 'a')
> or even better just ns.a? Is your Python code too fast and you need to
> slow it down? *wink*
>
> eval and exec are useful when the code you want to run needs to be
> constructed at runtime. Its not generally useful when you know what you
> want ahead of time as in your example above.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "within" keyword
On Fri, Jun 08, 2018 at 02:41:54PM -0400, David Teresi wrote:
> One of the features I miss from languages such as C# is namespaces that
> work across files - it makes it a lot easier to organize code IMO.
I too have often wanted a sub-module namespace without the need to
separate code into seperate files. Something like a package, but
existing inside a single physical file. In pseudo-code:
a = 1
def spam():
return a
namespace A:
# conceptually, this is like a module within a module
a = 100
def spam():
return a
# later
spam() + A.spam()
=> returns 101
If that looks similar to the class statement, except there's no need to
create an instance, that's deliberate.
> Here's an idea I had - it might not be the best idea, just throwing this
> out there: a "within" keyword that lets you execute code inside a
> namespace. For example:
[...]
> within cool_namespace:
> def foo():
> print("foo run")
[...]
> within A.cool_namespace:
> foo() # prints "foo run"
This sort of thing is similar to a old FAQ:
https://docs.python.org/3/faq/design.html#why-doesn-t-python-have-a-with-statement-for-attribute-assignments
so it isn't unambiguously clear what foo would mean: is it the current
namespace foo, the surrounding namespace, or the module namespace?
Its not necessarily undoable: currently Python has what the "Learning
Python" book calls the LEGB scoping rule:
L - local
E - enclosing function(s) or class
G - global (module)
B - builtins
It could be conceivable to add a rule to slot a namespace in there
somewhere, but the scoping rules already are fairly hairy (e.g. classes
and functions work a little differently, we have a sub-local scope for
comprehensions) and I'd be cautious about making them hairier with
something like your "within"/"with" statement.
The question I have is what is your motive for this? What problem does
this solve for you?
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Making Path() a built in.
Creating built in dynamically is not a good idea. Tools complain, new comers wonder where it comes from, it sets a precedent for adding more or debating about it. Better have the debate once here, make it official or decline it officially, and have a clean result. Le 08/06/2018 à 21:28, Barry a écrit : > I think you forgot to to reply to the list. > Barry > > >> On 8 Jun 2018, at 13:16, Michel Desmoulin wrote: >> >> Creating builtin dynamically is not a good idea. tools complains, new comers >> wonder where it comes from, it sets a precedent for adding more or debating >> about it. >> >> Better have the debate once here, make it official or decline it officially, >> and have a clean result. >> >>> Le 06/06/2018 à 20:05, Barry Scott a écrit : >>> I assume the the idea is that everybody has Path available without the need >>> to do the import dance first. >>> >>> If its for personal convenience you can always do this trick, that is used >>> by gettext to make _ a builtin. >>> >>> import pathlib >>> import builtings >>> >>> builtins.__dict__['Path'] = pathlib.Path >>> >>> Now Path *is* a builtin for the rest of the code. >>> >>> Barry >>> >>> >>> >>> >>> ___ >>> Python-ideas mailing list >>> Python-ideas@python.org >>> https://mail.python.org/mailman/listinfo/python-ideas >>> Code of Conduct: http://python.org/psf/codeofconduct/ >>> >> > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] A "within" keyword
On Fri, Jun 08, 2018 at 03:07:28PM -0700, Michael Selik wrote:
> You can use ``eval`` to run an expression, swapping in a different globals
> and/or locals namespace. Will this serve your purpose?
>
> In [1]: import types
> In [2]: ns = types.SimpleNamespace(a=1)
> In [3]: eval('a', ns.__dict__)
> Out[3]: 1
The public API for getting an object namespace is vars(ns).
But why would we write eval('a', vars(ns)) instead of getattr(ns, 'a')
or even better just ns.a? Is your Python code too fast and you need to
slow it down? *wink*
eval and exec are useful when the code you want to run needs to be
constructed at runtime. Its not generally useful when you know what you
want ahead of time as in your example above.
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Fwd: New suggested built in keyword: do
The benefit of list, dict, and set comprehensions and generator expressions is that they evaluate, as opposed to simply exec. The purpose of making them one-liners is to allow them to be assigned to a variable or passed as an argument. If you're not assigning or passing, then why not use a newline character? "Sparse is better than dense." On Friday, June 8, 2018 at 7:13:07 AM UTC-7, Randy Diaz wrote: > > I think that the keyword do would solve problems that occur when people > want a simple way to run a command over an iterable but they dont want to > store the data. > > example: > do print(x) for x in range(50) > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
