[Python-ideas] Deprecate PEP 249 (DB-API 2.0)

[Soni L.]

We would like to propose the following improvements to DB-API 2.0 that 
would require bumping it up to DB-API 3.0:


- Get rid of SQL strings
- Get rid of SQL strings
- Use package resources to store what would otherwise be SQL strings

While we cannot prevent someone from going out of their way to define 
package resources at runtime just so they can implement SQL injection, 
ultimately the goal is to provide a small speed bump so they don't feel 
so inclined to jump straight into SQL injection before trying to do 
easier, more secure things.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/STPNELT3ZP337ELTDTL7QR43N7BZOWXV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Virtual packages

[Soni L.]

Python allows setting __path__ to turn a module into a package. However, 
it's allegedly only defined for __init__.py?


Proposal:

A virtual package is a non-__init__.py whose first line of code, after 
comments and future imports, assigns to __path__.


This would allow -m virtualpackage to call virtualpackage.__main__, 
wherever that is. Instead of running virtualpackage.py as __main__.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UTSSQ7IHSPLHF5DCEP2TIGKCVTNLDLME/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] The exact operation of -m is unclear

[Soni L.]

A simple virtual package could be:

# foo.py
__path__ = [__file__ + ".d"]

But unfortunately this doesn't work with -m:

# foo.py.d/__main__.py
print("this will never run")

(n.b. -m foo.__main__ works but nobody's writing that.)

Is it possible to change this backwards-compatibly? If not, is it 
possible to document it slightly better?


Thanks.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AAE7CRMOVAUKVBH2U3RWTBV32HQIDWVF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: A modulo operator consistent with euclidean division.

[Soni L.]

We feel like this may make sense to bring up, too:

https://rust-lang.github.io/rfcs/2169-euclidean-modulo.html

On 2022-03-20 04:06, Om Joshi wrote:
> Has anyone in this thread linked this blog post yet?
>
> http://python-history.blogspot.com/2010/08/why-pythons-integer-division-floors.html
>
> Much is a rehash of this thread but for completeness it might be useful to 
> read Guido van Rossum's thoughts and the subsequent discussion in the 
> comments section.
>
> @Tim I believe you are mentioned at the bottom of the post.
>
> Om
>
>
>   On Sun, 20 Mar 2022 01:53:51 -0500 Ben Rudiak-Gould 
>  wrote 
>  > On Fri, Mar 18, 2022 at 8:30 PM MRAB  wrote:
>  > > Wikipedia describes Euclidean division.
>  > >
>  > > Basically, the modulo is non-negative:
>  > >
>  > >  a == b * q + r where 0 <= r < abs(b)
>  > 
>  > That convention in the Wikipedia article dates back to a 2004 edit by
>  > an anonymous (IP) editor. The only reference in that version was to a
>  > course handout that only considered positive denominators.
>  > 
>  > There's nothing wrong with the convention, but I'm suspicious of the
>  > idea that it's a widespread standard of some sort. I've never heard of
>  > it before.
>  > ___
>  > Python-ideas mailing list -- python-ideas@python.org
>  > To unsubscribe send an email to python-ideas-le...@python.org
>  > https://mail.python.org/mailman3/lists/python-ideas.python.org/
>  > Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MG7J6KWWBWZPMWVZYSMAXF4WVG2WL42A/
>  > Code of Conduct: http://python.org/psf/codeofconduct/
>  > 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/RAX6SUFI2DKLV6KQXJKQCF63XZIBLMUI/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2CVSPWNCBT3HCZHUYUKKLIXII2TRL3MK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Creating ranges with ellipsis

[Soni L.]

On 2022-02-16 10:45, Steven D'Aprano wrote:
> On Wed, Feb 16, 2022 at 09:44:07AM -0300, Soni L. wrote:
> > This might be a silly idea but, would it be a good idea to have
> > ...[a:b:c] return a range(a, b, c)?
>
> Similar ideas have been suggested before:
>
> https://mail.python.org/archives/list/python-ideas@python.org/thread/W44PPBJJXETTBQHWCMJB3DRCD6CTXWJT/
>
> https://bugs.python.org/issue42956
>
> What benefit do you see in writing [a:b:c] instead of range(a, b, c)?
>
>
>

*nod* we see.

syntax constructs like these are mostly about taste. it's like being
able to write generators in function calls like list(x for x in foo),
but also having (x for x in foo) instead of using a gen(x for x in foo)
function.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G2ZYUPM5MARUMELUV4FDDHJS6O6H42TB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Creating ranges with ellipsis

[Soni L.]

This might be a silly idea but, would it be a good idea to have
...[a:b:c] return a range(a, b, c)?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IBXS5ZHI2XTAKUFLOE2UWFBPCX7HUW75/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Antichecked exceptions

[Soni L.]

PEP 479 introduced this idea that basically boils down to raising your
exceptions only when you mean it.

Specifically: generators only raise StopIteration when they
return/exit/terminate. And they prevent the user from raising it under
other circumstances.

Can we call this "antichecked exceptions"? Generators specifically check
for StopIteration raised in undesired circumstances, and make sure they get
wrapped. This is a form of runtime checking, so the "checked" part of the
name makes sense. But this is unlike "checked exceptions", like those in
java, that mostly just encourage uhh silently ignoring the exception? (And
those are *statically* checked anyway...) So the prefix "anti"
distinguishes it from that.

Obviously this name is meant to be more broad than just the implementation
of generators. (We made a pypi package that tries to expand on the
concept.) But, thoughts? Naming things is powerful.

(Is this the wrong list for this? Uhh like it's a name idea for a thing in
python? But uhh... it's not so much a change proposal? Except to refer to
it by this new name? But anyway...)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PQKWKRMOHDOOEV5Q4HDT7FOVQO4MRHAG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-10-01 8:26 a.m., Steven D'Aprano wrote:
> I wish your proposals and ideas would be more *precise* in their 
> specifications. This is not the first time that I have found it very 
> hard to work out precisely what you are suggesting and what you are not. 
>
>

Thanks for insisting on context managers. Does the attached context
manager clarify what the proposal is about?
# Antichecked Exceptions for Python
# Copyright (c) 2021 Soni L.
#
# Permission is hereby granted, free of charge, to any person ("You") obtaining
# a copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# This license shall be void if You bring a copyright lawsuit, related or
# unrelated to the Software, against any of the copyright holders.
#
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.

class _hack:
def __init__(self, ctx):
self.ctx = ctx

def __enter__(self):
pass

def __exit__(self, exc_ty, exc_val, exc_tb):
exc_val.__context__ = self.ctx

class exceptions:
"""A context manager for antichecked exceptions.

Antichecked exceptions are the reverse of checked exceptions: rather than
requiring the caller to catch the exceptions, they actively check that
your function/block actually intended to raise them.

Examples:

>>> from antichecked import exceptions

Intercepting exceptions::

>>> with exceptions(StopIteration) as r:
... raise StopIteration
RuntimeError

Raising exceptions::

>>> with exceptions(StopIteration) as r:
... raise r(StopIteration)
StopIteration

Adding a cause::

>>> with exceptions(StopIteration) as r:
... raise r(StopIteration) from ValueError()
StopIteration from ValueError

Similarly, re-raising exceptions is done with ``raise r``.
"""

def __init__(self, *args):
"""Creates an antichecked exception context for the given exceptions.

Args:
Accepts an arbitrary amount of exception types.
"""
# each context manager gets an unique exception type
class WrappedError(Exception):
def __init__(self, exc=None):
if isinstance(exc, BaseException) or exc is None:
self.value = exc
else:
self.value = exc()
self._exceptions = args
self._error = WrappedError

def __enter__(self):
return self._error

def __exit__(self, exc_ty, exc_val, exc_tb):
if exc_ty is None:
return
if exc_ty is self._error:
# handle `raise r(exc)` similar to `raise exc`
if exc_val.value is not None:
exc = exc_val.value
exc.__cause__ = exc_val.__cause__
with _hack(exc_val.__context__):
raise exc.with_traceback(exc_tb)
# `raise r from foo` is unhandled, as `raise from foo` is not
# valid python. TODO make this case a RuntimeError.
pass
# handle `raise r` similar to `raise`
if exc_val.__context__ is None:
with _hack(None):
exc = RuntimeError("No active exception to re-raise")
raise exc.with_traceback(exc_tb)
with _hack(exc_val.__context__.__context__):
raise exc_val.__context__
if any(issubclass(exc_ty, e) for e in self._exceptions):
# handle this similar to StopIteration in generators
raise RuntimeError("Antichecked exception raised") from exc_val
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5D3CNOHU72SVRXUVB6B76HS6Z7QZL6T4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-10-01 8:26 a.m., Steven D'Aprano wrote:
> On Thu, Sep 30, 2021 at 01:25:42PM -0300, Soni L. wrote:
>
> >     def foo():
> >     raise Bar
> >     def baz() with Bar:
> >     foo()
> >     baz()
> > 
> > would make a RuntimeError, because foo raised a Bar and the VM sees that
> > Bar is in baz's with.
>
> So the syntax "with SomeException" in a function declaration would mean 
> that the function can not raise SomeException?
>
> Wouldn't "without SomeException" be a less misleading name then?
>
>
> > *except "raise" opcodes SKIP checking these (within the context of the
> > function), so the following:
> > 
> >     def baz() with Bar:
> >     raise Bar
> >     baz()
> > 
> > raises a Bar, not a RuntimeError from a Bar.
>
> Every time I think I understand your proposal, I read some more about 
> it, and get more confused.
>
> So the "with SomeException" syntax guarantees that the function *won't* 
> raise SomeException (raising RuntimeError instead), unless it does raise 
> SomeException, which under some unspecified conditions it will allow it
> through.
>
> And this syntax can only apply to an entire function at a time, 
> you can't apply it to a block of code like a with-statement unless you 
> move the entire block into its own function.
>
>
> > You can then document your exceptions as you normally do: "This raises
> > Bar when so and so are invalid", and then add "with Bar" to your
> > function. the VM then makes sure that Bar is only raised when so and so
> > are invalid, but under no other circumstances.
>
> Wait. This is even more magical. So if I document "this only raises Bar 
> if P == NP" the compiler will solve the P versus NP problem?
>
> https://en.wikipedia.org/wiki/P_versus_NP_problem
>
> Okay, I jest a bit. A more realistic example:
>
>
> def myfunc(s:str) with ValueError:
> """This raises ValueError only if s is a palindrome."""
> assert isinstance(s, str)
> do_stuff()  # Guard: ValueError --> RuntimeError
> if s == s[::-1]:
> raise ValueError("spam")  # No guard.
> n = math.sqrt(float(s))  # Guard: ValueError --> RuntimeError
> ... # more code here is still guarded
>
>
> Surely you aren't suggesting that the compiler infers meaning from the 
> natural language docstring, but if not, how does it know which parts of 
> the function to guard and which parts not to?
>
> How does the VM know that ValueError("spam") is okay but 
> ValueError("math domain error") is not? What is the actual criteria used 
> to allow some ValueErrors through and not others?

It's not the ValueError(...). It's the location of the "raise".

>
> > e.g. someone
> > monkeypatched your function to try and raise a Bar outside of the place
> > it was meant to raise a Bar? it'll raise a RuntimeError instead.
>
> So if my function is this:
>
> def function(arg) with ValueError:
> """Raises ValueError if arg == 0."""
> if arg = 0:
> raise ValueError
> return arg + 1
>
> you are saying that if I monkeypatch it to this:
>
> def patched(arg):
> if arg == 0:
> return 1
> raise ValueError
>
> function = patched
> function(99)
>
> that somehow it will raise RuntimeError? I don't want to believe that 
> this is what you actually mean, but if it isn't this, then I don't know 
> what you do mean.
>
> I wish your proposals and ideas would be more *precise* in their 
> specifications. This is not the first time that I have found it very 
> hard to work out precisely what you are suggesting and what you are not. 
>
>

We did specify it: "raise" in body raises anything, regardless of the
value matching something in the "with". deeper exceptions that match
those in "with" get caught and wrapped in a RuntimeError.

We're saying if your function is this:

def function(arg) with ValueError:
  if arg == 0: raise ValueError
  return foo(arg) + 1

and someone monkeypatches it:

def foo(arg):
  raise ValueError

then your function will raise a RuntimeError.

Could this be added to context managers instead? Could "raise" within
"with" call the context manager with the exception? Probably!

with foo:
  raise Bar

# becomes semantically equivalent to

with foo:
  exception = Bar
  # break out of the context without calling __exit__, and then...
foo.__raise__(exception)

(the default __raise

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-10-01 2:04 a.m., Steven D'Aprano wrote:
> Let's get to the fundamental problem with this. It is DWIM magic, and 
> you haven't (as far as I have seen) yet specified how we are supposed to 
> use it or predict how it is supposed to work.
>
> Here is your syntax again:
>
> > > def a_potentially_recursive_function(some, args) with 
> > > ExceptionWeCareAbout:
> > > some.user_code()
> > >     code_we_assume_is_safe()
> > > if args.something and some_condition:
> > > raise ExceptionWeCareAbout  # Line (A)
> > >
> > > How does the compiler know that *only* ExceptionWeCareAbout originating
> > > in Line (A) should be re-raised, and any other location turned into
> > > RuntimeError?
> >
> > Same way Rust decides whether to propagate or unwrap a Result: you
> > *must* tell the compiler.
>
> How do I tell the compiler? There is no sign in your syntax that Line
> (A) is special or different from the rest of the lines in the function.
>
> > > What if I factor out those last two lines and make it:
> > >
> > > def a_potentially_recursive_function(some, args) with 
> > > ExceptionWeCareAbout:
> > > some.user_code()
> > >     code_we_assume_is_safe()
> > > check_condition_or_raise(args.something, some_condition)
> > >
> > > How does the compiler decide to re-raise exceptions originating in the
> > > last line but not the first two?
> >
> > In this case, it explicitly doesn't.
>
> So by refactoring a conditional raise (if condition: raise) into a
> function, I completely change the meaning of the code? That's just
> great. Not.
>
>
> > You explicitly told it the last line doesn't raise any exceptions that
> > contribute to your API's exception surface.
>
> I did? I wasn't aware that I told the interpeter *explicitly* anything
> about the last line. How does this work?
>
> That's what I'm trying to understand about your proposal: I write a
> function, and stick "with ExceptionName" into the function declaration
> line:
>
> def function(some, args) with ExceptionWeCareAbout:
>
> and then write a block of code under that declaration, and *somehow* in
> some completely unspecified way the compiler Does What I Mean by
> deciding that *some* of those lines which raise ExceptionWeCareAbout it
> should let the exception through while *other* lines that raise the same
> exception should be guarded against ExceptionWeCareAbout and have them
> raise RuntimeError instead.
>
> And I have no idea how it decides which lines are guarded and which are
> not. You tell me that I explicitly instructed the compiler which
> lines should be guarded, but I don't know how I did it.
>
>
> > You *must* use try: check_condition_or_raise(args.something,
> > some_condition) except ExceptionWeCareAbout: raise
>
> But that's not what the pre-refactoring code had. All I did was move:
>
> # The original version, before refactoring.
> if args.something and some_condition:
> raise ExceptionWeCareAbout
>
> into a named function and call that. In the original code, I didn't have
> to explicitly catch the exception and then immediately re-raise it:
>
> # This was never used.
> try:
> if args.something and some_condition:
> raise ExceptionWeCareAbout
> except ExceptionWeCareAbout:
> raise
>
> But now you are telling me that if I move the `if... raise` lines into a
> function I have to also wrap it in a try...except, catch the exception,
> and immediately and unconditionally re-raise it.
>
> This behaviour may be clear to *you* but I cannot read your mind. Unless 
> you actually tell me how the compiler knows which part of the block need 
> to be guarded and which parts don't, I have no clue how this is 
> happening except "the compiler magically infers what I want it to do".
>
>
>
Look for the clarification post we posted as a reply to the original post.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F6W5PWT253XLOFQRGV4LSHUADHC2GGJD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 6:08 p.m., Chris Angelico wrote:
> I still think that your use of LookupError is confusing the issue
> somewhat, partly because it's a base class rather than something
> that's ever raised per se.
>
> If you were using a custom exception type, how likely is it that that
> exception would be raised during the unpacking? Under what situations
> would this happen, and might it actually be an intended way for that
> property value to say "actually, heh... I don't exist"? Because the
> logical way to spell that would be "raise PropertyNotFoundError" (or
> whatever you use instead of LookupError). IOW, bubbling is exactly
> correct. You still haven't explained how that isn't the case.

Bubbling is correct, but it doesn't deny that explicit is better than
implicit.

Rust's `?` operator (generally referred to as "try" by the devs) has a
lot that can be learned from, here. (Not to say it would be the
appropriate solution for Python.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4324QCNQDUVJZPFZR6BMCEKKV4LT3YVE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 5:34 p.m., Barry Scott wrote:
>
>
>> On 30 Sep 2021, at 17:25, Soni L. > <mailto:fakedme...@gmail.com>> wrote:
>>
>> Alright, some ppl asked us to rephrase this, so:
>>
>> The plan is to take the function syntax:
>>
>>     def name(args):
>>
>> and add an optional "with" to it:
>>
>>     def name(args) with exceptions:
>>
>> these then get added to the function, similar to e.g. default args. when
>> an exception is thrown*, the VM then checks these and converts relevant
>> exceptions into RuntimeError, e.g.:
>>
>>     def foo():
>>     raise Bar
>>     def baz() with Bar:
>>     foo()
>>     baz()
>>
>> would make a RuntimeError, because foo raised a Bar and the VM sees that
>> Bar is in baz's with.
>
> Does with Bar mean that Bar is expected?
>
> If so who cares if foo raises it? Are you really saying I cannot call
> functions to
> implement a complex algorithm that raises exceptions?

No, we're not saying that, altho it would be more verbose. Consider the
example:

    def foo():
    raise Bar
    def baz() with Bar:
    try:
    foo()
    except Bar:
    raise
    baz()

This would successfully propagate foo's Bar to baz's caller. That's all
- just making the exception propagation points explicit rather than
implicit. (heh.)

>
> The caller of baz is expecting Bar right?
>
>>
>> *except "raise" opcodes SKIP checking these (within the context of the
>> function), so the following:
>>
>>     def baz() with Bar:
>>     raise Bar
>>     baz()
>>
>> raises a Bar, not a RuntimeError from a Bar.
>
> You want to include the exceptions that a function can raise in its
> signature and have python enforce rules based on that information.
>
> C++ had/has this feature and it failed in practice so it has been
> deprecated.
> I'd be surprised that it will be useful in python given this
> experience in the C++ world.

Unlike checked exceptions, this idea explicitly doesn't affect the
caller. Take a look above - note how baz, being defined as "def baz()
with Bar:", can still be called as "baz()" without any exception
checking around it. We're trying to move away from checked exceptions
here, it just happens that using similar syntax is surprisingly
convenient. Sadly we do acknowledge the limitation that the best syntax
for this is similar to what's used for checked exceptions in other
languages. If you have any better syntax suggestions we're open to them tho.

(And yeah, we do agree that *checked exceptions*, not to be confused
with this idea, are a practical failure.)

>
> Barry
>
>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MXLZGFPPTR3O3MN2Q673WYEPR6M6LVS4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 2:04 p.m., Chris Angelico wrote:
> On Fri, Oct 1, 2021 at 1:10 AM Soni L.  wrote:
> >
> >
> >
> > On 2021-09-30 11:23 a.m., Chris Angelico wrote:
> > > > For example this: (real code)
> > > >
> > > > def get_property_values(self, prop):
> > > > try:
> > > > factory = self.get_supported_properties()[prop]
> > > > except KeyError as exc: raise PropertyError from exc
> > > > iterator = factory(self._obj)
> > > > try:
> > > > first = next(iterator)
> > > > except StopIteration: return (x for x in ())
> > > > except abdl.exceptions.ValidationError as exc: raise LookupError
> > > > from exc
> > > > except LookupError as exc: raise RuntimeError from exc  # don't
> > > > accidentally swallow bugs in the iterator
> > > > return itertools.chain([first], iterator)
> > >
> > > There's a lot of exception transformation happening here, and I don't
> > > understand, without context, the full purpose of it all. But the fact
> > > that you're raising LookupError directly seems a tad odd (it's a
> > > parent class for IndexError and KeyError), and it seems like a custom
> > > exception type would solve all of this. I *think* what's happening
> > > here is that you transform ValidationError into LookupError, but only
> > > if it happens on the very first yielded result?? Then how about:
> > >
> > > class PropertyNotFoundError(LookupError): pass
> > >
> > > def get_property_values(self, prop):
> > > try:
> > > factory = self.get_supported_properties()[prop]
> > > except KeyError as exc: raise PropertyError from exc
> > > iterator = factory(self._obj)
> > > try:
> > > yield next(iterator)
> > > except StopIteration: pass
> > > except abdl.exceptions.ValidationError as exc:
> > > raise PropertyNotFoundError from exc
> > > yield from iterator
> > >
> > > If the factory raises some other LookupError, then that's not a
> > > PropertyNotFoundError. And if it *does* raise PropertyNotFoundError,
> > > then that is clearly deliberate, and it's a signal that the property
> > > is, in fact, not found.
> > >
> > > Your code, as currently written (and including the rewrite), is
> > > *still* capable of leaking a faulty exception - just as long as the
> > > factory returns one good value first. So I'm not sure what you
> > > actually gain by this transformation.
> >
> > Okay let us stop you right there. This generator is not a function.
> >
> > Generators don't do anything until iterated.
> >
> > There's a HUGE semantic difference there. You're making assumptions that
> > you could easily invalidate yourself if you paid attention to the
> > original code. Don't do that.
> >
> > try:
> >   iterator = foo.get_property_values(prop)
> > except PropertyError, etc:
> >   pass # handle it
> > for thing in iterator: pass # exceptions in the iteration actually get
> > propagated because they mean something went wrong and the program should
> > terminate. in the real code, this includes spurious ValidationError etc,
> > because those are *never* supposed to happen.
>
> Congratulations, you showed me some extremely complicated code, and
> then got miffed when I messed up something in trying to simplify it.
> Well done, you successfully tripped me up in my attempt to make code
> better.

Fair enough, we went too far there, sorry.

>
> Can we stick to the point, please? You still haven't shown why (a) a
> context manager, nor (b) a custom exception, cannot solve this problem
> far better than this extremely magical syntax.

This syntax is far from magical. We're not sure how the context manager
would work here, but, a custom exception doesn't work, because you still
have the... whatever you'd call this issue:
https://mail.python.org/pipermail/python-ideas/2017-June/046109.html
(one of Steven's links also linked to this and it's fairly relevant.)

The imports issue sadly wouldn't be solved with this feature, but
attributes would:

def __getattr__(self, name) with AttributeError:
  stuff and things
  if the thing:
    return some thing
  raise AttributeError

This also applies anywhere else you'd have something wrapping another
thing and they happen to use the same exceptions in slightly distinct
ways. Yes, it's most noticeable with StopIteration (which got tweaked
with the change 

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

Alright, some ppl asked us to rephrase this, so:

The plan is to take the function syntax:

    def name(args):

and add an optional "with" to it:

    def name(args) with exceptions:

these then get added to the function, similar to e.g. default args. when
an exception is thrown*, the VM then checks these and converts relevant
exceptions into RuntimeError, e.g.:

    def foo():
    raise Bar
    def baz() with Bar:
    foo()
    baz()

would make a RuntimeError, because foo raised a Bar and the VM sees that
Bar is in baz's with.

*except "raise" opcodes SKIP checking these (within the context of the
function), so the following:

    def baz() with Bar:
    raise Bar
    baz()

raises a Bar, not a RuntimeError from a Bar.

You can then document your exceptions as you normally do: "This raises
Bar when so and so are invalid", and then add "with Bar" to your
function. the VM then makes sure that Bar is only raised when so and so
are invalid, but under no other circumstances. e.g. someone
monkeypatched your function to try and raise a Bar outside of the place
it was meant to raise a Bar? it'll raise a RuntimeError instead. this
gives you far more control over what your API exceptions, the exceptions
you documented in your function's documentation, are, and especially
under which conditions they'll actually be raised (and be catchable by
the caller). this is what we mean by exception hygiene.

On 2021-09-29 10:01 p.m., Soni L. wrote:
> So uh, this is a hardly at all fleshed out idea, but one thing we really
> don't like about python is having to do stuff like this so as to not
> swallow exceptions:
>
> def a_potentially_recursive_function(some, args):
>   """
>   Does stuff and things.
>   Raises ExceptionWeCareAbout under so and so conditions.
>   """
>   try: some.user_code() except ExceptionWeCareAbout as exc: raise
> RuntimeError from exc
>   code_we_assume_is_safe()
>   if args.something and some_condition:
>     raise ExceptionWeCareAbout
>
> It'd be nice if there was a way to... make this easier to deal with.
> Perhaps something where, something like this:
>
> def a_potentially_recursive_function(some, args) with ExceptionWeCareAbout:
>   """
>   Does stuff and things.
>   Raises ExceptionWeCareAbout under so and so conditions.
>   """
>   some.user_code()
>   code_we_assume_is_safe()
>   if args.something and some_condition:
>     raise ExceptionWeCareAbout
>
> becomes:
>
> def a_potentially_recursive_function(some, args):
>   """
>   Does stuff and things.
>   Raises ExceptionWeCareAbout under so and so conditions.
>   """
>   try:
>     some.user_code()
>     code_we_assume_is_safe()
>     if args.something and some_condition:
>   let_exception_through = True
>   raise ExceptionWeCareAbout
>   except ExceptionWeCareAbout as exc:
>     if let_exception_through:
>   raise
>     else:
>   raise RuntimeError from exc
>
> (or equivalent)
>
> and something like:
>
> def foo() with Bar:
>   try: baz() except Bar: raise
>
> becomes:
>
> def foo():
>   try:
>     try:
>   baz()
>     except Bar:
>   allow_exception = True
>   raise
>   except Bar as exc:
>     if allow_exception:
>   raise
>     else:
>   raise RuntimeError from exc
>
> (thus allowing explicit exception propagation)
>
> Additionally, note that it would only apply to the raise itself -
> something like `raise may_raise_the_same_error()` would desugar as if:
>
> exception = may_raise_the_same_error()
> allow_exception = True
> raise exception
>
> Obviously this doesn't solve exception handling, doesn't require the
> caller to catch the exceptions, etc etc. It does, however, encourage
> better exception hygiene. Chances are something like this would
> significantly reduce the amount of swallowed exceptions, if it gets
> widely adopted. We know something like this would've saved us a lot of
> trouble, so we're sharing the idea in the hopes it can, in the future,
> help others.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KMSLDPNK3UVP42YMWUWFR3RK6JXFZFGK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 11:23 a.m., Chris Angelico wrote:
> > For example this: (real code)
> >
> > def get_property_values(self, prop):
> > try:
> > factory = self.get_supported_properties()[prop]
> > except KeyError as exc: raise PropertyError from exc
> > iterator = factory(self._obj)
> > try:
> > first = next(iterator)
> > except StopIteration: return (x for x in ())
> > except abdl.exceptions.ValidationError as exc: raise LookupError
> > from exc
> > except LookupError as exc: raise RuntimeError from exc  # don't
> > accidentally swallow bugs in the iterator
> > return itertools.chain([first], iterator)
>
> There's a lot of exception transformation happening here, and I don't
> understand, without context, the full purpose of it all. But the fact
> that you're raising LookupError directly seems a tad odd (it's a
> parent class for IndexError and KeyError), and it seems like a custom
> exception type would solve all of this. I *think* what's happening
> here is that you transform ValidationError into LookupError, but only
> if it happens on the very first yielded result?? Then how about:
>
> class PropertyNotFoundError(LookupError): pass
>
> def get_property_values(self, prop):
> try:
> factory = self.get_supported_properties()[prop]
> except KeyError as exc: raise PropertyError from exc
> iterator = factory(self._obj)
> try:
> yield next(iterator)
> except StopIteration: pass
> except abdl.exceptions.ValidationError as exc:
> raise PropertyNotFoundError from exc
> yield from iterator
>
> If the factory raises some other LookupError, then that's not a
> PropertyNotFoundError. And if it *does* raise PropertyNotFoundError,
> then that is clearly deliberate, and it's a signal that the property
> is, in fact, not found.
>
> Your code, as currently written (and including the rewrite), is
> *still* capable of leaking a faulty exception - just as long as the
> factory returns one good value first. So I'm not sure what you
> actually gain by this transformation.

Okay let us stop you right there. This generator is not a function.

Generators don't do anything until iterated.

There's a HUGE semantic difference there. You're making assumptions that
you could easily invalidate yourself if you paid attention to the
original code. Don't do that.

try:
  iterator = foo.get_property_values(prop)
except PropertyError, etc:
  pass # handle it
for thing in iterator: pass # exceptions in the iteration actually get
propagated because they mean something went wrong and the program should
terminate. in the real code, this includes spurious ValidationError etc,
because those are *never* supposed to happen.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CPXRLKFULB44IK57SKCYV74AMY3GXDYE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 10:08 a.m., Chris Angelico wrote:
> On Thu, Sep 30, 2021 at 8:43 PM Soni L.  wrote:
> > You misnderstand exception hygiene. It isn't about "do the least stuff
> > in try blocks", but about "don't shadow unrelated exceptions into your
> > public API".
> >
> > For example, generators don't allow you to manually raise StopIteration
> > anymore:
> >
> > >>> next((next(iter([])) for x in [1, 2, 3]))
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > StopIteration
> >
> > The above exception was the direct cause of the following exception:
> >
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > RuntimeError: generator raised StopIteration
> >
> > This is a (limited) form of exception hygiene. Can we generalize it? Can
> > we do better about it? This effectively means all generators *are*
> > wrapped in a try/except, so your point about "too much stuff inside a
> > try block" goes directly against accepted practice and even existing
> > python features as they're implemented.
>
> The reason for this is that StopException is nothing more than an
> *implementation detail* of generators. Look at this code: where is
> StopException?
>
> def gen():
> yield 5
> x = (yield 7)
> yield 9
> if x: return 11
> yield 1
> return 3
>
> The code doesn't raise StopException other than because that's the way
> that iterables are implemented. As a function, it simply does its
> work, with yield points and the ability to return a value. That's why
> a leaking StopException can and should be turned into RuntimeError.
>
> But what you're talking about doesn't have this clear distinction,
> other than in *your own definitions*. You have deemed that, in some
> areas, a certain exception should be turned into a RuntimeError; but
> in other areas, it shouldn't. To me, that sounds like a job for a
> context manager, not a function-level declaration.

But generators *are* iterators. By definition. In fact this had to be a
breaking change *because there was code in the wild that relied on it*!

Imagine if that code could be changed to be:

def gen() with StopIteration:
  try: yield next(foo)
  except StopIteration: raise

and have the StopIteration propagate as a StopIteration instead of
RuntimeError! (altho supporting this *specific* use-case would probably
be painful given that this is mostly a purely syntactic transformation.)

>
> > > My comments asking how the compiler is supposed to know which part of
> > > the code needs to be guarded with a "re-raise the exception" flag still
> > > apply, regardless of whether I have misunderstood your API or not.
> > >
> > > Your syntax has:
> > >
> > > def a_potentially_recursive_function(some, args) with 
> > > ExceptionWeCareAbout:
> > > some.user_code()
> > > code_we_assume_is_safe()
> > > if args.something and some_condition:
> > > raise ExceptionWeCareAbout  # Line (A)
> > >
> > > How does the compiler know that *only* ExceptionWeCareAbout originating
> > > in Line (A) should be re-raised, and any other location turned into
> > > RuntimeError?
> >
> > Same way Rust decides whether to propagate or unwrap a Result: you
> > *must* tell the compiler.
>
> Please elaborate. We can already write this:
>
> def foo():
> with fail_on_exception(ExceptionWeCareAbout):
> some.user_code()
> if some_condition:
> raise ExceptionWeCareAbout
>
> Does that count as telling the compiler? If not, what is it you're
> trying to do, and how is the compiler supposed to know which ones to
> permit and which to wrap in RuntimeError?

With a source transformation, really. that is:

def foo() with exceptions:
  something
  raise ...

always transforms into:

def foo():
  set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = False
  try:
    something
    set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = True
    raise ...
  except exceptions as exc:
    if set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError:
  raise
    else:
  raise RuntimeError from exc

that is: the "with exceptions" becomes "except exceptions", and every
"raise" gains an
"set_to_True_to_pass_through_instead_of_wrapping_in_RuntimeError = True"
immediately before it (mostly - kinda glossing over the whole "the
expression of the raise doesn't get to, itself, raise its own
exceptions", but anyway).

It gets clearer/etc if you have a more comple

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-30 4:15 a.m., Steven D'Aprano wrote:
> On Thu, Sep 30, 2021 at 12:03:37AM -0300, Soni L. wrote:
>
> > > Only some.user_code is guarded by the try block. If it turns out that 
> > > code_we_assume_is_safe is not actually safe, and fails with an 
> > > exception, it won't be caught by the try block and you will know about 
> > > it.
> > 
> > Except no, because ExceptionWeCareAbout is part of the public API.
> [...]
>
> You have not convinced me that I have misunderstood the proposal. As I 
> said, some better, runnable code might help. But for the sake of the 
> argument, suppose I have misunderstood and your analysis is correct..
>
> You have just demonstrated that your proposed syntax hurts 
> readability. In your original function, it is easy to recognise 
> potentially poor exception hygiene at a glance:
>
> "Too much stuff inside a try block == potential bad hygiene"
>
> With your proposed syntactic sugar, there is no visible try block, and 
> it is exceedingly unclear which parts of the function body are protected 
> by an implicit try block, and which parts will have the exception caught 
> and turned into RuntimeError, and which parts will have the exception 
> caught and re-raised.

You misnderstand exception hygiene. It isn't about "do the least stuff
in try blocks", but about "don't shadow unrelated exceptions into your
public API".

For example, generators don't allow you to manually raise StopIteration
anymore:

>>> next((next(iter([])) for x in [1, 2, 3]))
Traceback (most recent call last):
  File "", line 1, in 
StopIteration

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "", line 1, in 
RuntimeError: generator raised StopIteration

This is a (limited) form of exception hygiene. Can we generalize it? Can
we do better about it? This effectively means all generators *are*
wrapped in a try/except, so your point about "too much stuff inside a
try block" goes directly against accepted practice and even existing
python features as they're implemented.

>
> > [no comments on the rest of your points because they're all based on
> > this core misunderstanding.]
>
> The rest of my post is more important.
>
> My comments asking how the compiler is supposed to know which part of 
> the code needs to be guarded with a "re-raise the exception" flag still 
> apply, regardless of whether I have misunderstood your API or not.
>
> Your syntax has:
>
> def a_potentially_recursive_function(some, args) with 
> ExceptionWeCareAbout:
> some.user_code()
>     code_we_assume_is_safe()
> if args.something and some_condition:
> raise ExceptionWeCareAbout  # Line (A)
>
> How does the compiler know that *only* ExceptionWeCareAbout originating 
> in Line (A) should be re-raised, and any other location turned into 
> RuntimeError?

Same way Rust decides whether to propagate or unwrap a Result: you
*must* tell the compiler.

>
> What if I factor out those last two lines and make it:
>
> def a_potentially_recursive_function(some, args) with 
> ExceptionWeCareAbout:
> some.user_code()
>     code_we_assume_is_safe()
> check_condition_or_raise(args.something, some_condition)
>
> How does the compiler decide to re-raise exceptions originating in the 
> last line but not the first two?

In this case, it explicitly doesn't. You explicitly told it the last
line doesn't raise any exceptions that contribute to your API's
exception surface.

You *must* use try: check_condition_or_raise(args.something,
some_condition) except ExceptionWeCareAbout: raise

(Verbosity can be improved if this feature gets widely used, but it's
beside the point.)

>
> What if I use a pre-prepared exception instance, or an alias, or both?
>
> BadThing = ExceptionWeCareAbout
>
> ERROR = BadThing("a thing happened")
>
> def a_potentially_recursive_function(some, args) with 
> ExceptionWeCareAbout:
> some.user_code()
>     code_we_assume_is_safe()
> if args.something and some_condition:
> raise ERROR
>
> Your proposal doesn't make it clear how the compiler decides which parts 
> of the body should allow the exception through and which should 
> re-raise.
>
>
>

This works fine because any explicit raise will always poke through the
generated try/except.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K27TNY6H5SHGNIBV3GC4PRAH2ANLO6ZL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-29 11:46 p.m., Steven D'Aprano wrote:
> In Soni's original code snippet, there is a clear separation of code 
> that is inside the try block from code that is outside the try block:
>
> > def a_potentially_recursive_function(some, args):
> >   try: 
> >   some.user_code() 
> >   except ExceptionWeCareAbout as exc:
> >   raise RuntimeError from exc
> >   code_we_assume_is_safe()  # and more code following...
>
> This is good exception hygiene.
>
> Only some.user_code is guarded by the try block. If it turns out that 
> code_we_assume_is_safe is not actually safe, and fails with an 
> exception, it won't be caught by the try block and you will know about 
> it.

Except no, because ExceptionWeCareAbout is part of the public API.

If it's not actually safe, it might raise an ExceptionWeCareAbout...

... which gets promptly swallowed by the API consumer, who thinks it was
intentional. But any ExceptionWeCareAbout in user code, because it's
explicitly guarded against/wrapped, doesn't get silently swallowed the
same way.

[no comments on the rest of your points because they're all based on
this core misunderstanding.]

>
> In your re-written syntactic sugar, you have:
>
> # new syntax
> > def a_potentially_recursive_function(some, args) with ExceptionWeCareAbout:
> >   some.user_code()
> >   code_we_assume_is_safe()  # and more code following...
>
> which becomes:
>
> > def a_potentially_recursive_function(some, args):
> >   try:
> >     some.user_code()
> >     code_we_assume_is_safe()  # and more code following...
> >   except ExceptionWeCareAbout as exc:
>
> which is the opposite of good exception hygiene. Too much code, 
> including the wrong code, is guarded by the try block, which means that 
> the compiler has to *guess your meaning* and set a flag to decide 
> whether to re-raise the exception or run the except block.
>
> Its not clear how the compiler guesses that. Is it only because you have 
> an explicit `raise ExceptionWeCareAbout` in the code? What if the 
> exception is not explicit?
>
> # explicitly raising ExceptionWeCareAbout
> raise ExceptionWeCareAbout
>
> # not explicit
> raise some_exception_object_we_prepared_earlier
>
> raise prepare_exception(*args)  # returns an ExceptionWeCareAbout instance
>
> verify_or_raise(condition)  # raises ExceptionWeCareAbout
>
>
>
> > Obviously this doesn't solve exception handling, doesn't require the
> > caller to catch the exceptions, etc etc.
>
> So that's two points against it.
>
> > It does, however, encourage better exception hygiene.
>
> Except it doesn't, it makes it worse. So that's three points against it.
>
>
> > Chances are something like this would
> > significantly reduce the amount of swallowed exceptions, if it gets
> > widely adopted.
>
> Or based on your examples, increase the number of swallowed exceptions, 
> and be much harder to refactor code safely.
>
>
>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ELHOHBNVPVKP5MS44EVUCQV2QXDDTMME/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Soni L.]

On 2021-09-29 10:09 p.m., Chris Angelico wrote:
> On Thu, Sep 30, 2021 at 11:03 AM Soni L.  wrote:
> >
> > So uh, this is a hardly at all fleshed out idea, but one thing we really
> > don't like about python is having to do stuff like this so as to not
> > swallow exceptions:
> >
> > def a_potentially_recursive_function(some, args):
> >   """
> >   Does stuff and things.
> >   Raises ExceptionWeCareAbout under so and so conditions.
> >   """
> >   try: some.user_code() except ExceptionWeCareAbout as exc: raise
> > RuntimeError from exc
> >   code_we_assume_is_safe()
> >   if args.something and some_condition:
> > raise ExceptionWeCareAbout
> >
> > It'd be nice if there was a way to... make this easier to deal with.
> > Perhaps something where, something like this:
> >
> > def a_potentially_recursive_function(some, args) with ExceptionWeCareAbout:
> >   """
> >   Does stuff and things.
> >   Raises ExceptionWeCareAbout under so and so conditions.
> >   """
> >   some.user_code()
> >   code_we_assume_is_safe()
> >   if args.something and some_condition:
> > raise ExceptionWeCareAbout
> >
> > becomes:
> >
> > def a_potentially_recursive_function(some, args):
> >   """
> >   Does stuff and things.
> >   Raises ExceptionWeCareAbout under so and so conditions.
> >   """
> >   try:
> > some.user_code()
> > code_we_assume_is_safe()
> > if args.something and some_condition:
> >   let_exception_through = True
> >   raise ExceptionWeCareAbout
> >   except ExceptionWeCareAbout as exc:
> > if let_exception_through:
> >   raise
> > else:
> >   raise RuntimeError from exc
> >
> > (or equivalent)
> >
> > and something like:
> >
> > def foo() with Bar:
> >   try: baz() except Bar: raise
> >
> > becomes:
> >
> > def foo():
> >   try:
> > try:
> >   baz()
> > except Bar:
> >   allow_exception = True
> >   raise
> >   except Bar as exc:
> > if allow_exception:
> >   raise
> > else:
> >   raise RuntimeError from exc
> >
> > (thus allowing explicit exception propagation)
> >
> > Additionally, note that it would only apply to the raise itself -
> > something like `raise may_raise_the_same_error()` would desugar as if:
> >
> > exception = may_raise_the_same_error()
> > allow_exception = True
> > raise exception
> >
> > Obviously this doesn't solve exception handling, doesn't require the
> > caller to catch the exceptions, etc etc. It does, however, encourage
> > better exception hygiene. Chances are something like this would
> > significantly reduce the amount of swallowed exceptions, if it gets
> > widely adopted. We know something like this would've saved us a lot of
> > trouble, so we're sharing the idea in the hopes it can, in the future,
> > help others.
>
> Can you give a realistic example of how this works, and how you could
> accidentally leak the exact same exception that you'd be intentionally
> raising? It looks to me like you possibly should be splitting the
> function into the recursive part and the setup part, where only the
> setup is capable of raising that exception.
>
> I've seen a LOT of bad Python code caused by an assumption that
> "recursion" must always mean "calling the external API". A much better
> model, in a lot of cases, is:
>
> def _somefunc_recursive(x, y, z):
> ...
> _somefunc_recursive(x, y + 1, z - 1)
>
> def somefunc(x, y, z=4):
> ...
> _somefunc_recursive(x, y, z)
> return someresult

    def get_property_value(self, prop):
    """Returns the value associated with the given property.

    If duplicated, an earlier value should override a later value.

    Args:
    prop (DataProperty): The property.

    Returns:
    The value associated with the given property.

    Raises:
    PropertyError: If the property is not supported by this data
    source.
    LookupError: If the property is supported, but isn't available.
    ValueError: If the property doesn't have exactly one value.
    """
    iterator = self.get_property_values(prop)
    try:
    # note: unpacking
    ret, = iterator
    except LookupError as exc: raise RuntimeError from exc  # don't
accidentally swallow bugs in the iter

[Python-ideas] Better exception handling hygiene

[Soni L.]

So uh, this is a hardly at all fleshed out idea, but one thing we really
don't like about python is having to do stuff like this so as to not
swallow exceptions:

def a_potentially_recursive_function(some, args):
  """
  Does stuff and things.
  Raises ExceptionWeCareAbout under so and so conditions.
  """
  try: some.user_code() except ExceptionWeCareAbout as exc: raise
RuntimeError from exc
  code_we_assume_is_safe()
  if args.something and some_condition:
    raise ExceptionWeCareAbout

It'd be nice if there was a way to... make this easier to deal with.
Perhaps something where, something like this:

def a_potentially_recursive_function(some, args) with ExceptionWeCareAbout:
  """
  Does stuff and things.
  Raises ExceptionWeCareAbout under so and so conditions.
  """
  some.user_code()
  code_we_assume_is_safe()
  if args.something and some_condition:
    raise ExceptionWeCareAbout

becomes:

def a_potentially_recursive_function(some, args):
  """
  Does stuff and things.
  Raises ExceptionWeCareAbout under so and so conditions.
  """
  try:
    some.user_code()
    code_we_assume_is_safe()
    if args.something and some_condition:
  let_exception_through = True
  raise ExceptionWeCareAbout
  except ExceptionWeCareAbout as exc:
    if let_exception_through:
  raise
    else:
  raise RuntimeError from exc

(or equivalent)

and something like:

def foo() with Bar:
  try: baz() except Bar: raise

becomes:

def foo():
  try:
    try:
  baz()
    except Bar:
  allow_exception = True
  raise
  except Bar as exc:
    if allow_exception:
  raise
    else:
  raise RuntimeError from exc

(thus allowing explicit exception propagation)

Additionally, note that it would only apply to the raise itself -
something like `raise may_raise_the_same_error()` would desugar as if:

exception = may_raise_the_same_error()
allow_exception = True
raise exception

Obviously this doesn't solve exception handling, doesn't require the
caller to catch the exceptions, etc etc. It does, however, encourage
better exception hygiene. Chances are something like this would
significantly reduce the amount of swallowed exceptions, if it gets
widely adopted. We know something like this would've saved us a lot of
trouble, so we're sharing the idea in the hopes it can, in the future,
help others.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TGHIGKWEHKS4MEDMDFMHS3FBAHENLTUB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-23 10:21 a.m., Steven D'Aprano wrote:
> > What about other functions implemented in C? If I write a C module
> > that calls PyObject_GetAttr, does it behave as if dot notation were
> > used in the module that called me, or does it use my module's
> > extension methods?
>
> That depends. If you write a C module that calls PyObject_GetAttr right 
> now, is that *exactly* the same as dot notation in pure-Python code?
>
> The documentation is terse:
>
> https://docs.python.org/3.8/c-api/object.html#c.PyObject_GetAttr
>
> but if it is correct that it is precisely equivalent to dot syntax, then 
> the same rules will apply. Has the current module opted in? If so, then 
> does the class have an extension method of the requested name?
>
> Same applies to code objects evaluated without a function, or whatever 
> other exotic corner cases you think of. Whatever you think of, the 
> answer will always be the same:
>
> - if the execution context is a module that has opted to use 
>   extension methods, then attribute access will see extension methods;
>
> - if not, then it won't.
>
> If you think of a scenario where you are executing code where there is 
> no module scope at all, and all global lookups fail, then "no module" 
> cannot opt in to use extension methods and so the code won't see them.
>
> If you can think of a scenario where you are executing code where there 
> are multiple module scopes that fight for supremacy using their two 
> weapons of fear, surprise and a fanatical devotion to the Pope, then the 
> winner will determine the result.
>
> *wink*
>
>
>
>

But if getattr is part of the builtins module, written in C, and the
builtins module is calling PyObject_GetAttr, and PyObject_GetAttr is
exactly the same as the dot notation...

Then getattr is exactly the same as the dot notation, **in the builtins
module**! The builtins module doesn't use any extension methods, as it
is written in C. As such getattr(foo, "bar") MUST NOT produce the same
result as foo.bar if extension methods are at play!

(You're still missing the point of extension methods. Do check out our
other reply.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XARO3NEUWZ7OVS47Y36KPPSDLKQWAUQZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-23 5:21 a.m., Steven D'Aprano wrote:
> On Tue, Jun 22, 2021 at 08:44:56AM -0300, Soni L. wrote:
>
> > Oh this is a long one.
> > 
> > Hypothetically, let's say you have a proxy object:
> > 
> > class Foo:
> >   def __getattribute__(self, thing):
> >     return getattr(super().__getattribute__(self, "proxied"), thing)
> > 
> > Should this really include extension methods into it by default?
>
> By default? Absolutely not. Extension methods are opt-in.
>
>
> > This is clearly wrong.
>
> What is clearly wrong? Your question? A "yes" answer? A "no" answer? 
> Your proxy object?

We're saying including local extension methods into the proxy object's
attribute lookup is wrong.

> Soni, and Chris, you seem to be responding as if extension methods are 
> clearly, obviously and self-evidently a stupid idea. Let me remind you 
> that at least ten languages (C#, Java, Typescript, Oxygene, Ruby, 
> Smalltalk, Kotlin, Dart, VB.NET and Swift) support it. Whatever the pros 
> and cons of the technique, it is not self-evidently wrong or stupid.

That's funny because we (Soni) have been arguing about and pushing for a
specific implementation of them. We're not opposed to them, quite the
opposite we even have an implementation we'd like to see, altho we don't
really see much of a use for them ourselves.

>
> > The local override for the LOAD_ATTR opcode
> > should NOT apply to proxy methods except where explicitly requested.
>
> Why not?
>
> Let me ask you this:
>
> - should proxy objects really include `__slots__` by default?
>
> - should they really include dynamic attributes generated by 
>   `__getattr__`?
>
> - should they include attributes in the inheritence hierarchy?
>
> - why should extension methods be any different?
>
>
> Let's step back from extension methods and consider a similar technique, 
> the dreaded monkey-patch. If I extend a class by monkey-patching it with 
> a new method:
>
> import library
> library.Klass.method = patch_method
>
> would you expect that (by default) the patched method are invisible to 
> proxies of Klass? Would you expect there to be a way to "opt-out" of 
> proxying that method?
>
> I hope that your answers are "No, and no", because if either answer is 
> "yes", you will be very disappointed in Python.
>
> Why should extension methods be different from any other method? Let's 
> go through the list of methods which are all treated the same:

Why shouldn't extension methods be different from monkey-patching? If
they were to be the same, why call them something different?

>
> - methods defined on the class;
> - methods defined on a superclass or mixin;
> - methods added onto the instance;
> - methods created dynamically by `__getattr__`.
>
> (Did I miss any?)
>
> And the list of those which are handled differently, with ways to 
> opt-out of seeing them:
>
> - ... um... er...
>
> Have I missed any?
>
>
> > The point is that the caller using your proxy object should opt-in to
> > the extension methods, rather than break with no way to opt-out of them.
>
> You opt-out by not opting in.
>
>
> > Your extension methods shouldn't propagate to proxy objects.
>
> Fundamentally, your proxy object is just doing attribute lookups on 
> another object. If you have a proxy to an instance `obj`, there should 
> be no difference in behaviour between extension methods and regular 
> methods. If `obj.method` succeeds, so should `proxy.method`, because 
> that's what proxies do.
>
> The origin of obj.method should not make any difference. I'm sorry to 
> have to keep harping on this, but it doesn't matter to the proxy whether 
> the method exists in the instance `__dict__`, or the class `__dict__`, 
> or `__slots__`, or a superclass, or is dynamically generated by 
> `__getattr__`. A method is a method.
>
> Extension methods are methods.

Extension methods are functions and are scoped like other functions.

>
>
> > To go even further, should all your class definitions that happen to
> > extend a class with in-scope extension methods automatically gain those
> > extension methods? Because with actual extension methods, that doesn't
> > happen.
>
> We might want to follow the state of the art here, assuming there was 
> consensus in other languages about inheriting extension methods. But I 
> would expect this behaviour:
>
>
> # --- extensions.py library ---
>
> @extends(list)
> def flatten(self):
> ...
>
>
> # --- module A.py ---
>
> uses extensions  # opt-in to use the extension method
&g

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 8:11 p.m., Chris Angelico wrote:
> On Wed, Jun 23, 2021 at 9:06 AM Soni L.  wrote:
> > On 2021-06-22 7:38 p.m., Chris Angelico wrote:
> > > Have you actually tried designing this into a larger project to see
> > > what problems you run into, or is this something you've only
> > > considered at this trivial level?
> >
> > 1. It's opt-in.
> > 2. It's designed to be used by a hypothetical extension methods module,
> > but without imposing any design constraints on such module. It could
> > return a named function every time a given name is looked up (a la "bind
> > the first argument" operator), or do dynamic dispatch based on types or
> > ABCs (a la proper extension methods).
> >
> > In practice, you don't def your own __dot__, but rather use someone
> > else's "__dot__ builder". If you don't wanna deal with it, just don't
> > use __dot__.
> >
> > It's also useful for the occasional domain-specific language.
> >
>
> What I'm hearing from you is: "No". Lots of words to say it, but, the
> answer to my question seems to be no.

We have never felt a need for extension methods in python, no.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/BZ7BR5ACN2VXJQA6LVV6FNSOCKFVQTWT/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q57V3QDZ6O6WZZXYQDG4E3RRFSPZO62L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 7:38 p.m., Chris Angelico wrote:
> On Wed, Jun 23, 2021 at 8:30 AM Soni L.  wrote:
> >
> >
> >
> > On 2021-06-22 5:54 p.m., Chris Angelico wrote:
> > > On Wed, Jun 23, 2021 at 6:41 AM Soni L.  wrote:
> > > > It would have local scope, similar to uh... locals. Y'know how locals
> > > > are just sugar for locals()['foo'] and stuff? Yeah.
> > >
> > > Not really, no, they're not. :) The dictionary returned by locals()
> > > isn't actually an implementation detail of local name lookups.
> >
> > It's... part of the language. Not an implementation detail. The
> > dictionary returned by locals() is an inherent part of local name
> > lookups, isn't it?
>
> No, it's not. Most definitely not.
>
> https://docs.python.org/3/library/functions.html#locals

Ohh. Fair enough, sorry.

> > > Have you put any thought into how you would deal with the problem of
> > > recursive __dot__ calls?
> >
> > Let it recurse!
> >
> > Globals and locals don't go through __dot__, so you can just... use
> > them. In particular, you can always use getattr(), and probably should.
> > Or even set __dot__ to getattr inside it, like so:
> >
> > def __dot__(left, right):
> >   __dot__ = getattr
> >   foo.bar # same as getattr(foo, "bar") because we set (local) __dot__
> > to getattr above
>
> I can't actually pin down what I'm averse to here, but it gives me a
> really REALLY bad feeling. You're expecting every attribute lookup to
> now look for a local or global name __dot__ (or, presumably, a
> nonlocal, class, or builtin), and do whatever that does. That seems
> like a really effective foot-gun.
>
> Have you actually tried designing this into a larger project to see
> what problems you run into, or is this something you've only
> considered at this trivial level?

1. It's opt-in.
2. It's designed to be used by a hypothetical extension methods module,
but without imposing any design constraints on such module. It could
return a named function every time a given name is looked up (a la "bind
the first argument" operator), or do dynamic dispatch based on types or
ABCs (a la proper extension methods).

In practice, you don't def your own __dot__, but rather use someone
else's "__dot__ builder". If you don't wanna deal with it, just don't
use __dot__.

It's also useful for the occasional domain-specific language.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/YOPTCIYV4GSW3N7EA7KPLJBKNVSNANXZ/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LQW3GLWRGPOZDOAS7LGUL4WLP6JOUKUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 5:34 p.m., Brendan Barnwell wrote:
> On 2021-06-22 13:09, Soni L. wrote:
>> Think about it like this, extension methods give you the ability to make
>> imported functions that look like this:
>>
>> foo(bar, baz)
>>
>> look like this instead:
>>
>> bar.foo(baz)
>>
>> That's all there is to them. They're just a lie to change how you
>> read/write the code. Some languages have an whole operator that has a
>> similar function, where something like bar->foo(baz) is sugar for
>> foo(bar, baz). The OP doesn't specify any particular mechanism for
>> extension methods, so e.g. making the dot operator be implemented by a
>> local function in the module, which delegates to the current attribute
>> lookup mechanism by default, would be perfectly acceptable. It's like
>> deprecating the existing dot operator and introducing a completely
>> different one that has nothing to do with attribute lookup!
>
> Okay, if that's the case, then I just think it's a bad idea.  :-)
>
> We already have a definition for what bar.foo does, and it's
> totally under the control of the bar object (via the
> __getattr__/__getattribute__ mechanism).  The idea that other things
> would be able to hook in there does not appeal to me at all.
>
> I don't really understand why you would want such a thing, to be
> honest.  I feel it would make code way more difficult to reason about,
> as it would break locality constraints every which way.  Now every
> time you see`bar.foo` you would have to think about all kinds of other
> modules that may be hooking in and adding their own complications.
> What's the point?
>
> Mostly the whole benefit of the dot notation is that it specifies
> a locally constrained relationship between the object and the
> attribute: you know that bar.foo does what bar decides, and no one
> else gets any say (unless bar asks for their opinion, e.g. by
> consulting global variables or whatever).  If we want to write
> foo(bar, baz). . . well, we can just do that!  What you're describing
> would just make existing attribute usages harder to understand while
> only "adding" something we can already do quite straightforwardly.
>
> Imagine a similar proposal for other syntax.  Suppose that in any
> module I could define a function called operator_add and then other
> modules could "import" this "extension" so that every use of the +
> operator would somehow hook into this operator_add function.  So now
> every time you do 2 + 2 you might be invoking some extension behavior.
> In my view that is unambiguously a road to madness, and as far as I
> can tell the extension mechanism you're proposing is equally ill-advised.
>

Imagine if Python didn't have an + operator, but instead an + *infix
function*.

Thus, every module would automatically include the global

def infix +(left, right):
  ...

And indeed, you could say we already have this. Except currently you
can't define your own local infix +. But what if you *could*?

What if you could just,

# file1.py
def infix +(left, right):
  return left << right
x = 4 + 4

# file2.py
def infix +(left, right):
  return left ** right
x = 4 + 4

# file3.py
import file1
import file2
print(file1.x) # 64
print(file2.x) # 256
print(4 + 4) # 8

How does this break locality?

Same idea with the dot operator, really.

(Some languages don't have operators, but only functions. They let you
do just this.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QBQCSJELOHSPOG24X2LMSYBSKCGQRVYP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 5:54 p.m., Chris Angelico wrote:
> On Wed, Jun 23, 2021 at 6:41 AM Soni L.  wrote:
> >
> >
> >
> > On 2021-06-22 5:23 p.m., Chris Angelico wrote:
> > > On Wed, Jun 23, 2021 at 6:13 AM Soni L.  wrote:
> > > > Think about it like this, extension methods give you the ability to make
> > > > imported functions that look like this:
> > > >
> > > > foo(bar, baz)
> > > >
> > > > look like this instead:
> > > >
> > > > bar.foo(baz)
> > > >
> > > > That's all there is to them. They're just a lie to change how you
> > > > read/write the code. Some languages have an whole operator that has a
> > > > similar function, where something like bar->foo(baz) is sugar for
> > > > foo(bar, baz). The OP doesn't specify any particular mechanism for
> > > > extension methods, so e.g. making the dot operator be implemented by a
> > > > local function in the module, which delegates to the current attribute
> > > > lookup mechanism by default, would be perfectly acceptable. It's like
> > > > deprecating the existing dot operator and introducing a completely
> > > > different one that has nothing to do with attribute lookup!
> > >
> > > uh... I'm lost. Are you saying that that's a good thing? You *want* to
> > > replace the existing dot operator with one that has nothing to do with
> > > attribute lookup?? I don't get it.
> >
> > Sure! As long as the new one can call getattr!
> >
> > Let's say the new dot operator looks like this:
> >
> > # file1.py
> > def __dot__(left, right):
> >   print(left)
> >   print(right)
> >   ...
> > foo = []
> > foo.bar
> >
> > Now, this would actually print the list [] and the string "bar".
> >
> > Then you can just use getattr to get attribute lookup behaviour out of it!
> >
> > def __dot__(left, right):
> >   return getattr(left, right)
> > foo = []
> > foo.bar
> >
> > It would have local scope, similar to uh... locals. Y'know how locals
> > are just sugar for locals()['foo'] and stuff? Yeah.
>
> Not really, no, they're not. :) The dictionary returned by locals()
> isn't actually an implementation detail of local name lookups.

It's... part of the language. Not an implementation detail. The
dictionary returned by locals() is an inherent part of local name
lookups, isn't it?

> Have you put any thought into how you would deal with the problem of
> recursive __dot__ calls?

Let it recurse!

Globals and locals don't go through __dot__, so you can just... use
them. In particular, you can always use getattr(), and probably should.
Or even set __dot__ to getattr inside it, like so:

def __dot__(left, right):
  __dot__ = getattr
  foo.bar # same as getattr(foo, "bar") because we set (local) __dot__
to getattr above

In languages with lexical scoping (instead of block scoping), the
compiler doesn't see things that haven't yet been declared. In those
languages, such a __dot__ function would actually inherit the global
__dot__ rather than recursing. But as you can see from the above
example, it's really not a big deal.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/NEI23TBWA3QKRGYIWTBJV3SK5O6MWXZC/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X7J5SNYPDJTCC5LKBG4SHA5YEQAU5AMA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 5:23 p.m., Chris Angelico wrote:
> On Wed, Jun 23, 2021 at 6:13 AM Soni L.  wrote:
> > Think about it like this, extension methods give you the ability to make
> > imported functions that look like this:
> >
> > foo(bar, baz)
> >
> > look like this instead:
> >
> > bar.foo(baz)
> >
> > That's all there is to them. They're just a lie to change how you
> > read/write the code. Some languages have an whole operator that has a
> > similar function, where something like bar->foo(baz) is sugar for
> > foo(bar, baz). The OP doesn't specify any particular mechanism for
> > extension methods, so e.g. making the dot operator be implemented by a
> > local function in the module, which delegates to the current attribute
> > lookup mechanism by default, would be perfectly acceptable. It's like
> > deprecating the existing dot operator and introducing a completely
> > different one that has nothing to do with attribute lookup!
>
> uh... I'm lost. Are you saying that that's a good thing? You *want* to
> replace the existing dot operator with one that has nothing to do with
> attribute lookup?? I don't get it.

Sure! As long as the new one can call getattr!

Let's say the new dot operator looks like this:

# file1.py
def __dot__(left, right):
  print(left)
  print(right)
  ...
foo = []
foo.bar

Now, this would actually print the list [] and the string "bar".

Then you can just use getattr to get attribute lookup behaviour out of it!

def __dot__(left, right):
  return getattr(left, right)
foo = []
foo.bar

It would have local scope, similar to uh... locals. Y'know how locals
are just sugar for locals()['foo'] and stuff? Yeah.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/XDY3VAYX3FXJI7ZVT3AYAFA7PGQMBWAA/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RGLMGLHAHLJVN3G5WEQSEKURUY7OP2UD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 3:43 p.m., Brendan Barnwell wrote:
> On 2021-06-22 05:14, Chris Angelico wrote:
>> Fair point. However, I've worked with a good number of languages that
>> have some notion of object methods, and generally, an object has or
>> doesn't have a method based on what the object*is*, not on who's
>> asking.
>
> I agree, and this is the aspect of the proposal that most confuses
> me. I still can't understand concretely what is being proposed,
> though, so I'm not sure I even understand it.  Can someone clarify? 
> Suppose I have this
>
> *
> ### file1.py
> @extend(list)
> def len2(self):
> return len(self)**2
>
> ### file2.py
> # or whatever I do to say "I want to use extensions to list defined in
> file1"
> from file1 extend list
>
> def coolness(some_list):
> return some_list.len2() + 1
>
> my_list = [1, 2, 3]
> print("My list len2:", my_list.len2())
> print("My list coolness:", coolness(my_list))
>
> ### file3.py
> import file2
>
> other_list = [1, 2, 3, 4]
> print("Other list len2:", other_list.len2())
> print("other list coolness:", file2.coolness(other_list))
> print("My list len2 from outside:", file2.my_list.len2())
> print("My list coolness from outside:", file2.coolness(file2.my_list))
> *
>
> What exactly is supposed to happen here if I run file3?  file2
> declares use of file1's extensions.  file2 does not.  But file3 uses a
> function in file2 that makes use of such extensions.  Who sees the
> extension?
>

NameError, value, NameError, value, respectively.

> The list object my_list in file2 is the same object accessed as
> file2.my_list in file3.  Likewise coolness and file2.coolness.  It is
> going to be super confusing if calling the same function object with
> the same list object argument gives different results depending on
> which file you're in.  Likewise it's going to be confusing if the same
> list object sometimes has a .len2 method and sometimes doesn't.

It isn't the list object that has the extension method.

>
> But if it doesn't work that way, then it would seem to mean either
> every module sees the extensions (even if they didn't opt in), or else
> my_list in file2 is not the same object as file2.my_list in file3. 
> And that would be even worse.  (In this example it may seem okay
> because you can ask why I would call len2 from file3 if I didn't want
> to use it. But what if the extension is an override of an existing
> method?  Is that not allowed?)
>
> In addition, if there is a difference between my_list and
> other_list, then that apparently means that the syntax for lists now
> does something different in the two files.  This is maybe the most
> reasonable approach, since it's at least remotely reminiscent of a
> __future__ import, which changes syntactic behavior.  But what exactly
> is the difference between the two objects here?  Are both objects
> lists?  If they are, then how can they have different methods?  If
> they're not, then what are they?
>
> Most __future__ imports don't work like this.  Maybe the closest
> thing is the generator_stop one, but at least that places a flag on
> the code object to indicate the difference.  Would "extended lists"
> have some kind of magic attribute indicating which extensions they're
> using?  That may have been marginally acceptable in the case of PEP
> 479, which was essentially a bugfix, and set the attribute on code
> objects which are an obscure internal data structure.  But allowing
> this kind of thing for "user-facing" objects like lists would create a
> profusion of different list objects with different behavior depending
> on some combination of attributes indicating "what extends me" --- or,
> even worse, create different behavior without any such overt
> indication of which extensions are in use for a given object.
>
> The idea that the file in which code is written would somehow
> determine this type of runtime behavior seems to me to break my
> assumption that by knowing an object's identity I should have all the
> information I need to know about how to use it.  Some of the posts
> earlier in this thread seem to suggest that somehow the module where
> something was defined (something --- not sure what --- maybe the
> object with the extended method?  maybe the extended method itself?)
> would somehow get a hook to override attribute access on some objects
> (again, not sure which objects).
>
> That to me is the exact opposite of encapsulation.  Encapsulation
> means the object itself contains all its behavior.  If there is some
> getattr-like hook in some other module somewhere that is lying in wait
> to override attribute access on a given object "only sometimes" then
> that's not encapsulation at all.  It's almost as bad as the infamous
> COME FROM statement!
>
> Existing mechanisms like __getattribute__ are not parallel at all.
> When you know an object's identity, you know its MRO, which tells you
> all you need to know about what __getattribute__ 

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-22 9:25 a.m., Chris Angelico wrote:
> (Oh, and another wrinkle, although a small one: Code objects would
> need to keep track of their modules. Currently functions do, but code
> objects don't. But that seems unlikely to introduce further
> complications.)

What? No you just stop emitting LOAD_ATTR and instead emit LOAD_LOCAL
'__opcode_load_attr_impl__' and call it with the relevant values.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/H5QSWNLWC4HOR3JPXBNDWCSE2QFHXPS6/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GLM2YWGRWHOAIQ7EJOXUNPGPY7HEEZPN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

Oh this is a long one.

Hypothetically, let's say you have a proxy object:

class Foo:
  def __getattribute__(self, thing):
    return getattr(super().__getattribute__(self, "proxied"), thing)

Should this really include extension methods into it by default?

This is clearly wrong. The local override for the LOAD_ATTR opcode
should NOT apply to proxy methods except where explicitly requested.
Also sometimes you are supposed to call the dunder directly, like in the
above example. It's not *bad* to do it if you know what you're doing.

The point is that the caller using your proxy object should opt-in to
the extension methods, rather than break with no way to opt-out of them.
Your extension methods shouldn't propagate to proxy objects.

To go even further, should all your class definitions that happen to
extend a class with in-scope extension methods automatically gain those
extension methods? Because with actual extension methods, that doesn't
happen. You can have class MyList(list): pass and other callers would
not get MyList.flatten even with you being able to use MyList.flatten
locally.

Extension methods are more like Rust traits than inheritance-based OOP.
Also note that they use instance method syntax, but no other. That is
they apply to LOAD_ATTR opcodes but should not apply to getattr!
(Indeed, reflection in C#/Kotlin doesn't see the extension methods!)

On 2021-06-22 6:57 a.m., Steven D'Aprano wrote:
> I'm sorry Soni, I don't understand what you are arguing here. See below.
>
>
> On Mon, Jun 21, 2021 at 10:09:17PM -0300, Soni L. wrote:
> > 
> > 
> > On 2021-06-21 9:39 p.m., Steven D'Aprano wrote:
> > 
> > 
> > >
> > > Fourth step is that you go ahead and use lists as normal. Whether you 
> > > use getattr or dot syntax, any extension methods defined in spam.py will 
> > > show up, as if they were actual list methods.
> > >
> > > hasattr([], 'head')  # returns True
> > > list.tail  # returns the spam.tail function object (unbound method)
> > >
> > > They're not monkey-patched: other modules don't see that.
> > >
> > >
> > 
> > Python is a dynamic language. Maybe you're using hasattr/getattr to
> > forward something from A to B. If "other modules don't see that" then
> > this must work as if there were no extension methods in place.
>
> What's "forward something from A to B" mean? What are A and B?
>
> If "this" (method lookups) "must work as if there were no extension 
> methods in place" then extension methods are a no-op and are pointless. 
> You write an extension method, register it as applying to a type, the 
> caller opts-in to use it, and then... nothing happens, because it "must 
> work as if there were no extension methods in place".
>
> Surely that isn't what you actually want to happen. But if not, I have 
> no idea what you mean.
>
> The whole point of extension methods is that once the caller opts in to 
> use them, method look ups (and that includes hasattr and getattr) must 
> work as if the extension methods **are in place**.
>
> The must be no semantic difference between:
>
> obj.method(arg)
>
> and
>
> getattr(obj, 'method')(arg)
>
> regardless of whether `method` is a regular method or an extension 
> method.
>
>
> > So you
> > actually wouldn't want the local load_attr override to apply to those.
> > If you did... well, just call the override directly.
>
> I have no idea what that means. What is "the local load_attr override"?
>
>
> > If the override was
> > called __opcode_load_attr_impl__ you'd just call
> > __opcode_load_attr_impl__ directly instead of going through getattr.
>
> As a general rule, you should not be calling dunders directly.
>
> You seem to have missed the point that extension methods are intended as 
> a mechanism to **extend a type** by giving it new methods on an opt-in 
> basis. I want to call them "virtual methods" except that would add 
> confusion regarding virtual subclasses and ABCs etc.
>
> Maybe you need to read the Kotlin docs:
>
> https://kotlinlang.org/docs/extensions.html
>
> and the C# docs:
>
> https://docs.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/extension-methods
>
> Wikipedia also has a broad overview from a language-agnostic 
> perspective:
>
> https://en.wikipedia.org/wiki/Extension_method
>
> Here's an example in TypeScript and Javascript:
>
> https://putridparrot.com/blog/extension-methods-in-typescript/
>
>
>
> In particular note these comments:
>
> # Kotlin
> "Such functions are available for calling in the usual way

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 9:39 p.m., Steven D'Aprano wrote:


>
> Fourth step is that you go ahead and use lists as normal. Whether you 
> use getattr or dot syntax, any extension methods defined in spam.py will 
> show up, as if they were actual list methods.
>
> hasattr([], 'head')  # returns True
> list.tail  # returns the spam.tail function object (unbound method)
>
> They're not monkey-patched: other modules don't see that.
>
>

Python is a dynamic language. Maybe you're using hasattr/getattr to
forward something from A to B. If "other modules don't see that" then
this must work as if there were no extension methods in place. So you
actually wouldn't want the local load_attr override to apply to those.
If you did... well, just call the override directly. If the override was
called __opcode_load_attr_impl__ you'd just call
__opcode_load_attr_impl__ directly instead of going through getattr.
There needs to be an escape hatch for this.

Or you *could* have getattr be special (called by load_attr) and
overridable, and builtins.getattr be the escape hatch, but nobody would
like that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VCNS6AEZ7NDMUOD2AEKOWWGH7XLTIRGP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 8:57 p.m., Thomas Grainger wrote:
> It seems odd that it would be per module and not per scope?

It's unusual to import things at the scope level. Usually things get
imported at the module level, so, using module language doesn't seem
that bad.

But yes, it's per scope, but in practice it's per module because nobody
would actually use this per scope even tho they could. :p

>
> On Tue, 22 Jun 2021, 00:55 Soni L.,  <mailto:fakedme%2...@gmail.com>> wrote:
>
>
>
> On 2021-06-21 8:42 p.m., Steven D'Aprano wrote:
> > On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:
> >
> > > Quite the opposite. You ask the local module (the one that the
> code was
> > > compiled in), and the module decides whether/when to ask the
> object itself.
> > >
> > > In other words, every
> > >
> > > foo.bar
> > >
> > > would be sugar for
> > >
> > > __getattr__(foo, "bar")
> > >
> > > (where __getattr__ defaults to builtins.getattr) instead of
> being sugar for
> > >
> > > (foo, "bar")
> >
> > All you've done here is push the problem further along -- how does
> > `__getattr__` (`__getattribute__`?) decide what to do?
> >
> > * Why is this extension-aware version per module, instead of a
> builtin?
> >
> > * Does that mean the caller has to write it in every module they
> want to
> >   make use of extensions?
> >
> > * Why do we need a second attribute lookup mechanism instead of
> having
> >   the existing mechanism do the work?
> >
> > * And most problematic, if we have an extension method on a
> type, the
> >   builtin getattr ought to pick it up.
> >
> >
> > By the way, per-module `__getattr__` already has a meaning, so
> this name
> > won't fly.
> >
> > https://www.python.org/dev/peps/pep-0562/
> <https://www.python.org/dev/peps/pep-0562/>
> >
> >
>
> No, you got it wrong. Extension methods don't go *on* the type being
> extended. Indeed, that's how they differ from monkeypatching.
>
> The whole point of extension methods *is* to be per-module. You could
> shove it in the existing attribute lookup mechanism (aka the
> builtins.getattr) but that involves runtime reflection, whereas
> making a
> new, per-module attribute lookup mechanism specifically designed to
> support a per-module feature would be a lot better.
>
> Extension methods *do not go on the type*.
>
> And sure, let's call it __opcode_load_attr_impl__ instead. Sounds
> good?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at
> 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/
> 
> <https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/>
> Code of Conduct: http://python.org/psf/codeofconduct/
> <http://python.org/psf/codeofconduct/>
>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MBHRH4FTGN7NCCHEHU3HVE4UPDDWHHHY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 8:42 p.m., Steven D'Aprano wrote:
> On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:
>
> > Quite the opposite. You ask the local module (the one that the code was
> > compiled in), and the module decides whether/when to ask the object itself.
> > 
> > In other words, every
> > 
> > foo.bar
> > 
> > would be sugar for
> > 
> > __getattr__(foo, "bar")
> > 
> > (where __getattr__ defaults to builtins.getattr) instead of being sugar for
> > 
> > (foo, "bar")
>
> All you've done here is push the problem further along -- how does 
> `__getattr__` (`__getattribute__`?) decide what to do?
>
> * Why is this extension-aware version per module, instead of a builtin?
>
> * Does that mean the caller has to write it in every module they want to
>   make use of extensions?
>
> * Why do we need a second attribute lookup mechanism instead of having
>   the existing mechanism do the work?
>
> * And most problematic, if we have an extension method on a type, the 
>   builtin getattr ought to pick it up.
>
>
> By the way, per-module `__getattr__` already has a meaning, so this name 
> won't fly.
>
> https://www.python.org/dev/peps/pep-0562/
>
>

No, you got it wrong. Extension methods don't go *on* the type being
extended. Indeed, that's how they differ from monkeypatching.

The whole point of extension methods *is* to be per-module. You could
shove it in the existing attribute lookup mechanism (aka the
builtins.getattr) but that involves runtime reflection, whereas making a
new, per-module attribute lookup mechanism specifically designed to
support a per-module feature would be a lot better.

Extension methods *do not go on the type*.

And sure, let's call it __opcode_load_attr_impl__ instead. Sounds good?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 3:01 p.m., Chris Angelico wrote:
> Thanks for clarifying. This doesn't change the problem though - it
> just changes where the issue shows up. (BTW, what you're describing is
> closer to __getattribute__ than it is to __getattr__, so if you're
> proposing this as the semantics, I strongly recommend going with that
> name.)

Oh, sorry, thought __getattribute__ was the fallback and __getattr__ the
one always called, what with getattr -> __getattr__. But yeah,
__getattribute__ then.

>
> So, here's the question - a clarification of what I asked vaguely up
> above. Suppose you have a bunch of these extension methods, and a
> large project. How are you going to register the right extension
> methods in the right modules within your project? You're binding the
> functionality to the module in which the code was compiled, which will
> make exec/eval basically unable to use them, and that means you'll
> need some way to set them in each module, or to import the setting
> from somewhere else. How do you propose doing this?

For exec/eval you just pass in the locals:

exec(foo, globals(), locals())

because this __getattribute__ is just a local like any other.

As for each module, you'd import them. But not quite with "import":

import extension_methods # magic module, probably provides an
@extend(class_) e.g. @extend(list)
import shallow_flatten
import deep_flatten
__getattribute__ = extension_methods.getattribute(
  shallow_flatten.flatten, # uses __name__
  deepflatten=deep_flatten.flatten, # name override
  __getattribute__=__getattribute__, # optional, defaults to
builtins.getattr
)

This would have to be done for each .py that wants to use the extension
methods.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/3DL46SIS2A7D3W4FNSTH37O6VDJJB2ZP/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ROZN62HQYL6X5LCJ7J5DNOC7W3DBGYHF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 12:49 p.m., Chris Angelico wrote:
> On Tue, Jun 22, 2021 at 1:44 AM Soni L.  wrote:
> >
> >
> >
> > On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> > > Soni L. writes:
> > >
> > >  > The trick to extension methods is that they're only available when you
> > >  > explicitly use them.
> > >
> > > What does "explicitly use them" mean?  How does this help avoid the
> > > kinds of problems we know that monkey-patching causes?
> >
> > Monkey-patching:
> >
> > ```py mod1.py
> > import foo
> >
> > foo.Bar.monkeymethod = ...
> > ```
> >
> > ```py mod2.py
> > import foo
> >
> > foo.Bar.monkeymethod = ...
> > ```
> >
> > "Extension methods":
> >
> > ```py mod1.py
> > import foo
> >
> > def __getattr__(o, attr):
> >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > return ...
> >   return getattr(o, attr)
> > ```
> >
> > ```py mod2.py
> > import foo
> >
> > def __getattr__(o, attr):
> >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > return ...
> >   return getattr(o, attr)
> > ```
> >
> > Note how the former changes foo.Bar, whereas the latter only changes the
> > module's own __getattr__. You can't have conflicts with the latter.
> > (Also note that this "module's own __getattr__" doesn't provide
> > extension methods by itself, but can be used as a mechanism to implement
> > extension methods.)
>
> So what you're saying is that, in effect, every attribute lookup has
> to first ask the object itself, and then ask the module? Which module?
> The one that the code was compiled in? The one that is currently
> running? Both?
>
> And how is this better than just using a plain ordinary function? Not
> everything has to be a method.

Quite the opposite. You ask the local module (the one that the code was
compiled in), and the module decides whether/when to ask the object itself.

In other words, every

foo.bar

would be sugar for

__getattr__(foo, "bar")

(where __getattr__ defaults to builtins.getattr) instead of being sugar for

(foo, "bar")

(where <> is used to indicate that it doesn't quite desugar that way -
otherwise you'd need to recursively desugar it to
builtins.getattr(builtins, "getattr") which uh, doesn't work.)

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/EMA2EBZLIR7DLNJAHWHURATJ4WVBOEUJ/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AGYFBB6XJ4V7ASID54PNGFZK74ZD4HWA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> Soni L. writes:
>
>  > The trick to extension methods is that they're only available when you
>  > explicitly use them.
>
> What does "explicitly use them" mean?  How does this help avoid the
> kinds of problems we know that monkey-patching causes?

Monkey-patching:

```py mod1.py
import foo

foo.Bar.monkeymethod = ...
```

```py mod2.py
import foo

foo.Bar.monkeymethod = ...
```

"Extension methods":

```py mod1.py
import foo

def __getattr__(o, attr):
  if isinstance(o, foo.Bar) and attr == "monkeymethod":
    return ...
  return getattr(o, attr)
```

```py mod2.py
import foo

def __getattr__(o, attr):
  if isinstance(o, foo.Bar) and attr == "monkeymethod":
    return ...
  return getattr(o, attr)
```

Note how the former changes foo.Bar, whereas the latter only changes the
module's own __getattr__. You can't have conflicts with the latter.
(Also note that this "module's own __getattr__" doesn't provide
extension methods by itself, but can be used as a mechanism to implement
extension methods.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RD5LKZUBBDAUH7UUMCRY6HUYZHPSKFQH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-20 7:48 p.m., Steven D'Aprano wrote:
> The technique you are calling "extension methods" is known as 
> "monkey-patching" in Python and Ruby.
>
> With respect to a fine language, Kotlin, it doesn't have the user-base 
> of either Python or Ruby. Python does not allow monkey-patching builtin 
> classes, but Ruby does:
>
> https://avdi.codes/why-monkeypatching-is-destroying-ruby/
>
> A cautionary tale. So what does Kotlin do to prevent that sort of thing?
>
> Can you use a regular function that takes a list as argument, instead of 
> monkey-patching the list class to add a method?
>
> The beauty of a function is that every module is independent, so they 
> can add their own list extensions (as functions) without stomping over 
> each other. Whereas if they add them directly on to the list class 
> itself, they can clash.
>

The trick to extension methods is that they're only available when you
explicitly use them.

In other words, a module/package can't force them on you.

The problem with "monkey-patching" is that it *does* get forced on you.

With the way Python works, your only option to implement extension
methods is to monkey-patch __getattr__. An alternative would be to have
a module-level __getattr__, which by default resolves to the object's
own attributes, but can be replaced by the module to provide proper
(module-local) extension methods functionality. Thus, any module-local
extension methods would also take priority over subclasses' attributes,
without conflicting with them.

(It would also slow everything down af, but that's beside the point.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q4WLJWTTY3AEK7SRVJ3TAFRHQOP6CKG3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: "except;" - semicolon after except, to get rid of indentation when doing error recovery

[Soni L.]

On 2021-06-15 10:13 p.m., Joao S. O. Bueno wrote:
> Sorry - personally I think this is absolutely ugly :-)  So I will
> bikeshed.
>
> If this thread even go ahead - since the idea is not that bad, maybe
> allowing `try` on the same line?
> Then it would be inline with `elif` - but still structured "English like" 
>
> try:
>    statement
> except ValueError try:
>    statement2
> except TypeError:
>    ...
>
> I had sometimes needed 2 and maybe up to 3 levels of this, nothing
> 'blocker', but maybe,
> just maybe, it would not be bad. 
> However, in real life, usually one wants to put more
> statements on the `except` clause than a bare nested try block.
> (logging, etc...)

For sure. The point is to turn the "rest of the function" into the
except block. Note that the try block would have to end with a return,
and no finally block would be allowed.

>
>
>
>
> On Tue, 15 Jun 2021 at 21:58, Chris Angelico  <mailto:ros...@gmail.com>> wrote:
>
> On Wed, Jun 16, 2021 at 10:51 AM Soni L.  <mailto:fakedme%2...@gmail.com>> wrote:
> >
> > Sometimes it would be useful to be able to write:
> >
> > def foo():
> >   try: return thing()
> >   except ValueError;
> >   try: return otherthing()
> >   except ValueError;
> >   try: return yetotherthing()
> >   except ValueError;
> >   if shouldraise(): raise
> >
> > But currently this needs to be written like so:
> >
> > def foo():
> >   try: return thing()
> >   except ValueError:
> >     try: return otherthing()
> >     except ValueError:
> >       try: return yetotherthing()
> >       except ValueError:
> >         if shouldraise(): raise
> >
> > Look at all that unnecessary indentation! Would be nice to get
> rid of it.
>
> Dangerous idea - my first interpretation of that syntax was that it
> would be equivalent to "except ValueError: pass", which would be very
> confusing (it's subtly different in your example with return, and
> drastically different in other cases).
>
> Are you doing this sort of thing a lot? And if you are, do you
> actually need/want the exception chaining that comes from burying more
> and more code into the except clauses? I know this is just a trivial
> example, but I'd be looking to see if it can be done with a loop
> instead.
>
> def foo():
>     for func in (thing, otherthing, yetotherthing):
>         try: return func()
>         except ValueError: pass
>
> or something like that.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at
> 
> https://mail.python.org/archives/list/python-ideas@python.org/message/CJCSPO4N3RBGMXDFPT7HHIFROM4BZLN6/
> 
> <https://mail.python.org/archives/list/python-ideas@python.org/message/CJCSPO4N3RBGMXDFPT7HHIFROM4BZLN6/>
> Code of Conduct: http://python.org/psf/codeofconduct/
> <http://python.org/psf/codeofconduct/>
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/ISYQKXGEVJOA3SY3SKJ7EZ4TUIS2ZGBC/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NVHMGSSJ6CG4EXEFUI34CHUUFLBTLXBV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] "except;" - semicolon after except, to get rid of indentation when doing error recovery

[Soni L.]

Sometimes it would be useful to be able to write:

def foo():
  try: return thing()
  except ValueError;
  try: return otherthing()
  except ValueError;
  try: return yetotherthing()
  except ValueError;
  if shouldraise(): raise

But currently this needs to be written like so:

def foo():
  try: return thing()
  except ValueError:
    try: return otherthing()
    except ValueError:
  try: return yetotherthing()
  except ValueError:
    if shouldraise(): raise

Look at all that unnecessary indentation! Would be nice to get rid of it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U4YH5O255RBMNXOM3VEUBCDDCHQDFES5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Have virtual environments led to neglect of the actual environment?

[Soni L.]

On 2021-02-23 9:45 p.m., Random832 wrote:

I was reading a discussion thread 
 about various 
issues with the Debian packaged version of Python, and the following statement stood 
out for me as shocking:

Christian Heimes wrote:
> Core dev and PyPA has spent a lot of effort in promoting venv because we 
don't want users to break their operating system with sudo pip install.

I don't think sudo pip install should break the operating system. And I think if it does, 
that problem should be solved rather than merely advising users against using it. And why 
is it, anyway, that distributions whose package managers can't coexist with pip-installed 
packages don't ever seem to get the same amount of flak for "damaging python's 
brand" as Debian is getting from some of the people in the discussion thread? Why is 
it that this community is resigned to recommending a workaround when distributions decide 
the site-packages directory belongs to their package manager rather than pip, instead of 
bringing the same amount of fiery condemnation of that practice as we apparently have for 
*checks notes* splitting parts of the stdlib into optional packages? Why demand that pip 
be present if we're not going to demand that it works properly?

I think that installing packages into the actual python installation, both via 
distribution packaging tools and pip [and using both simultaneously - the 
Debian model of separated dist-packages and site-packages folders seems like a 
reasonable solution to this problem] can and should be a supported paradigm, 
and that virtual environments [or more extreme measures such as shipping an 
entire python installation as part of an application's deployment] should 
ideally be reserved for the rare corner cases where that doesn't work for some 
reason.

How is it that virtual environments have become so indispensable, that no-one 
considers installing libraries centrally to be a viable model anymore? Are 
library maintainers making breaking changes too frequently, reasoning that if 
someone needs the old version they can just venv it? Is there some other cause?


OS-wide venv for OS packages, sudo pip install for pip packages? 
Generally you want to use OS packages for OS-packaged apps, so a venv is 
fine in that case.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KMRNKSVEPHAXA7BHFT4PWX4EKWYUF4G7/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SE6UR2KHIB5ODRAVIVTULMFFIAGKPOFG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Deprecate/change the behaviour of ~bool

[Soni L.]

On 2021-02-23 7:10 p.m., Steven D'Aprano wrote:

On Tue, Feb 23, 2021 at 01:57:08PM -0300, Soni L. wrote:


> What about getting the parser to recognize ^~ as an XNOR operator and 
> have __xnor__ on bools actually do xnor on bools? Make xnor a real 
> operator by fusing two operators together!


Let's not break existing code for the sake of an operator that hardly
anyone ever uses.

There are exactly 2**4 = 16 boolean operators of two variables. Python
only supports two: `and` and `or`. Plus a single unary operator `not`
(out of four possible unary operators). What makes xnor so special that
you want it to be an operator?

Do you have any use-cases for an xnor operator? What do you use it for?

These are not rhetorical questions. If you want to push this proposal
forward, you need to start with actual concrete use-cases, not "it would
be nice ..." and "how about ...".

Why can't you write a function to do what you want?




It's because someone was doing a comparison of a bool to the result of a 
comparison, a la (x == y) == z, and that looks kinda weird (especially 
because x == y == z has a completely different meaning). ^ works as a 
replacement for != but for == the only option is to either use "thing ^ 
(not foo)" or "not (thing ^ foo)". It's not that it's special - we 
"technically" already have it, if you abuse another language feature - 
it'd just make some code easier to understand.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CRGGZRCE4HVDHQWFBQ7OO4CQB4OVIWHX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Deprecate/change the behaviour of ~bool

[Soni L.]

64-bit bools were a mistake, tbh. Oh well, at least we tried.

What about getting the parser to recognize ^~ as an XNOR operator and 
have __xnor__ on bools actually do xnor on bools? Make xnor a real 
operator by fusing two operators together!


On 2021-02-23 5:49 a.m., Steven D'Aprano wrote:

On Mon, Feb 22, 2021 at 10:11:14PM -0300, Soni L. wrote:

> Currently ~False is -1 and ~True is -2. Would be nicer if ~bool was the 
> same as not bool.


That's your opinion. I disagree.

Bitwise-not is not the same thing as boolean-not, and they should not be
spelled the same, especially not when that would break the invariant
that if `a == b` then `~a == ~b`.

>>> 0 == False
True
>>> ~0 == ~False
True


If you want boolean-not, then use the boolean-not operator `not`.


> In particular, this is nice for xnor operator: a ^~ b. This currently 
> works on ints, but not on bools


If it works on ints, of course it works on bools, because bools are
ints. You get identical results whether you use int (0, 1) or bools
(False, True) in every combination:


>>> def xnor(a, b):
... return a ^ ~b
...
>>> [xnor(a, b) for a in (0, 1) for b in (0, 1)]
[-1, -2, -2, -1]
>>> [xnor(a, b) for a in (0, 1) for b in (False, True)]
[-1, -2, -2, -1]
>>> [xnor(a, b) for a in (False, True) for b in (0, 1)]
[-1, -2, -2, -1]
>>> [xnor(a, b) for a in (False, True) for b in (False, True)]
[-1, -2, -2, -1]



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KES7CSGDKZC2J6F5NSWZNLQH744BBLJR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Deprecate/change the behaviour of ~bool

[Soni L.]

Currently ~False is -1 and ~True is -2. Would be nicer if ~bool was the 
same as not bool. Hopefully nobody actually relies on this but 
nevertheless, it would be a backwards-incompatible change so the whole 
deprecation warnings and whatnot would be required.


In particular, this is nice for xnor operator: a ^~ b. This currently 
works on ints, but not on bools, while most other operators, including 
xor, do successfully work on bools.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PTIKNJRKI33UEC62UDJOR5N6WOENU6PB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Regular Expression | re - Support String List Input Type List[str]

[Soni L.]

would rather it took List[str] patterns instead, for matching against
List[str] input. so you could do things like:

pattern = ["foo", "?", "\", "?", "bar", "baz"]
input = ["foo", "?", "bar", "baz"] # matches
input = ["?", "bar", "baz"] # matches
input = ["foo?barbaz"] # doesn't match

bonus points if it also takes arbitrary objects:

pattern = [None, True, False, "?"]
input = [None, True] # matches
input = [None, False] # doesn't match
input = [None, True, False] #matches

On 2020-09-28 8:10 p.m., Giang Le wrote:
> Hi Everyone,
>
> I would like to propose an idea for the regular expression module 
> re.search(Pattern, Input) to support List[str] input type.
> So it will return a matched object list if the input is a string list. 
> Otherwise, it will return a normal matched object, if the input is a normal 
> string
>
> Best Regards
> Giang
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/4LOOP2FDK7JQ7XU3Z4Y7JNLOOW5FQOHH/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QI2QWN7PO7D5JXQ4VPITDTUWCX7SIVZ2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: exception instance call should raise exception

[Soni L.]

On 2020-07-05 12:58 p.m., MRAB wrote:

On 2020-07-05 14:10, Soni L. wrote:



On 2020-07-05 10:03 a.m., MRAB wrote:

On 2020-07-05 13:39, Soni L. wrote:

1. saves keywords and 2. can be passed as callables.

since there's no lambda: raise, 2 is a new feature.


How would you re-raise an exception?


just call it.


Wouldn't that just conflate raise with re-raise?


raise x vs except: raise?

just except foo: foo()





What would be the replacement for:

    raise ex from None

and so forth?


no idea. just don't use that anymore?


[snip]

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SKPHKNJKLYN6QQWBEY2XIVRYANPUUS33/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: exception instance call should raise exception

[Soni L.]

On 2020-07-05 10:03 a.m., MRAB wrote:

On 2020-07-05 13:39, Soni L. wrote:

1. saves keywords and 2. can be passed as callables.

since there's no lambda: raise, 2 is a new feature.


How would you re-raise an exception?


just call it.



What would be the replacement for:

    raise ex from None

and so forth?


no idea. just don't use that anymore?



I prefer the clarity of knowing that it's raising an exception and not 
just calling. Calls usually return.



On 2020-07-05 1:15 a.m., Steven D'Aprano wrote:

On Sun, Jul 05, 2020 at 12:48:08AM -0300, Soni L. wrote:
> ValueError()() should be the same as raise ValueError() but with 
an > extra call level.

> > and raise should be deprecated.

Proposals that are given without justification should be rejected
without justification:

"No they shouldn't."

If you feel like giving reasons for these changes other than 
"because I,

Soni, say so", then I will consider the merits of those reasons before
agreeing or disagreeing with the proposal.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MFNOEEOUMRGJXM433DQQVT57S5ERM7U3/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D4ECM6WBYNRY2MX2FSAXI6HNNCW6L47R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: exception instance call should raise exception

[Soni L.]

1. saves keywords and 2. can be passed as callables.

since there's no lambda: raise, 2 is a new feature.

On 2020-07-05 1:15 a.m., Steven D'Aprano wrote:

On Sun, Jul 05, 2020 at 12:48:08AM -0300, Soni L. wrote:
> ValueError()() should be the same as raise ValueError() but with an 
> extra call level.
> 
> and raise should be deprecated.


Proposals that are given without justification should be rejected
without justification:

"No they shouldn't."

If you feel like giving reasons for these changes other than "because I,
Soni, say so", then I will consider the merits of those reasons before
agreeing or disagreeing with the proposal.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WZGPUJ3D6PCKMYXEXUA7W3KD4AFV5W65/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] exception instance call should raise exception

[Soni L.]

ValueError()() should be the same as raise ValueError() but with an 
extra call level.


and raise should be deprecated.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3CY2XFVWPQBD7F7EGOGOSNJDKL5GOSMX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[Soni L.]

On 2020-07-03 9:37 p.m., Christopher Barker wrote:
On Fri, Jul 3, 2020 at 5:25 PM > wrote:


> I'd go for val[min:max] tbh.


another reason this is Not Good: in slicing syntax, a:b means >=a and 
< b -- this asymmetry is not what we would want here.


It doesn't make a difference tho, does it?



-CHB


--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UNOBF4O5MITA3IQ4OHBJOWJJW2ZZN3JK/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OFIHXBO6GAMFY5CNWFZYMW44CS3BNJ4S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[Soni L.]

On 2020-07-03 7:53 p.m., tcphon...@gmail.com wrote:

> I'd go for val[min:max] tbh.
> benefits:
> 
> - currently not allowed!

> - replaces min and max!

Is this a serious suggestion? No offence intended, but this seems 
ill-thought-out.
val[min:max] is perfectly legal syntax and it will only error if the variable 
val happens to not support indexing. This seems like it would break numpy 
arrays (which, IIRC, support both slicing and arithmetic operations), or, at 
the very least, be ambiguous on classes which choose to support both indexing 
and ordering (and thus clamping)


how do you plan to clamp a numpy array or a string?


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ASOXQV7M4HS3WOR7PHUNRXQUE4WQHAPE/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZT3OBOPNIMXQ2MU7N5RFBL5AJSYRZJ6Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[Soni L.]

On 2020-07-03 6:05 p.m., Federico Salerno wrote:


Even after years of coding it sometimes takes me a moment to correctly 
parse expressions like `min(max(value, minimum), maximum)`, especially 
when the parentheses enclose some local computation instead of only 
references, and the fact that `min` actually needs the *maximum* value 
as an argument (and vice-versa) certainly doesn't help.


It's a little surprising to me how shorthand functions akin to CSS's 
`clamp()` aren't more popular among modern programming languages. Such 
a tool is, in my opinion, even more missed in Python, what with it 
being focussed on readability and all. There would likely also be some 
(probably minor) performance advantages with implementing it at a 
builtins level.


Example usage:

>>> val = 100
>>> clamp(10, val, 50)
50
>>> val = 3
>>> clamp(10, val, 50)
10
>>> val = 25
>>> clamp(10, val, 50)
25

I'm undecided whether I would like `clamp`, `minmax` or something else 
as a name. I'm curious of other ideas.


As far as the signature is concerned, I like both `clamp(min, val, 
max)` for the logical position of the three arguments (which mirrors 
expressions such as `min < val < max`) and `clamp(val, min=x, max=y)`. 
I prefer the former, but declaring them as normal 
positional-and-keyword arguments would allow the programmer to use an 
alternative order if they so choose.




I'd go for val[min:max] tbh.

benefits:

- currently not allowed!
- replaces min *and* max!



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KWAOQFSV3YJYQV2Y5JXGXFCXHJ3WFLRS/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B3PGIHX2VUATNDAYW3VVHBNKVN3SEUT7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[Soni L.]

On 2020-06-29 11:46 a.m., Giampaolo Rodola' wrote:



On Mon, Jun 29, 2020 at 12:34 PM Nathaniel Smith > wrote:


On Mon, Jun 29, 2020 at 2:31 AM Steve Holden mailto:st...@holdenweb.com>> wrote:
> The commit message used, however, reveals implementation details
of the change which are irrelevant to the stated aim, which is
making the documentation clear and concise. Use of such language
is certainly regrettable, since it carries with it the implication
that the Python developer community has somehow been wilfully
sanctioning "relics of white supremacy" up until the change was made.
>
> There certainly is a place in tech for politics, as I have
argued many times, and I am sure nobody wishes to continue to use
language that might be offensive to readers. But I would suggest
that the politics can safely be omitted from commit messages,
since they can only properly be fully addressed in the
conversation about the PR in advance. The wording of the commit
message has the appearance (probably specious) of wanting to rub
former misdeeds in the face of a largely innocent community, and
that is the principal reason I found it distasteful and unnecessary.

I just re-read the commit message, and I think you're being
oversensitive and imagining things that aren't there. The actual
commit message is written in a straightforward and factual way, and
spends special effort on *absolving* the community of this kind of
guilt.


"The community" has nothing to be absolved of, "Strunk & White" has 
nothing to do with white supremacy and there is no guilt. If you feel 
guilty because you're white then that's your problem. I don't feel 
guilty for being white, the same way a black person should not feel 
guilty for being black. And I have literally ZERO excuses to make to 
you or anybody else in here because I'm white. Assuming guilt based on 
the color of your skin and constantly attacking that specific group 
because of that is racist. It's that simple. I find it astonishing how 
some people here don't seem to realize that (or pretend not to).


And what's the goal anyway? Make us all feel guilty, create yet 
another heated discussion, widen divisions, wait for the occasional 
folks who dare to speak up against this vitriol and kick them out? And 
then what? What is the plan here exactly? Don't you folks realize this 
is a technical forum? Don't you understand how inappropriate it is to 
constantly bring up these kinds of messages up here, and force people 
to either witness them silently for fear of repercussions, or to 
engage in the discussion and risk paying the consequences in terms of 
work / hiring / career / status / reputation etc.? Because that's what 
happens, and we all know it. This is a very public forum and we can 
all be traced back to here. There are professionals here, people who 
go to conferences and/or make a living out of Python, who pay the rent 
and support their family with it, and that don't want to be put in 
this position.


It does not scale. It will never scale. Because whether we like it or 
not we have to coexist together in this virtual space, including with 
people we don't like. And this is why it is such a good idea to leave 
politics out of the door and only stay focused on Python. We will 
still have different opinions and occasional clashes, but as long as 
they are technical they will be genuine, prolific and everything will 
be fine as it was before "this" started (I've been reading this list 
for 12 years now). Discussing politics, on the other hand, will only 
keep bringing conflict over and over again. There's tons of proof of 
this already, and I can't envision a different outcome in the long 
run. Because most of us are not OK with being put against a wall and 
being blamed for "supremacy", "guilt", "privilege" or whatever term 
you have in your jargon. I certainly am not. Furthermore, that jargon 
makes no sense outside of the US and it's just ridiculous. I'm 
European, am split between living here and in Asia, and I can 
guarantee you that much.


Please, stop this.


If your job depends on you ignoring white supremacy, it's likely that 
your workplace is unethical. Personally I'd rather starve than uphold 
white supremacy, but that's just me.


Do think about it, tho.



-
Giampaolo - gmpy.dev 


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TMLKHLDLDFE3JC3M6HOCUW4RIPQEXEIP/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to 

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[Soni L.]

On 2020-06-27 1:33 a.m., Steven D'Aprano wrote:

On Fri, Jun 26, 2020 at 11:36:47PM -0400, David Mertz wrote:
> On Fri, Jun 26, 2020, 8:40 PM Steven D'Aprano  wrote:
> 
> > "Clear and easily understandable" is subjective. What is clear and

> > understandable to me may be impenetrably confusing to others, or
> > obnoxiously dumbed down.
> >
> 
> Strunk and White's most famous mandate of "omit needless words" is likewise

> subjective.

Take the word out of the sentence, and does the sentence still mean the
same thing? Then the word was needless. That's an objective test.




Take word out of sentence, does sentence still mean same? Then word 
needless. Is objective test.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LM34VNRWPJPMM4HH45FWVFAGAYSY6HP6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: the 'z' string escape

[Soni L.]

On 2020-06-17 1:23 p.m., Christopher Barker wrote:


less than ideal, yes -- but please post the \z version -- is it any 
better?


I did, look again.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IPMERTJCEDUEJFTPTBAD2BHU6OYNO7V6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: the 'z' string escape

[Soni L.]

On 2020-06-17 12:46 p.m., Christopher Barker wrote:
On Wed, Jun 17, 2020 at 6:09 AM Soni L. <mailto:fakedme%2...@gmail.com>> wrote:


This also gives us two ways of doing indented strings (in Lua):

local ugly = "foo\n    \z
  bar\n    \z
  baz"
local nicer = "foo\n\z
   \x20   bar\n\z
   \x20   baz"


I'm having a really hard time seeing why that is either more readable, 
or easier to type than:


nicest = ("foo"
  "   bar"
  "   baz"
  )
(my prefered way these days)


I think you missed some \n's here.



or

nicest = \
"""foo
   bar"
   baz"""


this is ugly. what's wrong with textwrap.dedent?

however, I bring up again the original use-case which has nothing to do 
with textwrap.dedent, or nested indentation. But consider this 
artificial example:


foo = textwrap.dedent("""
    This is the help page for foo, a command \z
    with the following subcommands:
    bar - A very useful subcommand of foo \z
    and probably the subcommand you'll \z
    be using the most.
    baz - A simple maintenance command \z
    that you may need to use sometimes.
""")

Currently you'd have to write it as:

foo = (
    "This is the help page for foo, a command "
    "with the following subcommands:\n"
    "    bar - A very useful subcommand of foo "
    "and probably the subcommand you'll "
    "be using the most.\n"
    "    baz - A simple maintenance command "
    "that you may need to use sometimes."
)

or if you want it to look "nicer", and don't mind linters shouting at you:

foo = (
    "This is the help page for foo, a command "
    "with the following subcommands:\n"
    "    bar - A very useful subcommand of foo "
    "and probably the subcommand you'll "
    "be using the most.\n"
    "    baz - A simple maintenance command "
    "that you may need to use sometimes."
)

And I think this sucks.



-CHB



--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C2UFAEH2JUH7LUEL7TSVEO3ISTQSKSP2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: the 'z' string escape

[Soni L.]

On 2020-06-17 7:44 a.m., Stephen J. Turnbull wrote:

Soni L. writes:

  > Explicit is better than implicit.
  >
  > and \z is more explicit than #cont

It's not more explicit.  Line continuation in a string literal is
perfectly explicit and easy to explain *exactly*: "omit the '\' and
the following newline from the string being constructed".  You could
argue that '\z' is more easily visible ("readable" if you like), but
not that it's more explicit.

The simplest form of '\z' is "almost easy" to explain exactly ("omit
the following newline and all leading whitespace on the next line").
Complication enters when we ask, "but what happens if the character
following '\z' is not a newline?"

But more generally, string dedenting is hard to explain with much
precision, let alone exactly, except in (pseudo)code.  '\z' would be
even more complicated than textwrap.dedent, because it would apply
line by line.  Would the indentation that is stripped be string-wide
(so that '\z' could work like RFC 822 folding with the intended
whitespace at the *left* margin, where it's easy to see), or would
each '\z' just strip all the indentation on the next line?  Maybe '\z'
should replace all the surrounding whitespace with a single space?


Read the Lua docs. It makes it very simple:

"The escape sequence '|\z|' skips the following span of white-space 
characters, including line breaks; it is particularly useful to break 
and indent a long literal string into multiple lines without adding the 
newlines and spaces into the string contents."


Note that, in Lua, you can "break" the \z by inserting an \x20. This is 
because it only cares about literal whitespace.


> "foo\z   \x20bar"
foo bar

which is particularly useful for inserting indents into the string while 
also using \z. Oh, also, it matches zero-or-more, altho I guess an error 
would also be compliant with what's written in the manual. (And yes, Lua 
does leave stuff like this "undefined" all the time. We might want to 
explicitly specify "[...] span of zero-or-more [...]" tho.)


This also gives us two ways of doing indented strings (in Lua):

local ugly = "foo\n    \z
  bar\n    \z
  baz"
local nicer = "foo\n\z
   \x20   bar\n\z
   \x20   baz"

Obviously textwrap.dedent is nicer but again I don't propose \z for 
doing this, but rather for breaking up too long strings into multiple 
lines. The only point is to break up too long strings into multiple 
lines without accidentally turning them into (part of) a tuple. Which is 
why I put that #cont there. Also, that example with #cont? It's inside a 
tuple. Someone looking at it would see the tuple and the newline and the 
"missing" comma and would put a comma there so I need to put #cont there 
and having \z would be a better way to solve that.




I kinda like the string-wide idea, but

 s = textwrap.dedent("""\
 This\
  is\
  a\
  silly\
  example\
  of\
  a\
  dedented\
  and\
  folded\
  string.\
 """)
 assert s == "This is a silly example of a dedented and unfolded string."

works for me, specifically in real examples where *occasionally* for
some reason I want to fold a long line.  That is, although the
trailing '\' at the right margin is not so easy to see, the extra
space at the left margin of the next line is easy to see (at least
with a reasonable font).

Of course,

 s = "This"
 " is"
 " a"
 ...

works even better when what you want is an unfolded one-line string.

Steve



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/52EDFGIFAUFWS5D2MN3SU7SM7JNSPT37/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: the 'z' string escape

[Soni L.]

On 2020-06-16 2:37 a.m., Stephen J. Turnbull wrote:

Soni L. writes:

  > so I propose a \z string escape which lets me write the above as shown
  > below:
  >
  >      """switches to toml config format. the old 'repos' \z
  >      table is preserved as 'repos_old'"""

We already have that, if you don't care about left-alignment:

>>> """123456789\
... abcdefghi"""
'123456789abcdefghi'
>>> 


So '\z' would only give us removal of indentation whitespace from the
string.  Too much magic, and it has the same problem that trailing '\'
has: most people will have to intentionally look for it to see it
(though it's easier to spot than trailing '\' for me).
That's a -1 for me.


Explicit is better than implicit.

and \z is more explicit than #cont


Steve

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DB6ICUF5GNYRIWYWIOZJYK32M6IW62OL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] the 'z' string escape

[Soni L.]

in Lua 5.2+, there's this string escape that allows you to put 
"whitespace" (in particular, including newlines) in a string literal, by 
skipping them entirely. now, unlike lua's long strings, python *does* 
have escapes in long strings. however, sometimes you have help text:


    "switches to toml config format. the old 'repos' " #cont
    "table is preserved as 'repos_old'"

and... well yeah you can see what I did to make it work. if I used a 
long string here, I'd get a newline and a bunch of indentation in the 
middle.


so I propose a \z string escape which lets me write the above as shown 
below:


    """switches to toml config format. the old 'repos' \z
    table is preserved as 'repos_old'"""

(side note to avoid unnecessary comments: this help text refers to 
database tables and migrations. it's not about lua tables.)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UOLOQXYDJA7QCXUDM2YCU5MLARKVPYAS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New attribute __convert__ for classes

[Soni L.]

On 2020-06-14 12:33 p.m., artem6191 wrote:

This attribute will use for convertion classes to other types, i.e. 
int(MyClass) will return __convert__ result in MyClass

Example:
class MyClass:
def __init__(self, numer: int):
   self.number = number
def __convert__(self, type):
   assert type == int # Only integer is allowed
   return self.number*5 # Example

test = MyClass(1)
test_2 = MyClass(5)

print( int(test) ) # 5
print( int(test2) ) # 25


This is wrong. Do not use assert like this.


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IOJVYBX2PXGBMJ4G4TJQ5APIQHHKSXND/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C2TGR6B53TGIVLRMJDFHSVMNQIJ7L3M4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: await outside async function

[Soni L.]

n with the assumption that the 
proposal would provide some real benefit in the transition from sync 
to async (which I disagree with), I don't think this would come even 
close to justifying a full deprecation of asyncio into a separate "TBD 
API".


Thanks for sharing the idea Soni, but I would personally be opposed to 
every component of the proposal.


What's the point of having async, if it has to be such a massive pain to 
refactor my existing code into? Some of the issues caused by only 
partially switching to async, namely that you have to go out of your way 
to call an async function from non-async code, feel to me like 
unnecessary friction. I understand and love having 2 calling 
conventions, but, heck even having coroutines carry a run() method would 
be better than what we have today.


And then there's the issue of nested event loops. It just makes things 
even more problematic in hybrid codebases. :/


Consider my proposal the best of both worlds: you get all the benefits 
of async/await, and all the benefits of blocking APIs, and you get to 
choose whether to use autodetection or explicitly force the latter. (you 
could also force the existence of a nested event loop, but I don't think 
that'd be a good idea.)




On Fri, Jun 12, 2020 at 7:20 PM Soni L. <mailto:fakedme%2...@gmail.com>> wrote:




On 2020-06-12 5:47 p.m., J. Pic wrote:

Hi all,

Currently, you can not use await outside an async function, the
following code:

  async def lol():
    return 'bar'

  def test():
    return await lol()

  print(test())

Will fail with: SyntaxError: 'await' outside async function

Of course, you can use asyncio.run and then it works fine:

  import asyncio

  async def lol():
    return 'bar'

  def test():
    return asyncio.run(lol())

  print(test())

Why not make using await do asyncio.run "behind the scenes" when
called outside async function ?

Thank you in advance for your replies


What if we extend awaitables to be optionally "runnable"?

An await outside async def, would map to awaitable.run(). In the
specific case of coroutines, this would also automatically
propagate down. E.g.

async def foo():
  await bar()

def baz()
  await foo()

A call to baz() would thus create a coroutine foo(), and then
run() it. By my proposed semantics, the await bar() would also
call run() on the result of bar()!

In particular, this could allow one to turn any arbitrary async
coroutine into a blocking subroutine, as long as it supports this
run() protocol. This would be great for refactoring existing
blocking code into async - just use stuff as if it's sync, then
when you're done, switch up your function with async def, and
change relevant calls into it into "await" calls. Rinse and repeat
until you're ready to add a top-level event loop.

Optionally, this would also allow us to deprecate blocking APIs
(like open(), etc) and asyncio, and merge them into one TBD API.
Perhaps as part of Python 4. (e.g. turn existing open() into await
open(), particularly in non-async-def. I *think* this could be
done automatically as part of some sort of "3to4", even.)



-- 
∞


___
Python-ideas mailing list --python-ideas@python.org  
<mailto:python-ideas@python.org>
To unsubscribe send an email topython-ideas-le...@python.org  
<mailto:python-ideas-le...@python.org>
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived 
athttps://mail.python.org/archives/list/python-ideas@python.org/message/LOCYSYVRKXI45QQJOLYGZV6H2CBYTB7F/
Code of Conduct:http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
<mailto:python-ideas@python.org>
To unsubscribe send an email to python-ideas-le...@python.org
<mailto:python-ideas-le...@python.org>
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at

https://mail.python.org/archives/list/python-ideas@python.org/message/Y73VH53OXZZ2GDWRVGRX2B76U3TSHYZF/
Code of Conduct: http://python.org/psf/codeofconduct/



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NI6BP5Z5RTB7OOOJCFMGM2ZBYW2EXVXM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] await outside async function (was: Re: await outside async function could map to asyncio.run ?)

[Soni L.]

On 2020-06-12 5:47 p.m., J. Pic wrote:

Hi all,

Currently, you can not use await outside an async function, the 
following code:


  async def lol():
    return 'bar'

  def test():
    return await lol()

  print(test())

Will fail with: SyntaxError: 'await' outside async function

Of course, you can use asyncio.run and then it works fine:

  import asyncio

  async def lol():
    return 'bar'

  def test():
    return asyncio.run(lol())

  print(test())

Why not make using await do asyncio.run "behind the scenes" when 
called outside async function ?


Thank you in advance for your replies


What if we extend awaitables to be optionally "runnable"?

An await outside async def, would map to awaitable.run(). In the 
specific case of coroutines, this would also automatically propagate 
down. E.g.


async def foo():
  await bar()

def baz()
  await foo()

A call to baz() would thus create a coroutine foo(), and then run() it. 
By my proposed semantics, the await bar() would also call run() on the 
result of bar()!


In particular, this could allow one to turn any arbitrary async 
coroutine into a blocking subroutine, as long as it supports this run() 
protocol. This would be great for refactoring existing blocking code 
into async - just use stuff as if it's sync, then when you're done, 
switch up your function with async def, and change relevant calls into 
it into "await" calls. Rinse and repeat until you're ready to add a 
top-level event loop.


Optionally, this would also allow us to deprecate blocking APIs (like 
open(), etc) and asyncio, and merge them into one TBD API. Perhaps as 
part of Python 4. (e.g. turn existing open() into await open(), 
particularly in non-async-def. I *think* this could be done 
automatically as part of some sort of "3to4", even.)




--
∞

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LOCYSYVRKXI45QQJOLYGZV6H2CBYTB7F/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Y73VH53OXZZ2GDWRVGRX2B76U3TSHYZF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip(x, y, z, strict=True)

[Soni L.]

On 2020-05-06 12:48 p.m., Christopher Barker wrote:
On Tue, May 5, 2020 at 5:43 PM Steven D'Aprano > wrote:


Christopher's quoting is kinda messed up and I can't be bothered
fixing
it, sorry, so you'll just have to guess who said what :-)


Ideally, we are evaluating ideas independently of who expressed them, 
so I'll pretend I did that on purpose :-)


First: really people, it's all been said. I think we all (and I DO 
include myself in that) have fallen into the trap that "if folks don't 
agree with me, I must not have explained myself well enough" -- but in 
this case, we actually do disagree. And not really on the facts, just 
on the relative importance.


But since, I apparently did not explain myself well enough in this case:
> no -- but we could (and I think should) have a ternary flag, so that

> zip_longest becomes unnecessary. And we'd never get to eight
combinations:
> you can't have longest and shortest behavior at the same time!

A ternary flag of strict = True, False or what?


Come on:

ternary: having three elements, parts, or divisions 
(https://www.merriam-webster.com/dictionary/ternary)


did you really not know that? (and "flag" does not always mean 
"boolean flag", even thoughit often does 
(https://techterms.com/definition/flag) )


(by the way, I'm posting those references because I looked them up to 
make sure I wasn't using terms incorrectly)


This has been proposed multiple times on this list:

a flag that takes three possible values: "shortest" | "longest" | 
"equal" (defaulting to shortest of course). Name to be bikeshed later :-)

(and enum vs string also to be bikeshed later)



how about "length"?

length=True # longest
length=False # shortest (default)
length=None # equal

(altho I still think the "YAGNI function" system would be better >.>)


This demonstrates why the "constant flag" is so often an
antipattern. It
doesn't scale past two behaviours. Or you end up with a series of
flags:

    zip(*iterators, strict=False, longest=False, fillvalue=None)


I don't think anyone proposed an API like that -- yes, that would be 
horrid.


There are all sorts of reasons why a ternary flag would not be good, 
but I do think it should be mentioned in the PEP, even if only as a 
rejected idea.


But I still like it, 'cause the "flag for two behaviors and another 
function for the third" seem sliek the worse of all options.


-CHB


--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LSKG4XDKBEL5DHDWVOVDBGLA3QMF22YH/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C7BQYAYC3MF7ZNG5TJPQWG6WDZNDETBE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Equality between some of the indexed collections

[Soni L.]

On 2020-05-03 10:19 p.m., Steven D'Aprano wrote:

On Sat, May 02, 2020 at 11:01:21PM +0200, Alex Hall wrote:
> On Sat, May 2, 2020 at 10:52 PM Dominik Vilsmeier 
> wrote:
> 
> > `frozenset` and `set` make a counterexample:

> >
> > >>> frozenset({1}) == {1}
> > True
> >
> 
> Nice catch! That's really interesting. Is there reasoning behind

> `frozenset({1}) == {1}` but `[1] != (1,)`, or is it just an accident of
> history? 


Conceptually, sets are sets, whether they are mutable or frozen.


> Isn't a tuple essentially just a frozenlist? I know the intended
> semantics of tuples and lists tend to be different, but I'm not sure that's
> relevant.

o_O

If the intended semantics aren't relevant, I'm not sure what is...




for what it's worth, I see myself using tuples as frozen lists more 
often than their "intended semantics".


more specifically, you can't pass lists to:

1. isinstance
2. issubclass
3. str.endswith

among others. so I sometimes just convert a list of strings into a tuple 
of strings and store it somewhere so I can use it with str.endswith 
later. (this is not how you're "supposed" to implement domain suffix 
blocks but w/e)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2TGZOAQGCWTNOZDASV5TROR2H7B5PKK3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce 100 more built-in exceptions

[Soni L.]

On 2020-05-02 7:29 p.m., Steven D'Aprano wrote:

On Sat, May 02, 2020 at 03:39:50PM -0300, Soni L. wrote:

> def foo():
>   yield from ()
>   raise ValueError

 def foo():
 yield from ()
     raise ValueUnpackingError

Does that help?



the idea is that it'd become a RuntimeError when you unpack it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GQVFEDUZGVND7Q2ZCTVDSHOXWWUOCQDU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce 100 more built-in exceptions

[Soni L.]

On 2020-05-02 1:45 p.m., Steven D'Aprano wrote:

On Sat, May 02, 2020 at 01:20:01PM -0300, Soni L. wrote:
> 
> 
> On 2020-05-02 1:07 p.m., Steven D'Aprano wrote:

> >On Sat, May 02, 2020 at 12:50:19PM -0300, Soni L. wrote:
> >
> >> how about:
> >> 
> >> result = Foo.save()

> >> try:
> >>   x, y = result
> >> except ValueUnpackingError:
> >>   return ...
> >
> >If you do that, what benefit is ValueUnpackingError over just
> >ValueError?
> >
> >
> >
> unpacking (a generator) still doesn't wrap ValueError

Sorry, I don't understand that comment. What do you mean, "wrap"?

Unpacking a generator with too many values raises ValueError:

 py> a, b = (x for x in range(3))
 Traceback (most recent call last):
   File "", line 1, in 
 ValueError: too many values to unpack (expected 2)

> and ValueError is 
> raised a lot more than ValueUnpackingError.


Why does that matter, and how does that relate to the specific code
sample given?

Given the line of code `x, y = result`, the only possible way ValueError
could be raised is if `result` has the wrong number of items when
unpacking. So what benefit does ValueUnpackingError bring to this
example?



or if iterating result raises ValueError.

consider:

def foo():
  yield from ()
  raise ValueError

iterator = foo()
x, y = iterator

you get a ValueError, but not a ValueUnpackingError.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KISHW53CLRMACGEWZ2WRLD5DKPCEBNBS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce 100 more built-in exceptions

[Soni L.]

On 2020-05-02 1:07 p.m., Steven D'Aprano wrote:

On Sat, May 02, 2020 at 12:50:19PM -0300, Soni L. wrote:

> how about:
> 
> result = Foo.save()

> try:
>   x, y = result
> except ValueUnpackingError:
>   return ...

If you do that, what benefit is ValueUnpackingError over just
ValueError?



unpacking (a generator) still doesn't wrap ValueError, and ValueError is 
raised a lot more than ValueUnpackingError.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GFYD6CN6OAEFZ7AALKX5QFJRNX3SFOF7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce 100 more built-in exceptions

[Soni L.]

On 2020-05-02 11:59 a.m., Eric V. Smith wrote:

On 5/2/2020 7:18 AM, Alex Hall wrote:
On Sat, May 2, 2020 at 12:39 PM Henk-Jaap Wagenaar 
mailto:wagenaarhenkj...@gmail.com>> wrote:


Of course, there are other ways of writing this code, but imagine
this for a database interface where a save normally returns the
saved object (inspired by Django)

```
try:
    x, y = Foo.save()
except ValueUnpackingError:
    // oh... this means saving failed (it returned None instead)
    // let's return a sensible response
    return ExceptionToResponse(FooStateToException(foo.state))
```


My concern is that this is risky code. If `Foo.save()` has a bug 
somewhere inside (or maybe it calls back to your own code which has a 
bug) it could raise `ValueUnpackingError` for different reasons, and 
then that gets caught and misleads people into thinking that Foo.save 
returned None which leads to all sorts of frustration and confusion.


This code:

```
result = Foo.save()
if result is None:
    return ExceptionToResponse(...)
x, y = result
```

is not sexy, but it's explicit, safe, and it targets the real problem 
precisely.


Creating these new exceptions would encourage people to write code 
that will bite them later.


I agree that we don't want to encourage code like the above example 
with catching the hypothetical ValueUnpackingError. I contend that 
even if we added ValueUnpackingError, no one would ever write code 
that used it, but should instead do what they do now: unpack it after 
checking it (as Alex's example shows). Of course, that's a horrible 
API, but sometimes you have to call horrible APIs. The fact that the 
API is bad doesn't mean we should add a way to handle it poorly.


The only reason I'd want it to exist (and I don't) is to see it in the 
REPL. But for that, there's the exception message and the traceback, 
so a unique exception for this would be useless to me.




how about:

result = Foo.save()
try:
  x, y = result
except ValueUnpackingError:
  return ...

this would also handle generators (and, the unpacking operation should 
wrap the generator's ValueUnpackingError (if any) in a RuntimeError so 
it doesn't conflict with the unpacking operation's API)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DSCJVOQYV4NJR6AOHSTJJ4DUOVKESUZ4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 618: Add Optional Length-Checking To zip

[Soni L.]

On 2020-05-01 4:43 p.m., Chris Angelico wrote:

On Sat, May 2, 2020 at 5:21 AM Soni L.  wrote:
>
>
>
> On 2020-05-01 3:41 p.m., Chris Angelico wrote:
> > On Sat, May 2, 2020 at 4:38 AM Soni L.  wrote:
> > >
> > >
> > >
> > > On 2020-05-01 3:10 p.m., Brandt Bucher wrote:
> > > > I have pushed a first draft of PEP 618:
> > > >
> > > > https://www.python.org/dev/peps/pep-0618
> > > >
> > > > Please let me know what you think – I'd love to hear any *new* feedback 
that hasn't yet been addressed in the PEP!
> > >
> > > What about using an optional kwarg for a handler for mismatched lengths?
> > > I made a post about it on the other thread and it's not addressed in the
> > > PEP. It'd make zip capable of doing zip_shortest, zip_equal (aka
> > > zip(strict=True)) and zip_longest, it's not stringly-typed, and it's
> > > user-extensible. Something along the lines of zip(foo, bar, baz,
> > > and_then=lambda consumed_items, iters: ...).
> > >
> >
> > YAGNI.
>
> examples:
>
> # iterates in chunks, e.g. a very large file that wouldn't fit all in RAM
> zip(*[iter(x)]*32, and_then=lambda res, _: (yield res))

I'm honestly not sure how useful this really is in practice. Iterating
over a file is already going to be chunked. What do you gain by
wrapping it up in an opaque zip call?

> # strict zip
> sentinel = object()
> def zip_eq(res, iters):
>if res or any(next(x, sentinel) is not sentinel for x in iters):
>  raise ValueError
> zip(a, b, c, and_then=zip_eq)
> # this would ideally be zip.strict e.g. zip(a, b, c,
> and_then=zip.strict), but w/e.

So a messier and noisier spelling of what's already in this proposal...

> # normal (shortest) zip but using an explicit function
> def no_op(*args, **kwargs):
>pass
> zip(a, b, c, and_then=no_op)
>

... and a messier and noisier spelling of what we already have.

I say again, YAGNI. Give an actual use-case for the excessive
generality of your proposal - namely, the ability to provide a custom
function. And show that it's better with zip than just with a custom
generator function.


we can finally push for the no_op function, for starters. but the main 
benefit is, again, being able to get the iterated values which were 
silently swallowed by zip when the iteration stopped.


also: I don't like booleans. they're not extensible, unless you consider 
None. you either get it right the first time, add a new boolean argument 
later, or use enum.Flag from the beginning. this callback-based API 
sidesteps all these issues.


and just in case maybe zip(strict=True) should be zip(errors=True) and 
we can later change it to zip(errors="replace", value=Foo) to get 
zip_longest >.< (no really bools = bad please don't use them in new APIs)


ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UAQGBSUUFSQJRE56VGTHVXAHCJHUAYTM/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F43SYFQAK7O7TVUGLHMKIKJDESES4W25/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 618: Add Optional Length-Checking To zip

[Soni L.]

On 2020-05-01 3:41 p.m., Chris Angelico wrote:

On Sat, May 2, 2020 at 4:38 AM Soni L.  wrote:
>
>
>
> On 2020-05-01 3:10 p.m., Brandt Bucher wrote:
> > I have pushed a first draft of PEP 618:
> >
> > https://www.python.org/dev/peps/pep-0618
> >
> > Please let me know what you think – I'd love to hear any *new* feedback 
that hasn't yet been addressed in the PEP!
>
> What about using an optional kwarg for a handler for mismatched lengths?
> I made a post about it on the other thread and it's not addressed in the
> PEP. It'd make zip capable of doing zip_shortest, zip_equal (aka
> zip(strict=True)) and zip_longest, it's not stringly-typed, and it's
> user-extensible. Something along the lines of zip(foo, bar, baz,
> and_then=lambda consumed_items, iters: ...).
>

YAGNI.


examples:

# iterates in chunks, e.g. a very large file that wouldn't fit all in RAM
zip(*[iter(x)]*32, and_then=lambda res, _: (yield res))

# strict zip
sentinel = object()
def zip_eq(res, iters):
  if res or any(next(x, sentinel) is not sentinel for x in iters):
    raise ValueError
zip(a, b, c, and_then=zip_eq)
# this would ideally be zip.strict e.g. zip(a, b, c, 
and_then=zip.strict), but w/e.


# normal (shortest) zip but using an explicit function
def no_op(*args, **kwargs):
  pass
zip(a, b, c, and_then=no_op)



ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZJAGEHQTK46UBEFTNUD6RMDPZ7UEZ6YD/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WMEQWVMTX5GP53HB5CYHYURAM2UBWF5P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 618: Add Optional Length-Checking To zip

[Soni L.]

On 2020-05-01 3:10 p.m., Brandt Bucher wrote:

I have pushed a first draft of PEP 618:

https://www.python.org/dev/peps/pep-0618

Please let me know what you think – I'd love to hear any *new* feedback that 
hasn't yet been addressed in the PEP!


What about using an optional kwarg for a handler for mismatched lengths? 
I made a post about it on the other thread and it's not addressed in the 
PEP. It'd make zip capable of doing zip_shortest, zip_equal (aka 
zip(strict=True)) and zip_longest, it's not stringly-typed, and it's 
user-extensible. Something along the lines of zip(foo, bar, baz, 
and_then=lambda consumed_items, iters: ...).




Brandt
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZBB5L2I45PNLTRW7CCV4FDJO5DB7M5UT/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZORX3VCKXFLSODJW6NA33B7MLGFA7UTE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: is a

[Soni L.]

On 2020-05-01 2:46 p.m., Steele Farnsworth wrote:
So this would make `a` a new keyword. I don't think that could be 
added into python 4 at the earliest because it would immediately break 
all code for which `a` is a variable name.


we could have keyphrases instead of keywords, tbh.



I can appreciate wanting to make simple operations easy to read, 
though I think this relies too much on understanding English and 
wouldn't be intuitive for people who aren't English speaking. I am 
native English speaking so I wouldn't know for sure, but I think 
accepting that "or" is the same as "||" is an easier jump to make than 
"x is a y" being a construct for indicating that x belongs to the y class.


If you need this English-style syntax, I believe `type(x) is y` is 
guaranteed to be True if x is exactly of the y type and not one of its 
super classes.


On Fri, May 1, 2020, 1:27 PM gbs--- via Python-ideas 
mailto:python-ideas@python.org>> wrote:


In cases where it makes sense to do explicit type checking (with
ABCs or whatever), I really detest the look of the isinstance()
function.

if isinstance(thing, Fruit) and not isinstance(thing, Apple):

Yucky.

What I really want to write is:

if thing is a Fruit and thing is not an Apple:

and after thinking about it on and off for a while I wonder if it
might indeed be possible to teach the parser to handle that in a
way that eliminates almost all possible ambiguity with the regular
"is", including perhaps 100% of all existing standard library code
and almost all user code?

Maybe this has been considered at some point in the past? The "is
[not] a|an" proposal would at least be a strong contender for
"hardest thing to search for on the internet" lol.

Thanks!

Gavin
___
Python-ideas mailing list -- python-ideas@python.org

To unsubscribe send an email to python-ideas-le...@python.org

https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at

https://mail.python.org/archives/list/python-ideas@python.org/message/YKLNQXONLLZ7OXEMUHXF5HD4PCX4SNVT/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PQSJ6PSRS3JZZUV7CRRKRSGJAPVBYP6G/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JFSNV7FAKLMTZWQDHJWPX3ZSMKVHA6V3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce 100 more built-in exceptions

[Soni L.]

On 2020-05-01 3:48 a.m., Ram Rachum wrote:

Hi,

Here's something I wanted in Python for many years. If this has been 
discussed in the past, please refer me to that discussion.


On one hand, it's something that I can't imagine the python-dev 
community supporting. On the other hand, it would maintain backward 
compatibility.


I wish there were a 100 more built-in exceptions in Python, that will 
be very specific about what went wrong.


If I do this:

    >>> x, y = range(3)

I know it'll raise a ValueError, because I've memorized that, but it 
did take me a few years to remember where I should expect ValueError 
and where I should expect TypeError.


It would be nice if the operation above raised UnpackingOverflowError, 
which will be a subclass of UnpackingError, along with 
UnpackingUnderflowError. UnpackingError can be a subclass of 
ValueError, for backward compatibility.


oh yes please let me distinguish the exception from the unpacking and 
the exception from the iterator


def foo():
  yield from ()
  raise ValueError

it = foo()
try:
  x, y = it
except ValueError:
  print("Is this a problem with the unpacking or a problem with the 
generator?")

  raise

also please have unpacking wrap any UnpackingError from the generator 
into a RuntimeError.




Similarly, if I did this:

    >>> def f(x, y): return x + y
    >>> f(1)

I would get a TypeError. Would be a lot cooler if I got 
MissingArgumentsError, which would be a subclass of SignatureError, 
which would be a subclass of TypeError.


There are 2 reasons I want this:

1. When I'm writing a try..except clause, I want to catch a specific 
exception like MissingArgumentsError rather than ValueError or 
TypeError. They're too ubiquitous. I don't want some other unexpected 
failure producing the same ValueError and triggering my except clause.


2. When I get an error, especially from some shitty corporate system 
that truncates the traceback, I want to get as many hints as possible 
about what went wrong.


It's true that today, most Python exceptions have good text in their 
message, like "TypeError: f() missing 1 required positional argument: 
'y'". But that isn't guaranteed everywhere, and specific exception 
types could help.



What do you think?


Ram.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XZXWXICKS2RCQLLX73NJOWCJPRY7IUX2/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LK5QSTJCABDGH72333T7GFRF3DWMW36O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip(x, y, z, strict=True)

[Soni L.]

On 2020-04-30 1:07 p.m., Ethan Furman wrote:

On 04/30/2020 07:58 AM, Christopher Barker wrote:

On 04/29/2020 10:51 PM, Stephen J. Turnbull wrote:


I think that the issue of searchability and signature are pretty
compelling reasons for such a simple feature to be part of the
function name.


I would absolutely agree with that if all three function were in the 
same namespace (like the string methods referred to earlier), but in 
this case, one is a built in and the others will not be — which makes 
a huge difference in discoverability.


Imagine someone that uses zip() in code that works for a while, and 
then discovers a bug triggered by unequal length inputs.


If it’s a flag, they look at the zip docstring, and find the flag, 
and their problem is solved.


So update the `zip` docstring with a reference to `zip_longest`, 
`zip_equal`, and `zip_whatever`.


-1 on using a flag.


what about letting `zip` take a `leftover_func` with arguments 
`partial_results` and `remaining_iterators`, and then provide 
`zip_longest`, `zip_equal` and `zip_shortest` (default) as functions you 
can use with it?


an iteration of `zip(a, b, c, leftover_func=foo)` would:

1. call next on the first iterator (internal iter(a))
2. if it fails, call leftover_func with the () tuple as first arg and 
the (internal iter(b), internal iter(c)) tuple as second arg

3. call next on the second iterator (internal iter(b))
4. if it fails, call leftover_func with the (result from a,) tuple as 
the first arg and the (internal iter(a), internal iter(c)) tuple as 
second arg

5. call next on the third iterator (internal iter(c))
6. if it fails, call leftover_func with the (result from a, result from 
b) tuple as the first arg and the (internal iter(a), internal iter(b)) 
tuple as second arg

7. yield the (result from a, result from b, result from c) tuple

the leftover_func should return an iterator that replaces the zip, or 
None. (zip_shortest would be the no_op function)




--
~Ethan~
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T7OGRPXEU2UHGGL6FW42DIK7ZVHCMDUS/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KKHW4ZUMIFPUVPQVWQD7KAHGSGBCCE6H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

On 2020-04-28 12:32 p.m., Edwin Zimmerman wrote:

On April 28, 2020 9:38 AM Soni L. wrote:
> On 2020-04-28 7:50 a.m., Edwin Zimmerman wrote:
> > On 4/27/2020 11:47 PM, Soni L. wrote:
> > [snip]
> > > tbh my particular case doesn't make a ton of practical sense. I have 
config files and there may be errors opening or
deserializing
> them, and I have a system to manage configs and overrides. which means you 
can have multiple config files, and you may wanna log
> errors. you can also use a config manager as a config file in another config 
manager, which is where the error logging gets a bit
weirder.
> I'm currently returning lists of errors, but another option would be to yield 
the errors instead. but, I'm actually not sure what
the best
> approach here is. so yeah. I can't *use* that motivating example, because if 
I did, everyone would dismiss me as crazy. (which I
am,
> but please don't dismiss me based on that :/)
> > >
> > Maybe you could start by actually writing the code, and then demonstrate 
how your idea would make the code easier to read or
> understand, or less error prone, or whatever other improvement it would bring.
> 
> that thing about wrapping things from other generators and returning the

> number of generators I messed with?
> 
> yeah. that's what I wanted to do. it doesn't do any of those things,

> it's just fun.


I think I will stop replying to this thread, since we aren't getting anywhere.  
Either I'm a poor communicator or someone else is a
poor listener.  Either way, my attempts to constructively approach this thread 
are failing.
--Edwin



this code:

def foo(self):
  for x in self.things:
    for y in x.foo():
  yield Wrap(y)
    else with z:
  yield z
  return len(things)

this is what it'd look like. and it'd be fun. the point of it is to be 
fun. it's a nuisance to actually use the results but if you're just 
printing them out (which is what I'd do) it's fine.


if you don't want me telling you, maybe don't ask.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7LPHFR7WGB6WNFA77E3MGKSYABR3WTSX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

On 2020-04-28 7:50 a.m., Edwin Zimmerman wrote:

On 4/27/2020 11:47 PM, Soni L. wrote:
[snip]
> tbh my particular case doesn't make a ton of practical sense. I have config 
files and there may be errors opening or deserializing them, and I have a system 
to manage configs and overrides. which means you can have multiple config files, 
and you may wanna log errors. you can also use a config manager as a config file 
in another config manager, which is where the error logging gets a bit weirder. 
I'm currently returning lists of errors, but another option would be to yield the 
errors instead. but, I'm actually not sure what the best approach here is. so 
yeah. I can't *use* that motivating example, because if I did, everyone would 
dismiss me as crazy. (which I am, but please don't dismiss me based on that :/)
>
Maybe you could start by actually writing the code, and then demonstrate how 
your idea would make the code easier to read or understand, or less error 
prone, or whatever other improvement it would bring.


that thing about wrapping things from other generators and returning the 
number of generators I messed with?


yeah. that's what I wanted to do. it doesn't do any of those things, 
it's just fun.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PUTKHWMUUVLSMFK66LPM3DHBF4TFFFHM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

On 2020-04-28 12:28 a.m., Andrew Barnert wrote:

On Apr 27, 2020, at 17:01, Soni L.  wrote:
> 
>>> On 2020-04-27 8:37 p.m., Andrew Barnert wrote:

>>> On Apr 27, 2020, at 14:38, Soni L.  wrote:
>> [snipping a long unanswered reply]
>>> The explicit case for zip is if you *don't* want it to consume anything 
after the stop.
>> Sure, but *when do you want that*? What’s an example of code you want to 
write that would be more readable, or easier to write, or whatever, if you could work 
around consuming anything after the stop?
> 
> so here's one example, let's say you want to iterate multiple things (like with zip), get a count out of it, as well as partially consume an external iterator without swallowing any extra values from it.


What do you want to do that for? This still isn’t a concrete use case, so it’s still 
not much more of a rationale than “let’s say you want to intermingle the bits of two 
16-bit integers into a 32-bit integer”. Sure, that’s something that’s easy to do in 
some other languages (it’s the builtin $ operator in INTERCAL) but very hard to do 
readably or efficiently in Python. If we added a $ operator with a __bigmoney__ 
protocol and made int.__bigmoney__ implement this operation in C, that would 
definitely solve the problem. But it’s only worth proposing that solution if anyone 
actually needs a solution to the problem in the first place. When’s the last time 
anyone ever needed to efficiently intermingle bits? (Except in INTERCAL, where the 
language intentionally leaves out useful operators like +, |, and << and even 
32-bit literals to force you to write things in clever ways around $ and ~ instead).


(OT: Z-order curves. they're amazing.)



On top of that, this abstract example you want can already be written today.

> it'd look something like this:
> 
>def foo(self, other_things):

>  for x in zip(range(sys.maxsize), self.my_things, other_things):
>do_stuff
>  else as y:
>return y[0] # count

> using extended for-else + partial-zip. it stops as soon as self.my_things 
stops. and then the caller can do whatever else it needs with other_things. (altho 
maybe it's considered unpythonic to reuse iterators like this? I like it tho.)

Here are four ways of doing this today:

def foo(self, other_things):
for x in zip(count(1), self.my_things, other_things):
do_stuff
return x[0]

def foo(self, other_things):
c = count(-1)
for x in zip(c, self.my_things, other_things):
do_stuff
return next(c)

def foo(self, other_things):
c = count()
for x in zip(self.my_things, other_things, c):
do_stuff
return next(c)

def foo(self, other_things):
c = lastable(count())
for x in zip(c, self.my_things, other_things):
do_stuff
return c.last

So, why do we need another way to do something that’s probably pretty uncommon 
and can already be done pretty easily? Especially if that new way isn’t more 
readable or more powerful?


the only one with equivalent semantics is the last one.



> if anything my motivating example is because I wanna do some very unpythonic 
things.

Then you should have given that example in the first place.

Sure, the fact that it’s unpythonic might mean it’s not very convincing, but it 
doesn’t become more convincing after multiple people have to go back and forth 
to drag it out of you. All that means is that everyone else has already tuned 
out and won’t even see your example, so your proposal has basically zero chance 
instead of whatever chance it should have had.

And sometimes unpythonic things really do get into the language—sometimes 
because they’re just so useful, but more often, because they point to a reason 
for changing what everyone’s definition of “pythonic” is. Think of the abc 
module. Or, better, if you can dig up the 3.1-era vs. 3.3-era threads on the 
original coroutine PEP 3152, you can see how the consensus changed from “wtf, 
that doesn’t look like Python at all and nobody will ever understand it” to 
“this is obviously the pythonic way to write reactors (modulo a bunch of 
bikeshedding)”. That wouldn’t have happened if Greg Ewing had refused to tell 
anyone that he wanted coroutines to provide a better, if unfamiliar, way to 
write things like reactors, and instead tried to come up with 
less-unpythonic-looking but completely useless examples.


tbh my particular case doesn't make a ton of practical sense. I have 
config files and there may be errors opening or deserializing them, and 
I have a system to manage configs and overrides. which means you can 
have multiple config files, and you may wanna log errors. you can also 
use a config manager as a config file in another config manager, which 
is where the error logging gets a bit weirder. I'm currently returning 
lists of errors, but anot

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

On 2020-04-27 8:37 p.m., Andrew Barnert wrote:

On Apr 27, 2020, at 14:38, Soni L.  wrote:

[snipping a long unanswered reply]

> The explicit case for zip is if you *don't* want it to consume anything after 
the stop.

Sure, but *when do you want that*? What’s an example of code you want to write 
that would be more readable, or easier to write, or whatever, if you could work 
around consuming anything after the stop?


so here's one example, let's say you want to iterate multiple things 
(like with zip), get a count out of it, as well as partially consume an 
external iterator without swallowing any extra values from it. it'd look 
something like this:


    def foo(self, other_things):
  for x in zip(range(sys.maxsize), self.my_things, other_things):
    do_stuff
  else as y:
    return y[0] # count

using extended for-else + partial-zip. it stops as soon as 
self.my_things stops. and then the caller can do whatever else it needs 
with other_things. (altho maybe it's considered unpythonic to reuse 
iterators like this? I like it tho.)




> btw: I suggest reading the whole post as one rather than trying to pick it 
apart.

I did read the whole post, and then went back to reply to each part in-line. 
You can tell by the fact that I refer to things later in the post. For example, 
when I refer to your proposed code being better than “the ugly mess that you 
posted below“ as the current alternative, it should be pretty clear that I’ve 
already read the ugly mess that you posted below.

So why did I format it as replies inline? Because that’s standard netiquette 
that goes back to the earliest days of email lists. Most people find it 
confusing (and sometimes annoying) to read a giant quote and then a giant reply 
and try to figure out what’s being referred to where, so when you have a giant 
message to reply to, it’s helpful to reply inline.

But as a bonus, writing a reply that way makes it clear to yourself if you’ve 
left out anything important. You didn’t reply to multiple issues that I raised, 
and I doubt that it’s because you don’t have any answers and are just trying to 
hide that fact to trick people into accepting your proposal anyway, but rather 
than you just forgot to get to some things because it’s easy to miss important 
stuff when you’re not replying inline.


you kept bringing up how I should talk about things first and break them 
down, rather than build them up and expand on them as the post goes on. 
I prefer the latter. I don't mind inline replies, and in fact I prefer 
them (altho I'm not always great at that), and that's not what I raise 
an issue with.




> the purpose of the proposal, as a whole, is to make it easier to pick things 
- generators in particular - apart. I tried to make that clear but clearly I 
failed.

No, you did make that part clear; what you didn’t make clear is (a) what 
exactly you’re trying to pick apart from the generators and why, (b) what 
actual problems look like, (c) how your proposal could make that code better, 
and (d) why existing solutions (like manually nexting iterators in a while 
loop, or using tools like peekable) don’t already solve the problem.

Without any of that, all you’re doing is offering something abstract that might 
conceivably be useful, but it’s not clear where or why or even whether it would 
ever come up, so for all we know it’ll *never* actually be useful. Nobody’s 
likely to get on board with such a change.

> Side note, here's one case where it'd be better than using zip_longest:

Your motivating example should not be a “side note”, it should be the core of 
any proposal.


that is not my motivating example. if anything my motivating example is 
because I wanna do some very unpythonic things.


like this:

for x in things:
  yield Wrap(x)
else with y:
  yield y
return len(things)

and then we nest this and we get a nice wrap of wraps wrapped in wraps 
with lengths at the end. why? ... because I want it to work like this, 
tbh. .-.




But it should also be a real example, not a meaningless toy example. Especially 
not one where even you can’t imagine an actual similar use case. “We should add 
this feature because it would let you write code that I can’t imagine ever 
wanting to write” isn’t a rationale that’s going to attract much support.

> for a, b, c, d, e, f, g in zip(*[iter(x)]*7): # this pattern is suggested by 
the zip() docs, btw.
>use_7x_algorithm(a, b, c, d, e, f, g)
> else as x: # leftovers that didn't fit the 7-tuple.
>use_slow_variable_arity_algorithm(*x)

Why do you want to unpack into 7 variables with meaningless names just to pass 
those 7 variables? And if you don’t need that part, why can’t you just write 
this with zip_skip (which, as mentioned in the other thread, is pretty easy to 
write around zip_longest)?

The best guess I can come up with is that in a real life example maybe that 
would have some performance cost that’s hard to see in this 

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

the point of posting here is that someone else may have a similar 
existing use-case where this would make things better. I can't take a 
look at proprietary code so I post about stuff in the hopes that the ppl 
who can will back this stuff up.


(doesn't proprietary software make things so much harder? :/)

I don't want ppl to do homework for me. I want to access the inaccessible.

(but also yes, it'd be cool if this list was about cooperating on ideas.)

On 2020-04-27 8:26 p.m., Edwin Zimmerman wrote:

I doubt that you will find many people on this list who are willing to do your homework 
for you.  It will be very hard to convince most of the people on this list if the only 
reason you can give is "I think this would look great".  Great ideas are based 
on real needs, not on flights of fancy.

On 4/27/2020 7:15 PM, Soni L. wrote:
> the point of posting here is to find other use-cases! there's a reason this 
isn't a PEP!
>
> On 2020-04-27 7:14 p.m., Chris Angelico wrote:
>> On Tue, Apr 28, 2020 at 7:41 AM Soni L.  wrote:
>> > I haven't found a real use-case for this yet, tho.
>>
>> So maybe hold off until you have one?
>>
>> ChrisA
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6PP3NQJPRX3FCGUPADCQOPYBVXFU5MZ5/
>> Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F5HFUOC4LYXT3AAUG5ACVYBRVDDW54DK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KG3OBXFXZ44FCYIZK36D6PDTUFFOBIUA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

the point of posting here is to find other use-cases! there's a reason 
this isn't a PEP!


On 2020-04-27 7:14 p.m., Chris Angelico wrote:

On Tue, Apr 28, 2020 at 7:41 AM Soni L.  wrote:
> I haven't found a real use-case for this yet, tho.

So maybe hold off until you have one?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6PP3NQJPRX3FCGUPADCQOPYBVXFU5MZ5/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F5HFUOC4LYXT3AAUG5ACVYBRVDDW54DK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: extended for-else, extended continue, and a rant about zip()

[Soni L.]

On 2020-04-27 6:11 p.m., Andrew Barnert wrote:

On Apr 27, 2020, at 12:49, Soni L.  wrote:
> 
> I wanna propose making generators even weirder!


Why? Most people would consider that a negative, not a positive. Even if you 
demonstrate some useful functionality with realistic examples that benefit from 
it, all you’ve done here is set the bar higher for yourself to convince anyone 
that your change is worth it.

> so, extended continue is an oldie: 
https://www.python.org/dev/peps/pep-0342/#the-extended-continue-statement
> 
> it'd allow one to turn:
> 
> yield from foo
> 
> into:
> 
> for bar in foo:

> continue (yield bar)

And what’s the advantage of that? It’s a lot more verbose, harder to read, 
probably easier to get wrong, and presumably less efficient. If this is your 
best argument for why we should revisit an old rejected idea, it’s not a very 
good one.

(If you’re accepting that it’s a pointless feature on its own but proposing it 
because, together with your other proposed new feature, it would no longer be 
pointless, then say that, don’t offer an obviously bad argument for it on its 
own.)

> but what's this extended for-else? well, currently you have for-else:
> 
> for x, y, z in zip(a, b, c):

> ...
> else:
> pass
> 
> and this works. you get the stuff from the iterators, and if you break the loop, the else doesn't run. the else basically behaves like "except StopIteration:"...
> 
> so I propose an extended for-else, that behaves like "except StopIteration as foo:". that is, assuming we could get a zip() that returns partial results in the StopIteration (see other threads), we could do:
> 
> for x, y, z in zip(a, b, c):

> do_stuff_with(x, y, z)
> else as partial_xy:
> if len(partial_xy) == 0:
> x = dummy
> try:
> y = next(b)
> except StopIteration: y = dummy
> try:
> z = next(c)
> except StopIteration: z = dummy
> if (x, y, z) != (dummy, dummy dummy):
> do_stuff_with(x, y, z)
> if len(partial_xy) == 1:
> x, = partial_xy
> y = dummy
> try:
> z = next(c)
> except StopIteration: z = dummy
> do_stuff_with(x, y, z)
> if len(partial_xy) == 2:
> x, y = partial_xy
> z = dummy
> do_stuff_with(x, y, z)
> 
> (this example is better served by zip_longest. however, it's nevertheless a good way to demonstrate functionality, thanks to zip_longest's (and zip's) trivial/easy to understand behaviour.)


Would it always be this complicated and verbose to use this feature? I mean, 
compare it to the “roughly equivalent” zip_longest in the docs, which is a lot 
shorter, easier to understand, harder to get wrong, and more flexible (e.g., it 
works unchanged with any number of iterables, while yours to had to rewritten 
for any different number of iterables because it requires N! chunks of explicit 
boilerplate).

Are there any examples where it lets you do something useful that can’t be done 
with existing features, so it’s actually worth learning this weird new feature 
and requiring Python 3.10+ and writing 22 lines of extra code?

Even if there is such an example, if the code to deal with the post-for state 
is 11x as long and complicated as the for loop and can’t be easily simplified 
or abstracted, is the benefit of using a for loop instead of manually nexting 
iterators still a net benefit? I don’t know that manually nexting the iterators 
will always avoid the problem, but it certainly often is (again, look at many 
of the equivalents in the itertools docs that do it), and it definitely is in 
your emulating-zip_longest example, and that’s the only example you’ve offered.

Also notice that many cases like this can be trivially solved by a simple 
peekable or unnextable (I believe more-itertools has both, and the first one is 
a recipe in itertools too, but I can’t remember the names they use; if not, 
they’re really easy to write) or tee. We don’t even need any of that for your 
example, but if you can actually come up with another example, make sure it 
isn’t already doable a lot more simply with peekable/etc.

> this would enable one to turn:
> 
> return yield from foo
> 
> into:
> 
> for bar in foo:

> continue (yield bar)
> else as baz:
> return baz
> 
> allowing one to pick apart and modify the yielded and sent parts, while still getting access to the return values.


Again, this is letting you turn something simple into something more 
complicated, and it’s not at all clear why you want to do that. What exactly 
are you trying to pick apart that makes that n

[Python-ideas] extended for-else, extended continue, and a rant about zip()

[Soni L.]

I wanna propose making generators even weirder!

so, extended continue is an oldie: 
https://www.python.org/dev/peps/pep-0342/#the-extended-continue-statement


it'd allow one to turn:

    yield from foo

into:

    for bar in foo:
    continue (yield bar)

but what's this extended for-else? well, currently you have for-else:

    for x, y, z in zip(a, b, c):
    ...
    else:
    pass

and this works. you get the stuff from the iterators, and if you break 
the loop, the else doesn't run. the else basically behaves like "except 
StopIteration:"...


so I propose an extended for-else, that behaves like "except 
StopIteration as foo:". that is, assuming we could get a zip() that 
returns partial results in the StopIteration (see other threads), we 
could do:


    for x, y, z in zip(a, b, c):
    do_stuff_with(x, y, z)
    else as partial_xy:
    if len(partial_xy) == 0:
    x = dummy
    try:
    y = next(b)
    except StopIteration: y = dummy
    try:
    z = next(c)
    except StopIteration: z = dummy
    if (x, y, z) != (dummy, dummy dummy):
    do_stuff_with(x, y, z)
    if len(partial_xy) == 1:
    x, = partial_xy
    y = dummy
    try:
    z = next(c)
    except StopIteration: z = dummy
    do_stuff_with(x, y, z)
    if len(partial_xy) == 2:
    x, y = partial_xy
    z = dummy
    do_stuff_with(x, y, z)

(this example is better served by zip_longest. however, it's 
nevertheless a good way to demonstrate functionality, thanks to 
zip_longest's (and zip's) trivial/easy to understand behaviour.)


this would enable one to turn:

    return yield from foo

into:

    for bar in foo:
    continue (yield bar)
    else as baz:
    return baz

allowing one to pick apart and modify the yielded and sent parts, while 
still getting access to the return values.


currently if you have an arbitrary generator, you can't modify the 
yielded values without breaking send or return. in fact you can either 
pass-through yield and send and collect the return, or modify yield and 
forget about send and the return, or use very ugly syntax that makes 
everything look lower-level than it should, to be able to pick apart 
everything:


    try:
    bar = next(foo)
    except StopIteration as exc:
    baz = exc.value
    else:
    while True:
    try:
    bar = foo.send((yield bar))
    except StopIteration as exc:
    baz = exc.value
    break
    return baz

(this is exactly equivalent (if I didn't overlook anything) to the 
previous endearingly simple loop with extended for-else and extended 
continue)

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4NZXWPOCV6CSM3CBIN4UDQGRWCVENCTY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make type(None) the no-op function

[Soni L.]

On 2020-04-24 12:26 p.m., Chris Angelico wrote:

On Sat, Apr 25, 2020 at 1:20 AM Soni L.  wrote:
>
> Currently type(None) returns None if you call it with no args:
>
>  >>> print(type(None)())
> None
>
> it'd be nice if it did the same regardless of args. currently it raises:
>
>  >>> print(type(None)(1))
> Traceback (most recent call last):
>File "", line 1, in 
> TypeError: NoneType takes no arguments
>
> inspired by PEP 559: https://www.python.org/dev/peps/pep-0559/
>
>  >>> NoneType
> Traceback (most recent call last):
>File "", line 1, in 
> NameError: name 'NoneType' is not defined

Why should NoneType be any different from other types? If you want a
null function, just create one:

def return_none(*_a, **_kw): pass

What is your use-case for wanting NoneType to behave in this way? Is
this yet another idea that actually has no use-case and is just
"wouldn't it be nice if we did weird stuff to the language"? I'm
starting to have to read python-ideas by first looking at the poster
and only then consider the content of the suggestion, instead of what
I usually prefer, which is to evaluate an idea on its merits
regardless of who posted it.


it's not my own use case for once. the PEP clearly lists a use-case. we 
should support that use-case.




ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KHRYFINMX4UW5NLB55UJEZT67GUOVK6C/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L2EZBCGOVOCXYI46OXHHPZDFPWJ3IU3P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Make type(None) the no-op function

[Soni L.]

Currently type(None) returns None if you call it with no args:

>>> print(type(None)())
None

it'd be nice if it did the same regardless of args. currently it raises:

>>> print(type(None)(1))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: NoneType takes no arguments

inspired by PEP 559: https://www.python.org/dev/peps/pep-0559/

>>> NoneType
Traceback (most recent call last):
  File "", line 1, in 
NameError: name 'NoneType' is not defined
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7NZM6XGGX2OUZVG6HWJU43BXCCX5QYLX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-22 11:08 a.m., Soni L. wrote:



On 2020-04-22 6:47 a.m., Steven D'Aprano wrote:

On Wed, Apr 22, 2020 at 01:43:10AM -0300, Soni L. wrote:

> are there *any* solutions for getting partial results out of zip 
with > different-length iterators of unknown origin?


No. If you want to continue getting results even when one or more of
the iterators is exhausted, `zip` is the wrong tool. Use `zip_longest`.


> >> 3. I feel like it makes intuitive sense for zip to return a 
partial > >> result somehow, especially now that StopIteration and 
return work > >together.

[...]

> it makes intuitive sense that there'd be some way to get the 
iterator > elements silently swallowed by zip against the wishes of 
the programmer.

If you use zip, it is because you want zip's behaviour, which is
explicitly documented as truncating on the shortest iterator. (Either
that or you don't pay attention to the docs.)

You can't say "its against the wishes of the programmer". Nobody is
forcing you to use zip except yourself. If you are hammering nails with
a screwdriver, that's not the screwdriver's fault.

As I said, if you want to continue collecting values even after one or
more iterators are exhausted, use zip_longest.


> there isn't. so we avoid it because it's useless for anything more 
> advanced than the "you've checked the lengths beforehand" use-case.

I frequently use zip all the time with infinite iterators, or one
infinite iterator like itertools.count() and one or more finite
iterators, or more than one finite iterator, and I've never checked
their lengths ahead of time.

Your statement that zip is "useless" unless you have checked the
lengths first is simply nonsense.



I have, more than once, had a desire to use zip to partially consume 
an iterator from a set of iterators, and then pass the rest onto 
something else. it'd fit in with that.


it's a small quality of life improvement that'd make zip even more 
useful than it is today, and I'm pretty sure it can't break anything 
because zip (as is documented) already puts None in the StopIteration. 
(whether the implementation follows that is another story, I haven't 
checked, but that'd be even more of a reason to change this.) not sure 
why so much pushback from you.


I'm gonna try to use zip_longest for my thing, but I don't see that 
stopping me from pushing for this "return tuple(result)" change. 
especially considering (as I just checked) that the CPython 
implementation of zip() doesn't agree with the documentation.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5ATC4APJ6IBHBXT6FBJZMDNSSCMZJOCK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-22 6:47 a.m., Steven D'Aprano wrote:

On Wed, Apr 22, 2020 at 01:43:10AM -0300, Soni L. wrote:

> are there *any* solutions for getting partial results out of zip with 
> different-length iterators of unknown origin?


No. If you want to continue getting results even when one or more of
the iterators is exhausted, `zip` is the wrong tool. Use `zip_longest`.


> >> 3. I feel like it makes intuitive sense for zip to return a partial 
> >> result somehow, especially now that StopIteration and return work 
> >together.

[...]

> it makes intuitive sense that there'd be some way to get the iterator 
> elements silently swallowed by zip against the wishes of the programmer. 


If you use zip, it is because you want zip's behaviour, which is
explicitly documented as truncating on the shortest iterator. (Either
that or you don't pay attention to the docs.)

You can't say "its against the wishes of the programmer". Nobody is
forcing you to use zip except yourself. If you are hammering nails with
a screwdriver, that's not the screwdriver's fault.

As I said, if you want to continue collecting values even after one or
more iterators are exhausted, use zip_longest.


> there isn't. so we avoid it because it's useless for anything more 
> advanced than the "you've checked the lengths beforehand" use-case. 


I frequently use zip all the time with infinite iterators, or one
infinite iterator like itertools.count() and one or more finite
iterators, or more than one finite iterator, and I've never checked
their lengths ahead of time.

Your statement that zip is "useless" unless you have checked the
lengths first is simply nonsense.



I have, more than once, had a desire to use zip to partially consume an 
iterator from a set of iterators, and then pass the rest onto something 
else. it'd fit in with that.


it's a small quality of life improvement that'd make zip even more 
useful than it is today, and I'm pretty sure it can't break anything 
because zip (as is documented) already puts None in the StopIteration. 
(whether the implementation follows that is another story, I haven't 
checked, but that'd be even more of a reason to change this.) not sure 
why so much pushback from you.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ULMVQC4JBPRFT3VB7WCQ2TKFMCPRWTFO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-21 7:12 p.m., Steven D'Aprano wrote:

On Tue, Apr 21, 2020 at 05:33:24PM -0300, Soni L. wrote:

> 1. see the other thread (strict zip), especially the parts where they 
> brought up the lack of peekable/unput iterators in the context of 
> getting a count out of an iterator.


I've seen it, as far as I can tell there are already good solutions
for getting a count out of an iterator.


are there *any* solutions for getting partial results out of zip with 
different-length iterators of unknown origin?



> 2. it'd help me clean up some of my code. :p

I don't see how it would clean up code rather than make it uglier. Can
you give an example?

> 3. I feel like it makes intuitive sense for zip to return a partial 
> result somehow, especially now that StopIteration and return work together.


With respect Soni, your intuitions do not have a good track record of
either being shared by other people, or of making good programming
features.

Maybe it's time for you to stop suggesting things that make "intuitive
sense" and start putting some careful, logical thought into the proposal
instead of going by intuition?





it makes intuitive sense that there'd be some way to get the iterator 
elements silently swallowed by zip against the wishes of the programmer. 
there isn't. so we avoid it because it's useless for anything more 
advanced than the "you've checked the lengths beforehand" use-case. I've 
recently posted my code on this list, something about a class 
LogAndCompare[1]. it's horrible. a bit of zip and it could be a bit 
nicer. but I need those partial results.


[1] 
https://creationix.github.io/http-clone/?https://soniex2.autistic.space/git-repos/abdl.git#cbdcd9dd71f3215520b40b0454cd21c03101bd33

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Z7FRZPJQ33C7VIHO2KSD4B3IZPDGCRZ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-21 12:48 p.m., Serhiy Storchaka wrote:

21.04.20 17:10, Soni L. пише:

I feel like zip could return partial results:

try:
   next(zip([0], []))
except StopIteration as exc:
   assert StopIteration.args == (0,)

how much would this break?


Why do you need this feature?


3 reasons:

1. see the other thread (strict zip), especially the parts where they 
brought up the lack of peekable/unput iterators in the context of 
getting a count out of an iterator.

2. it'd help me clean up some of my code. :p
3. I feel like it makes intuitive sense for zip to return a partial 
result somehow, especially now that StopIteration and return work together.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LKXZZEBXDBFFJ7K7MG6XPZYONQZYXRM5/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4U7AILTPE2SVEEODGLRDTP7WWT5GS2S5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-21 12:01 p.m., Soni L. wrote:



On 2020-04-21 11:20 a.m., Eric V. Smith wrote:

On 4/21/2020 10:14 AM, Eric V. Smith wrote:

On 4/21/2020 10:10 AM, Soni L. wrote:

I feel like zip could return partial results:

try:
  next(zip([0], []))
except StopIteration as exc:
  assert StopIteration.args == (0,)

how much would this break?


It would break a lot of code, and so it won't happen.
Actually, I'm not so sure of that, presuming you mean "exc.args", not 
"StopIteration.args". And since that's currently set to "()", at 
least in my tests, maybe setting it wouldn't break much code. But 
it's probably a non-zero amount.


... yeah I did not pay as much attention to that hypothetical code as 
I should have.





looking further into it, zip() already guarantees the StopIteration 
isn't gonna preserve the original StopIteration, so this should be a 
non-breaking change:


https://docs.python.org/3/library/functions.html#zip


Equivalent to:
def  zip(*iterables):
 # zip('ABCD', 'xy') --> Ax By
 sentinel  =  object()
 iterators  =  [iter(it)  for  it  in  iterables]
 while  iterators:
 result  =  []
 for  it  in  iterators:
 elem  =  next(it,  sentinel)
 if  elem  is  sentinel:
 return
 result.append(elem)
 yield  tuple(result)


my proposal is to change that "return" into "return tuple(result)".




___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BBDE2XE57JZOKT55WECYWYZCW7JLIWAN/

Code of Conduct: http://python.org/psf/codeofconduct/




___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R2RGM4GM44SYDLL3EVNY4JFGXIPCLUUC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip should return partial results in StopIteration

[Soni L.]

On 2020-04-21 11:20 a.m., Eric V. Smith wrote:

On 4/21/2020 10:14 AM, Eric V. Smith wrote:

On 4/21/2020 10:10 AM, Soni L. wrote:

I feel like zip could return partial results:

try:
  next(zip([0], []))
except StopIteration as exc:
  assert StopIteration.args == (0,)

how much would this break?


It would break a lot of code, and so it won't happen.
Actually, I'm not so sure of that, presuming you mean "exc.args", not 
"StopIteration.args". And since that's currently set to "()", at least 
in my tests, maybe setting it wouldn't break much code. But it's 
probably a non-zero amount.


... yeah I did not pay as much attention to that hypothetical code as I 
should have.




You can use itertools.zip_longest with your own sentinel to get the 
same behavior.


In your proposal, what would next(zip([0], [], [0])) result in?


This point still stands.


it'd be the partial results. by the time zip sees the StopIteration it 
has only seen the first iterator, so you get (0,) and the third iterator 
doesn't advance.


if you had stored the third iterator somewhere, you'd be able to call 
next on it directly as well.



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BBDE2XE57JZOKT55WECYWYZCW7JLIWAN/

Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CGBZDLSZCMC6C5HHN7ZN67RVFHBHRS7L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] zip should return partial results in StopIteration

[Soni L.]

I feel like zip could return partial results:

try:
  next(zip([0], []))
except StopIteration as exc:
  assert StopIteration.args == (0,)

how much would this break?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RSSOCK2CWM2MBC6E3SYHP3BDP457OKFX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: zip(x, y, z, strict=True)

[Soni L.]

+1. I implemented my own zip (because exceptions[1]) and it's so easy to 
accidentally have length-related bugs everywhere because your tests are 
just stopping short with no error.


[1] 
https://creationix.github.io/http-clone/?https://soniex2.autistic.space/git-repos/abdl.git#cbdcd9dd71f3215520b40b0454cd21c03101bd33


On 2020-04-20 2:42 p.m., Ram Rachum wrote:

Here's something that would have saved me some debugging yesterday:

    >>> zipped = zip(x, y, z, strict=True)

I suggest that `strict=True` would ensure that all the iterables have 
been exhausted, raising an exception otherwise.


This is useful in cases where you're assuming that the iterables all 
have the same lengths. When your assumption is wrong, you currently 
just get a shorter result, and it could take you a while to figure out 
why it's happening.


What do you think?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6GFUADSQ5JTF7W7OGWF7XF2NH2XUTUQM/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2ZFOV2SZOPVHTAWZH2FZVK6BGULBD2EG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict

[Soni L.]

On 2020-04-16 2:42 p.m., Andrew Barnert wrote:

On Apr 16, 2020, at 07:19, Soni L.  wrote:


 what about a dict.setdefaults (note the s) that takes in an 
iterable or a dict, and uses insert-or-ignore behaviour? 
(unfortunately a lot of iterables and all generators are hashable, 
and we can't break that.)


I could really use something like that tbh.


Then tell us the use cases that you have in mind.

I can think of lots of common uses for “override this dict with that 
one”, such as the typical “defaults, config file, environment, argv” 
thing that most command line tools do. But I can already write that, 
in idiomatic and readable code (and in fact with variations—update, |, 
or ChainMap); why would I want to be able to write the same thing in 
another way that’s conceptually backward, probably less efficient, and 
doesn’t work in Python 3.8? If the issue is just that you have those 
default values in a list of pairs rather than a dict, it’s trivial to 
just construct a dict from that.


Maybe there are good use cases where the usual way would feel backward 
or wrong and setdefaults would be nicer. But if so, you need to show some.


And they have to be pretty common. You can obviously already write 
that as `for k, v in defaults: d.setdefault(k, v)`, which is 
acceptable if it comes up once a year, but demands improvement if 
people are (or should be, if they only realized it) writing it all 
over the place.


(If the API were obviously symmetrical, the bar might be a lot lower. 
See the Haskell dict update operators—I’ll bet I’ve often used the 
“backward” ones and not even realized it, because they don’t look 
backward or unusual, and because Haskell in general encourages that 
kind of symmetrical thinking everywhere, to the extent that even “pass 
this arg to that  function” is almost as commonplace as “call this 
function with that arg”. But I don’t think Python could get there from 
here, even if it were a good idea.)


More generally, it seems like you think Python needs to include 
everything anyone can think of that might theoretically be useful 
somewhere. But it’s designed to only include (almost) everything 
people need regularly and can’t just trivially and build for 
themselves, because aiming for that sweet spot keeps Python 
discoverable and Python code readable—and similarly for Rust, Swift, 
Ruby, and other languages that attempt to go that way, even if none of 
them (including Python) pulls it off perfectly. A kitchen sink 
language is even worse than a no-batteries-included language—it isn’t 
just harder to design and implement, it’s harder to use. You have more 
docs to wade through to find anything, more subtle distinctions to 
think through to decide, more stuff to remember when you come back to 
the language after time spent doing other stuff, more trouble reading 
code written by other people who internalized a different subset of 
the kitchen sink, etc.


I prefer putting things in order of priority in my ConfigManagers and 
RepoListManagers and whatnot. mainly because in the RepoListManager, I 
keep the ConfigManager as the first thing in the list. wouldn't be fun 
if I had to keep it last, as I need to modify that list regularly.


I also just concatenate the iterators of all those configs and repo 
lists together :v

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OQ6O6CU65M5VK2NW32T2VHEBU4CRENL7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict

[Soni L.]

On 2020-04-16 11:43 a.m., Steven D'Aprano wrote:

On Thu, Apr 16, 2020 at 11:17:33AM -0300, Soni L. wrote:
> what about a dict.setdefaults (note the s) 


Do you have any idea of how much confusion and many silent errors will
be caused by having methods setdefault and setdefaults on the same
class?



we can't break setdefault (particularly for tuple keys), do you have a 
better idea?

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W6VMX375Z7CCAI64W2F3WZZBJ7HP4472/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict

[Soni L.]

what about a dict.setdefaults (note the s) that takes in an iterable or 
a dict, and uses insert-or-ignore behaviour? (unfortunately a lot of 
iterables and all generators are hashable, and we can't break that.)


I could really use something like that tbh.

On 2020-04-16 9:47 a.m., Alex Hall wrote:

I just tried playing with this idea:

from collections import UserDict


class InsertOrIgnoreDict(UserDict):
    __setitem__ = UserDict.setdefault


print(InsertOrIgnoreDict([(1, 2), (3, 4)]))

It caused an infinite chain of exceptions:

Traceback (most recent call last):
  File 
"/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", 
line 845, in setdefault

    return self[key]
  File 
"/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", 
line 1003, in __getitem__

    raise KeyError(key)
KeyError: 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/home/alex/.pyenv/versions/3.8.0/lib/python3.8/_collections_abc.py", 
line 845, in setdefault

    return self[key]
  File 
"/home/alex/.pyenv/versions/3.8.0/lib/python3.8/collections/__init__.py", 
line 1003, in __getitem__

    raise KeyError(key)
KeyError: 1

During handling of the above exception, another exception occurred:

...

I thought that was amusing.

Anyway, one could come up with infinitely many variations of 
potentially useful semantics. Why include any of them in the language 
instead of letting users implement what they need?


On Thu, Apr 16, 2020 at 2:37 PM Soni L. <mailto:fakedme%2...@gmail.com>> wrote:


currently dicts have insert-or-update semantics, e.g.:

 >>> dict([((), 1), ((), 2)])
{(): 2}

this is fine. however, in many cases insert or insert-or-ignore
are more
useful: (hypothetical examples)

 >>> InsertOrIgnoreDict([((), 1), ((), 2)])
{(): 1}

 >>> InsertDict([((), 1), ((), 2)])
Traceback: [...]
ValueError: key exists

these are useful when dealing with custom config file formats, config
overrides, among other things.

additionally we could also get an UpdateDict, a *view* into a dict
that
only allows updating existing entries:

 >>> a = {'a': 1}
 >>> b = {'a': 2, 'b': 3}
 >>> UpdateDict(a).update(b)
 >>> a
{'a': 2}

but I don't see an obvious usefulness to this one. it'd just be for
consistency with other SQL-esque dict wrappers (like the above
InsertOrIgnoreDict and InsertDict).
___
Python-ideas mailing list -- python-ideas@python.org
<mailto:python-ideas@python.org>
To unsubscribe send an email to python-ideas-le...@python.org
<mailto:python-ideas-le...@python.org>
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at

https://mail.python.org/archives/list/python-ideas@python.org/message/7PCWDORFLBFY6HRLNNS6UBL2CRER26SM/
Code of Conduct: http://python.org/psf/codeofconduct/



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/44PKB5DAEJO7F2CEVK4FY66L4ZJPC6EM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] collections.UpdateDict, collections.InsertDict, collections.InsertOrIgnoreDict

[Soni L.]

currently dicts have insert-or-update semantics, e.g.:

>>> dict([((), 1), ((), 2)])
{(): 2}

this is fine. however, in many cases insert or insert-or-ignore are more 
useful: (hypothetical examples)


>>> InsertOrIgnoreDict([((), 1), ((), 2)])
{(): 1}

>>> InsertDict([((), 1), ((), 2)])
Traceback: [...]
ValueError: key exists

these are useful when dealing with custom config file formats, config 
overrides, among other things.


additionally we could also get an UpdateDict, a *view* into a dict that 
only allows updating existing entries:


>>> a = {'a': 1}
>>> b = {'a': 2, 'b': 3}
>>> UpdateDict(a).update(b)
>>> a
{'a': 2}

but I don't see an obvious usefulness to this one. it'd just be for 
consistency with other SQL-esque dict wrappers (like the above 
InsertOrIgnoreDict and InsertDict).

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7PCWDORFLBFY6HRLNNS6UBL2CRER26SM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exception spaces

[Soni L.]

On 2020-04-11 8:39 p.m., Dominik Vilsmeier wrote:


If I understand correctly, you want a way for distinguishing between 
exceptions that were explicitly and intentionally `raise`'ed as part 
of an API and exceptions that were unintentionally raised due to bugs. 
So for example:


    raise ValueError(f'Illegal value for xyz: {xyz}')  # part of the API
    foo['bug']  # this should have been 'bag' so it's a bug

In addition to that, the user of the API should be able to decide 
whether they let the API raise in their *exception space* or not.


I think you could realize this in today's Python by using `raise ... 
from espace` where espace is an instance of a custom exception and 
then check the resulting exception's `__cause__`. So for example:


    class ESpace(Exception):
    pass

    # I'm the user of an API and I want to distinguish their API 
errors from bugs.

    espace = ESpace()
    try:
        api_func(espace=espace)
    except KeyError as err:
        if err.__cause__ is espace:  # it's part of the API
            pass
        else:  # it's a bug
            pass

And the API functions would have to raise their exceptions explicitly 
from the provided `espace`:


    def api_func(espace=None):
        raise KeyError() from espace  # part of the API; sets the 
exception's __cause__ to `espace`
        foo['bug']  # here __cause__ will be None, just like if no 
`espace` had been provided


It's probably an abuse of the exception mechanism and also relies on a 
dunder, but for your own projects it could serve the purpose.




I figured something better instead. you can have a class ESpace, but you 
use it like so:


    espace = ESpace()

    try:
    foo(espace=espace)
    except espace.module.submodule.Exception:
    ...

e.g. for builtins:

    espace = ESpace()

    try:
    raise espace.ValueError
    except espace.ValueError:
    ...

and it dynamically creates subclasses of whatever you give it. I'm not 
sure how doable this is in current python, but it's super close to what 
I want. so hey if it works well, we can promote it to the stdlib? just 
need to encourage ppl not to check the type of their espace argument so 
you can silently swap the external one for the stdlib one and nothing 
breaks.


(still need a better way to pass it into operators but eh)



On 11.04.20 18:07, Soni L. wrote:
the reason I'm proposing this is that I like standard exception types 
having well-defined semantics. this "special, very fragile, syntax 
for the same purpose" doesn't take away from that, and instead just 
adds to it.


it's a way of having multiple, independent, "local" dynamic scopes. 
instead of just one global dynamic scope. and enables you to freely 
use standard exception types.


if anything it's closer to passing in a namespace with a bunch of 
standard exception types every time you wanna do stuff. which... I 
could also get behind tbh. an stdlib addition that dynamically 
exposes subclasses of the standard exception types, unique to that 
namespace instance. would still need some way of passing it in to 
operators and stuff tho.


___
Python-ideas mailing list --python-ideas@python.org
To unsubscribe send an email topython-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived 
athttps://mail.python.org/archives/list/python-ideas@python.org/message/HAT3Y77ONNJ5VKOFH6ZPFQ3LZXEQB2Q4/
Code of Conduct:http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X7YXXEBILZNA52UUDN7PC4R6RHVAIXGO/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/44WBWPOA36S2H5ZLUEFIZXUHG2AZGYHS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exception spaces

[Soni L.]

On 2020-04-11 1:01 p.m., David Mertz wrote:
It sounds like you've entirely missed the point of Python exceptions. 
You are right that they are a kind of dynamic scope. That's their 
entire reason for existing, not a bug.


In Python, we can handle an exception at the particular scope in which 
it is most appropriate and useful to handle it, and not worry about it 
elsewhere.


On the very rare occasions where I've stumbled on an exception being 
raised "for the wrong reason" the solution is ALWAYS either to make 
try/except surround less, or to subclass exceptions to be more specific.


If some library or framework is unwilling to subclass, they will be 
that much less willing to add special, very fragile, syntax for the 
same purpose.


the reason I'm proposing this is that I like standard exception types 
having well-defined semantics. this "special, very fragile, syntax for 
the same purpose" doesn't take away from that, and instead just adds to it.


it's a way of having multiple, independent, "local" dynamic scopes. 
instead of just one global dynamic scope. and enables you to freely use 
standard exception types.


if anything it's closer to passing in a namespace with a bunch of 
standard exception types every time you wanna do stuff. which... I could 
also get behind tbh. an stdlib addition that dynamically exposes 
subclasses of the standard exception types, unique to that namespace 
instance. would still need some way of passing it in to operators and 
stuff tho.




On Sat, Apr 11, 2020, 11:37 AM Soni L.

I'm not saying rust has the best solution. I don't like how it's
so explicit about error handling. sometimes letting it bubble up
is great. but rust doesn't have the problem of exception handling
being inherently dangerous, context-sensitive, and destructive to
application logic. python's exception handling is effectively a
form of dynamic scoping, as defined by wikipedia: "In some
languages, however, "part of a program" refers to "portion of run
time (time period during execution)", and is known as *dynamic
scope*.".



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HAT3Y77ONNJ5VKOFH6ZPFQ3LZXEQB2Q4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exception spaces

[Soni L.]

On 2020-04-11 12:52 p.m., Chris Angelico wrote:

On Sun, Apr 12, 2020 at 1:37 AM Soni L.  wrote:
> in current python, you can't safely handle exceptions, *because* of the 
remote chance of them being raised *by the wrong thing/for the wrong reason*.
>
> rust doesn't have that problem, as it effectively forces you to handle 
exceptions all the time, and is constantly wrapping and unwrapping exceptions in 
exceptions and whatnot.
>
> I'm not saying rust has the best solution.

You can't add 0.5 to a Python integers, because there's a remote
chance that you'll get OverflowError trying to convert it to float.
Some other language, in which all integers are signed 32-bit values,
doesn't have this problem, as it effectively forces you to handle
overflows all the time. Is that better than Python?

> I don't like how it's so explicit about error handling. sometimes letting it bubble up is great. but rust 
doesn't have the problem of exception handling being inherently dangerous, context-sensitive, and destructive to 
application logic. python's exception handling is effectively a form of dynamic scoping, as defined by 
wikipedia: "In some languages, however, "part of a program" refers to "portion of run time 
(time period during execution)", and is known as dynamic scope.".
>
> this is the whole problem.
>

You haven't yet proven that this is a problem. You haven't
demonstrated that your solution is, in any way, better than simply
subclassing the exceptions. Your proposal requires that every level in
the call stack explicitly handle exception scoping and that every
exception raising site magically know whether to scope the exception
or not; Python *already* allows the raise site to choose to use a
subclass of the exception type, which is a far more useful mechanism.

Your proposal is heavy on magic and very light on actually-usable
recommendations.


It is literally a problem I'm running into, in my code, today. and not 
something I can workaround. yes, I can (and do) limit the things I put 
within the "try". yes, I can (and do) use my own exceptions. yes, I can 
(and do) write unit tests and integration tests. but those are just 
mitigations, not proper solutions. they reduce the risk but don't 
eliminate it. why shouldn't we eliminate that risk?



I'm done here.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/54EVAE6E7HSD5RGKDWZUKDPC3ZFNUWEQ/
Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2P3DYLPEBWT3FZ4VCS7RKRYZEHYLPPZ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exception spaces

[Soni L.]

On 2020-04-11 10:46 a.m., Eric V. Smith wrote:

On 4/11/2020 9:38 AM, Soni L. wrote:



On 2020-04-11 10:27 a.m., Eric V. Smith wrote:
tl;dr: I show how the goal of Soni L's exception spaces can be 
addressed today, via less intrusive means. (Assuming I understand 
their proposal, that is.)


what's the point of having standard types if you're not supposed to 
use them because they mask bugs?


what if you end up with a nested call to your own library and catch 
it at the wrong place? (or, indeed, someone else also uses this 
technique but somewhere along the way a built-in exception is being 
caught)


you're just delaying the issue/shifting it around, instead of solving 
it once and for all.


You show no examples where your proposal addresses those issues but 
mine doesn't.


Your responses sound like the only thing you're interested in is 
having your proposal accepted, not in the problem that needs to be 
solved. The fact that you don't address some of my points (for 
example: changes to intermediate libraries, which your proposal 
requires) just reinforces this impression. As such, I'm going to drop 
out of this discussion.


yes, I absolutely do want my proposal accepted. I'd think anyone coming 
up with a proposal would want that. but that's not the point I'm trying 
to make.


in current python, you can't safely handle exceptions, *because* of the 
remote chance of them being raised *by the wrong thing/for the wrong 
reason*.


rust doesn't have that problem, as it effectively forces you to handle 
exceptions all the time, and is constantly wrapping and unwrapping 
exceptions in exceptions and whatnot.


I'm not saying rust has the best solution. I don't like how it's so 
explicit about error handling. sometimes letting it bubble up is great. 
but rust doesn't have the problem of exception handling being inherently 
dangerous, context-sensitive, and destructive to application logic. 
python's exception handling is effectively a form of dynamic scoping, as 
defined by wikipedia: "In some languages, however, "part of a program" 
refers to "portion of run time (time period during execution)", and is 
known as *dynamic scope*.".


this is the whole problem.



Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L5UDJMCLMQGPPSGK3OY65OJ3GFYIZI2T/

Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GJUCXNDVMSOW7AB53LLD6V7E54XDYXBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exception spaces

[Soni L.]

On 2020-04-11 10:27 a.m., Eric V. Smith wrote:
tl;dr: I show how the goal of Soni L's exception spaces can be 
addressed today, via less intrusive means. (Assuming I understand 
their proposal, that is.)


what's the point of having standard types if you're not supposed to use 
them because they mask bugs?


what if you end up with a nested call to your own library and catch it 
at the wrong place? (or, indeed, someone else also uses this technique 
but somewhere along the way a built-in exception is being caught)


you're just delaying the issue/shifting it around, instead of solving it 
once and for all.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TIHO3MVSLFMXEMKN2NUPIQ5VA2RT3UWM/
Code of Conduct: http://python.org/psf/codeofconduct/