[Python-ideas] Re: use type hints and slices to specify a valid numerical range, example: `Angle = int[0:361]`

[David Mertz]

On Sat, Aug 8, 2020 at 5:49 PM Michael Smith  wrote:

> This kind of thing is so powerful, and I would love to see tooling capable
> of it in the python ecosystem. I believe folks who say it's very hard to
> implement correctly, but I don't know that that's a good reason not to make
> the proposed range syntax legal for some primitive types like int. Tools
> like mypy can choose not to handle int[A:B] until they are ready, or
> forever, but someone else can still work on it.


As Steven notes, there *are* some compilers in static languages that do a
limited amount of this.  But the cases where it is *provable* that some
bounds are not exceeded are very limited, or take a great deal of work to
construct the proof.  I don't feel like Mypy or similar tools are going to
stray quite there.  And in those static languages, a great deal of the
time, they must give up and just insert assertions to raise exceptions if
the bounds are violated rather than *proving* the constraint.

I slightly regret using the randint() example because you and I can so very
easily recognize that `random.randint(0, 1_000_000_001)` MIGHT return a
result of a billion-and-one.  That was just to have something short.  But a
great many computations, even ones that mathematically HAVE bounds, are not
easy to demonstrate the bounds.  Let me try again, still slightly
hand-wavey.

def final_number(seed: int):
"Something mathy with primes, modulo arithmetic, factorization, etc"
x = seed
while some_condition(x):
while other_condition(x):
if my_predicate(x):
x += 1
elif other_predicate(x):
x += 2
   else:
   x -= 1

def my_fun(i: int[1:1_000_000]):
pass

my_fun(final_number(42))

Depending on what the called functions do, this is the sort of thing where
number theorists might prove bounds.  But the bounds might depend in weird,
chaotic ways on the seed.  For some it will fly off to infinity or negative
infinity.  For others it bounces around in a small cycle.  For others it
bounces around for billions of times through the loop within small bounds
until flying off.  It's really NOT that hard to write functions like that,
even without really realizing their oddness.

Yours, David...

P.S. Compilers can do shockingly clever things.  LLVM has an optimization
that took me by surprise.  I was playing around with it via Numba a few
years ago.  Actually, for teaching material; my intention was just to say
"Look, this is way faster than plain Python."  But here's the case I
stumbled on:

Folks all probably know the story of Gauss in elementary school in which
(at least in anecdote) the teacher assigned the students to add all the
integers from 1 to 100 so she could get some coffee or whatever.  Gauss had
the solution in a couple seconds to her annoyance.  Because sum(range(1,
N+1)) == (N*(N+1))/2 by some straightforward reasoning that most 10 year
olds don't see (and in the story, neither did the teacher).

Anyway, my silly example for my training students (i.e. working programmers
and scientists) was loop to 100, add latest, with and without Numba.  Then
do it up to a billion.  My intentions was "Numba is lots faster, but has a
minor JIT overhead" ... my discovery was that "LLVM figures out Gauss'
simplification and does it in constant time no matter the N.  After that I
looked at the LLVM bytecode to see, "Yup, it does."  The optimizer is
pretty smart about variations in writing the code in some slightly
different ways, but I don't know exactly what it would take to fool it into
missing the optimization.

I asked one of the LLVM developers how I could find out what optimizations
are there.  Their answer was "Become a core developer and study the source
code of LLVM." Apparently the Gauss thing isn't actually documented anyway
outside the code. :-)

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YZRSXAGZHD7XS7LKG6OIKFLUB5YQBRDZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: use type hints and slices to specify a valid numerical range, example: `Angle = int[0:361]`

[David Mertz]

Oops, obviously I meant:

def plusone(i: int[1:1_000_000_000]):
return i+1

random.seed(42)
for n in range(1_000_000):
  plusone(random.randint(1, 1_000_000_001))

Or a zillion other things. I can construct orbitals of Mandelbrot set that
may or may not be bounded. Or bounds that depend on the twin prime
conjecture. Or whatever. Mersenne Twister is just a non-obvious calculation
that we have convenient functions for.

On Sat, Aug 8, 2020, 1:28 AM David Mertz  wrote:

>
>
> On Sat, Aug 8, 2020, 1:12 AM Steven D'Aprano
>
>> Static languages often check what
>> bounds they can at compile time, and optionally insert bound checking
>> runtime code for ambiguous places.
>
>
> Yep. That's an assert, or it's moral equivalent.
>
> Here's a deterministic program using the hypothetical new feature.
>
> def plusone(i: int[1:1_000_000_000]):
> return i+1
>
> random.seed(42)
> for n in range(1_000_000):
>   random.randint(1, 1_000_000_001)
>
> Is this program type safe? Tell me by static analysis of Mersenne Twister.
>
> Or if you want to special case the arguments to randint, will, lots of
> things. Let's say a "random" walk on the integer number line where each
> time through the loop increments or decrements some (deterministic but hard
> to calculate) amount. After N steps are we within certain bounds?
>
>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SST7J7IVHW5ZWVKJSROIVBB5EEJVNT4B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: use type hints and slices to specify a valid numerical range, example: `Angle = int[0:361]`

[David Mertz]

On Sat, Aug 8, 2020, 1:12 AM Steven D'Aprano

> Static languages often check what
> bounds they can at compile time, and optionally insert bound checking
> runtime code for ambiguous places.


Yep. That's an assert, or it's moral equivalent.

Here's a deterministic program using the hypothetical new feature.

def plusone(i: int[1:1_000_000_000]):
return i+1

random.seed(42)
for n in range(1_000_000):
  random.randint(1, 1_000_000_001)

Is this program type safe? Tell me by static analysis of Mersenne Twister.

Or if you want to special case the arguments to randint, will, lots of
things. Let's say a "random" walk on the integer number line where each
time through the loop increments or decrements some (deterministic but hard
to calculate) amount. After N steps are we within certain bounds?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BUV7DSKL5NA3XX3V4WSD4BII5OI5ZZWC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: use type hints and slices to specify a valid numerical range, example: `Angle = int[0:361]`

[David Mertz]

On Sat, Aug 8, 2020, 12:18 AM Ricky Teachey

> Yes, it's hard in the sense that it would require solving the halting
>> problem.
>>
>
> That doesn't sound so hard. ;)
>
> Thanks for educating me. Could it at least be useful for:
>
> 1. Providing semantic meaning to code (but this is probably not enough
> reason on its own)
> 2. Couldn't it still be useful for static analysis during runtime? Not in
> cpython, but when the type hints are used in cython, for example?
>

Being static like CPython doesn't help. You cannot know statically what the
result of an arbitrary computation will be.

There are certainly languages with guards. For example, Python. I can
definitely write a function like this:

def small_nums(i: int):
assert 0 < i < 100
do_stuff(i)

x = small_nums(arbitrary_computation())

In concept, an annotation could be another way to spell an assertion. But I
don't think we need that.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/53RDLJ7BVQHCTDUSOZ4JLDWVJNVHXA3B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: use type hints and slices to specify a valid numerical range, example: `Angle = int[0:361]`

[David Mertz]

On Fri, Aug 7, 2020, 6:03 PM Paul Moore  wrote:

> > x: int[0:]  # any ints greater than or equal to zero would match, others
> would fail
> > x: int[:101]  # any ints less than 101 match
> > x: int[0:101:2]  # even less than 101
>
> I suspect the biggest issue with this is that it's likely to be
> extremely hard (given the dynamic nature of Python) to check such type
> assertions statically.


Yes, it's hard in the sense that it would require solving the halting
problem.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/REC5J4KEFJA6N3W4MFNTZFHQWIER2NTR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Inline Try-Except Clause

[David Mertz]

On Fri, Aug 7, 2020 at 11:32 AM Alex Hall  wrote:

> > def not_valid(instance, schema):
>
>> > try:
>> > return validate(instance, schema)
>> > except ValidationError as err:
>> > return str(err)
>
>
> David, your example involves capturing the exception which was deferred in
> the PEP:
> https://www.python.org/dev/peps/pep-0463/#capturing-the-exception-object
>

It does, I recognize that.  If I had the "exception ternary" that didn't
capture the exception, I'd probably just evaluate to a True instead in the
exception case... then go back to revalidate manually only in the
unexpected case of validity failing. Of course, if some nice looking syntax
did both, so much the better.  Even the simpler form would have helped me
though.

I also know it is better in my example to return the exception object
itself rather than just its stringification.  If I were publishing my API
more widely, rather than the internal use I had, I would have done that.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WSRJECQ3C6PIEBFWNQGKSLVPBLRO2TXM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Inline Try-Except Clause

[David Mertz]

>
> For an example. Anyone is free to use, but I'm not claiming it's
> necessarily the best.  This is from... well, probably not yesterday like I
> said in other comment, but a couple days ago. The module `jsonschema` has
> an API where it raises an exception if `validate()` doesn't succeed (None
> if things are happy). I don't love that API, so want to wrap it.
>

> def not_valid(instance, schema):
> try:
> return validate(instance, schema)
> except ValidationError as err:
> return str(err)
>
> I really wanted that to be one line rather than a helper function, and it
> really feels like it should be possible... and yet.
>

Incidentally, this is also helped greatly by the Walrus operator.  With the
support function I write:

if msg := not_valid(instance, schema):
print("The problem is:", msg)   # The exception contains detailed
diagnosis
else:
do_json_stuff(instance)

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/37XWNBFW5WQQI32TONYES6GZTNYU5W3I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Inline Try-Except Clause

[David Mertz]

On Fri, Aug 7, 2020 at 4:58 AM Brendan Barnwell 
wrote:

> It seems that the rationale that was used in the PEP was fairly
> narrowly focused on the comparison with things like dict.get() and the
> idea of EAFP.  A somewhat broader justification might be something along
> these lines:
>

For an example. Anyone is free to use, but I'm not claiming it's
necessarily the best.  This is from... well, probably not yesterday like I
said in other comment, but a couple days ago. The module `jsonschema` has
an API where it raises an exception if `validate()` doesn't succeed (None
if things are happy). I don't love that API, so want to wrap it.

def not_valid(instance, schema):
try:
return validate(instance, schema)
except ValidationError as err:
return str(err)

I really wanted that to be one line rather than a helper function, and it
really feels like it should be possible... and yet.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F3TSOX3MOQ7JSJA5CH6PQ25464RCRXWZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Inline Try-Except Clause

[David Mertz]

I think getting Guido on board would be a huge step.  Python has added
quite a bit of new syntax since 2014, and Guido himself is currently
advocating another new big change (pattern matching).  His opinion may have
shifted.

FWIW, I'm +1 on the concept. I've wanted it quite often, as recently as
yesterday.  I don't really love the syntax that was settled on.  Some sort
of keyword based version to resemble the ternary expression feels much more
natural to me.  It really *is* a kind of ternary, after all.  Exactly which
words in exactly which order... well, I could bring in yet more paint
samples for the bikeshed, but the concept is good.

Yours, David...

On Fri, Aug 7, 2020 at 1:55 AM Chris Angelico  wrote:

> On Fri, Aug 7, 2020 at 11:01 AM Jonathan Grant
>  wrote:
> >
> > How can we start to revive this PEP? And I completely agree, making the
> syntax `... except ... with ...` is much better than `eor`.
> >
>
> Have a read of the PEP's rejection notice at the top. To revive the
> PEP, the objections to it need to be solved.
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/5HNSLOTK45U2NQOD26NK3CCG5IL3VGGN/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NNNPDB4CQ3Z67E55PS6QFROXSQEYCWJP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Decorators for class non function properties

[David Mertz]

On Thu, Aug 6, 2020 at 2:52 AM Greg Ewing 
wrote:

> On 6/08/20 6:42 am, David Mertz wrote:
> >>>> @unit("meter") a = 3  # a = unit("meter")("a", 3)
> >>>> @unit("foot") b = 4   # b = unit("foot")("b", 4)
>
> This still doesn't explain why the decorator syntax would be
> significantly better than just calling the function directly.
>
> meters = unit("meter")
> feet = unit("foot")
>
> a = meters(3)
> b = feet(4)
>

The only difference is that in the usual existing style, 'a' doesn't know
that it's called "a".  You and Steven have both, basically, said "Why would
you possibly care about that?"  And honestly, I don't actually disagree.  I
was just trying to make a *plausible* case for wanting it to try to
extrapolate from the suggestion.

I think in the fairly rare case the (original) name matters, attaching it
manually is every bit as good.

N_a = mol(6.02214076e23)
N_a.name = "Avogadro"

I think in the oddball corner cases where we might care about the "binding
name" inside the object itself, __set_name__() gets us enough.  Yes, it's
not for globals or locals.  But putting that funny extra stuff inside an
instance seems like a low burden.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R4NLH4PHMINQWHQT42SS6F3VUXB6OT4P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Decorators for class non function properties

[David Mertz]

On Wed, Aug 5, 2020 at 2:25 PM Jonathan Fine  wrote:

> Real world examples where it would be useful are generally worth much more
> than invented examples.
>

I agree that @const is not really a useful "value decorator." I was just
picking up the example that occurred up-thread.

Here's something I think could be more useful (again, I'm +0 at best
myself).

>>> @unit("meter") a = 3  # a = unit("meter")("a", 3)
>>> @unit("foot") b = 4   # b = unit("foot")("b", 4)
>>> a, a.name, a.unit
(3, "a", "meter")

Implementation left to reader, but basically it's exactly Ricky's
__new__(), just wrapped in a class factory function to parameterize the
unit.



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JV7SYYTRYMODU4KKKFLXEI473SHK6RMA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Decorators for class non function properties

[David Mertz]

On Wed, Aug 5, 2020 at 1:27 PM Ricky Teachey  wrote:

> On Wed, Aug 5, 2020 at 11:41 AM Marco Sulla 
> wrote:
>
>> On Wed, 5 Aug 2020 at 15:53, Ricky Teachey  wrote:
>> from mypython import *
>> @const a = 5
>>
>
>  I'm probably dim but I have no idea what that is supposed to mean or do.
> Is this just calling const(a=5)...? what is the point of that?
>

I'm not advocating it, and I'm not the one that came up with it. But my
impression is that it is intended to mean:

a = const('a', 5)

This doesn't seem completely pointless:

>>> class const():
... def __init__(self, name, val):
... self.name = name
... self.val = val
... def about(self):
... print(self.name, '=', self.val)
...
>>> a = const('a', 5)
>>> a.val
5
>>> a.about()
a = 5

There might be a way to subclass, e.g. int, so that you don't need to use
`a.val` to get the value.  It wasn't obvious to me how to do it in pure
Python with 3 minutes thought.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X3ECJ4ZURNWANTMDFAXNL7PLTTXIKYOZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Introduce a boundary parameter for typing.get_type_hints when used with a class object

[David Mertz]

This definitely feels to me like though if an oddball case that "write your
own function" seems like the best solution. I accept the OP needs it, but I
have trouble imagining that many others would.

On Tue, Aug 4, 2020, 7:42 AM Dominik Vilsmeier 
wrote:

> In one of my projects I'm reusing class-level type annotations to identify
> relevant attributes for serialization, e.g. similar to the following:
>
> attrs = {name: getattr(obj, name) for name in
> get_type_hints(type(obj))}
>
> This is convenient because it merges the type annotations from the various
> stages in the class hierarchy, e.g.
>
> class Base:
> a: int
> class Derived(Base):
> b: str
>
> results in `attrs == dict(a=..., b=...)`.
>
> However it becomes inconvenient if external base classes are involved that
> define their own, unrelated type annotations, e.g.
>
> class External:  # from some other distribution
> unrelated: float
> class Base(External):
> a: int
>
> It would be helpful if `get_type_hints` had a `boundary` parameter that,
> when used with a class object, determines the upper boundary for the MRO.
> So it could be used in the following way:
>
> get_type_hints(type(derived_obj), boundary=Base)
>
> to exclude any type annotations further up the class hierarchy (including
> the ones from `Base`).
>
> Regarding the implementation this would effectively skip over base classes
> in the reverse MRO until it reaches the `boundary`.
>
> What do you think?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/T6K4DWENPM7LYXSDVYQYDVFEVBMA5K3L/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OBZ5UEYTPEJG65MYFYMSBUTI46N7HSW7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[David Mertz]

On Thu, Jul 30, 2020 at 11:55 AM Christopher Barker 
wrote:

>  > smaller_dict = dict(islice(large_dict.items(), 0, 255))
> well, beauty is in the eye of the beholder, of course. But really, you
> think it's pretty to import itertools, then make a function call, for
> what's very much a slicing operation?
>

In your post that introduced that,that is *exactly* what I thought of when
you first described the issue, before I read down to your solution.  It
took a fraction of a second to think of for me.  It *is* slightly verbose.
What do you think about this in current Python:

>>> large = {i:i**2 for i in range(1000)}
>>> import sys
>>> from itertools import islice
>>> class IterSlicer:
... def __getitem__(self, what):
... it, sl = what
... return islice(it, sl.start or 0, sl.stop or sys.maxsize,
sl.step or 1)
...
>>> IS = IterSlicer()
>>> dict(IS[large.items(), 3:10:2])
{3: 9, 5: 25, 7: 49, 9: 81}
>>> from itertools import count
>>> set(IS[count(), 10:100:9])
{64, 37, 73, 10, 46, 82, 19, 55, 91, 28}

Potentially IterSlicer, or IS, could live in itertools (or more likely
more-itertools) just to provide a short way to use slice notation with an
arbitrary iterable... not limited to dict.items().


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OEO7XAM4FJM2742M4AWJVEEHBGIKZPJV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

[David Mertz]

On Sat, Jul 25, 2020, 3:12 PM Christopher Barker

> I'm not advocating for any change (Obviously :-) ) -- but if there were to
> be one, I"d strongly vote for some version of else not break or else if
> not break because it introduces no keywords, and it's a lot more
> compatible documentation-wise than a different word
>

As bikesheds go, I like a "soft keyword" which PEG apparently makes easier.
Maybe 'nobreak' or 'notbreak' or 'finished'.

But as Christopher notes, the spelling 'else: # not break' is fine for
readability, even if not for remembering it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/A7SQWPGGJDDLYVHKZUA6A4TFEZRJAVNY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

[David Mertz]

On Thu, Jul 23, 2020, 9:19 PM Rob Cliffe via Python-ideas

> The upholders of the status quo regularly provide gallant explanations of
> why "else" is perfectly natural, even intuitive.
> The fact is, it isn't.  If it were, it wouldn't **need** to be repeatedly
> explained by gurus to lesser mortals.
>

I have used Python for 22 years, and I still cannot remember what else does
in a loop for more than a few minutes at a time. It's easy to remember "it
has something to do with whether there was a break." But neither direction
in that choice yet feels obvious to me.

I've read all the posts in this thread. I watched Raymond's explanation
from a talk a few years ago. I've read Guido's explanation. I've read
Knuth. I have a graduate degree in humanities and am a professional writer.
I'm a native English speaker.

It's still not intuitive to me.

In my own code I just avoid the construct. I reckon I'm +1 on "some better
spelling" ... although I manage fine without using it. I rarely see it in
the wild, probably because it is confusing.

I can't think of any other area of Python that needs to be defended so
> regularly and so vociferously, nor that very experienced Python programmers
> confess they find confusing.  Swelp me, someone in this very thread (Barry)
> misunderstood it[1].  And suggesting that those of us who don't find it
> clear lack skill in English is just plain insulting.
>
> YMMV, but the only way I can grok it when I see it is to mentally
> translate "else" to "if no break".  This is mental effort that would be
> spared or at least diminished if it were spelt in a more obvious way in the
> first place.
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VBYFNP2ZZNF733AAP3GL3THVOC2WQSB7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Tue, Jul 21, 2020, 12:14 PM Sebastian Berg

> First, using it for named dimensions, means you don't actually need to
> mix it with normal tuple indexing, mixing both seems rather confusing?
>
> temperature.loc(method="nearest")[longitude=longs, latitude=lats]
>

I probably don't disagree on the API for xarray or similar library. But
that choice would be up to the library, not Python developers. Whatever the
way keyword argument are passed to .__getitem__()[*], they are available
for the class to do whatever it likes with them.

[*] In my opinion, passing the keyword argument using anything other than
the standard **kws style sounds crazy. We have perfectly good ways to do
that for every other function or method. Don't invent something new and
different that only works with .__getitem__().
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SA5FXO3R5LC52EKKABFH7PBXZEKP53CD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Simplifying functions syntax

[David Mertz]

There is absolutely no possibility that this will go anywhere. It breaks
every single Python program written since 1989, with absolutely ZERO new
functionally. Changing the spelling of a few of the most come keywords is a
non-starter.

If you can time travel to 1987 and propose differently spelled keywords,
yours are no worse... But also no better.

On Mon, Jul 20, 2020, 6:06 PM Thiago Carvalho D' Ávila <
thiagocav...@gmail.com> wrote:

> Python is a simple language to understand, but to me functions are a
> special case. And according to the Zen of Python (PEP20
> ): `Special cases aren't
> special enough to break the rules.`.
>
> For beginners, Python functions can cause some pain that boils down to
> three points that shows some lack of standardization.
>
> 1. Function declaration needs keyword `def`:
>
> If you are a long time Python (1991) developer, you surely got used to the
> keyword `def` to declare a function, same goes if you are a Ruby (1995)
> programmer. In Fortran (1957) you start them with the word `function` and
> in Perl (1987) you use the word `sub`.
>
> In C-family languages C (1972), C++ (1985), C# (2000), also in Java (1995)
> you don't need any word to declare it:
>
> return_type function_name(args)
>
> In languages from the last decade you have Rust (2010) with the keyword
> `fn`, Kotlin with `fun`. Dart (2011) follows C-family style and Julia
> (2012) offers two ways of defining a named function: using the word
> `function` or the compact version with no keyword `f(x,y) = x + y`.
>
> Following PEP20  aphorism of
> `Simple is better than complex.` I would say no word like C-family is
> simpler. But there is also `Explicit is better than implicit.`. I would
> choose `fun` or even `fn` since they at least resemble the word function.
> Eg:
>
> >>> def x(args):
>
> ... pass
>
> would turn into:
>
> >>> fun x(args):
>
> ... pass
>
> Resemblance of function leads me to my next point.
>
> 2. Class name
>
> Again, functions are an exception.
>
> >>> x = 1
>
> >>> type(x)
>
>  *# not integer*
>
> >>> x = {"a": 1}
>
> >>> type(x)
>
>  *# not dictionary*
>
> >>> def x(args):
>
> ... pass
>
> ...
>
> >>> type(x)
>
>  *# not fun =(*
>
> 3. Type hinting
>
> This is when the syntax gets trickier. With PEP 585
>  you don't have to do:
>
> >>> from typing import Dict, List, Set
>
> to have:
>
> >>> def x(foo: dict[string, int], bar: list[float]) -> None:
>
> ... pass
>
> But if you have a function as a parameter, your only option is to use
> Callable. It breaks the rule again having a totally different syntax:
>
> >>> from typing import Callable
>
> >>> def x(method: Callable[[int, dict], None]) -> None:
>
> ... pass
>
> So you have `def`, `function`, `Callable` and also this [[args], return]
> syntax, also different from the rest of type hinting syntax. I would add a
> `fun` type without the need of an import. Also Callable syntax could also
> be simplified with the -> operator into something like:
>
> >>> def x(method: fun[int, dict] -> None) -> None:
>
> ... pass
>
> === Preliminary Proposal ===
>
> With the same goal of removing the pain and making life of Python teachers
> easier (PEP 585 ), I would
> change this:
>
> >>> from typing import Callable
>
> >>> def x(method: Callable[[int, dict], None]) -> None:
>
> ... pass
>
> ...
>
> >>> type(x)
>
> 
>
> Into something like this:
>
> >>> fun x(method: fun[int, dict] -> None) -> None:
>
> ... pass
>
> ...
>
> >>> type(x)
>
> 
>
> In order to make this possible the following changes would be needed:
>
> 1. adding `fun` keyword to method definition (maintaining `def` for
> backwards compatibility);
>
> 2. adding `fun` built-in type in addition to Callable;
>
> 3. supporting `->` as return syntax for type hinting.
>
> Isn't it much more `fun` to learn the language this way?
>
> Kind regards,
>
> Thiago Carvalho D'Ávila
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/D5ZF3QBOP35XYI5NSOSHYIU44MV3YFBY/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SJ2GFPSNVPSPUEHW6RJEJ2MRQ63R54H2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Sun, Jul 19, 2020 at 10:27 PM Jonathan Goble  wrote:

> One use case that comes up in xarray and pandas is support for indicating
>> indexing "modes". For example, when indexing with floating point numbers
>> it's convenient to be able to opt-in to approximate indexing, e.g.,
>> something like:
>> array.loc[longitude, latitude, method='nearest', tolerance=0.001]
>>
>
> I had to stare at this for a good 30 seconds before I realized that this
> wasn't a function/method call. Except for the square brackets instead of
> parentheses, it would be.
> Honestly, this whole idea smells to me like just wanting another type of
> function call with different semantics.
> IMHO the above example would be better spelled as:
> array.loc.get(longitude, latitude, method='nearest', tolerance=0.001)
>

The problem is that with Pandas, Xarray, and other data frame/data array
libraries, using slices is typical.  Continuing with the example:

arr.loc[45:46, 69:70, altitude=300:400, tolerance=0.0001, projection="
WGS1984"]

There's no reason to tack a function onto the .loc accessor, it could just
be a method on the array/data frame itself.  So there are no extra
characters.  But doing this with parentheses would need a different new
feature of allowing slices directly as arguments.

That said, pandas.IndexSlice and numpy.s_ both provide accessors to allow
passing slices more easily.  So this is possible now:

arr.get(I[45:46], I[69:70], altitude=I[300:400], tolerance=0.0001,
projection="WGS1984")

... I mean, assuming someone writes an appropriate .get() method for their
favorite data array/frame library.  But Python itself has everything needed.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W6UH7CWZK5BELTR6JB2B5XHZLAPS4N74/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Sun, Jul 19, 2020 at 6:35 PM Dominik Vilsmeier 
wrote:

> But this looks unnecessarily complicated. Why can't xarray allow the
> following:
>
> ds["empty"]["lon", 1:5, "lat", 3:] = 10
>
> which looks very close to the proposed syntax below. Not that I'm against
> the proposal but I think that any use case involving *only* keyword
> arguments isn't a very strong one, because it can easily be solved that way
> without a change to existing syntax.
>
Xarray already allows positional slices in multiple dimensions.  The
existing dict weirdness is to have a way to introduce the named
dimensions.  But suppose that the array in question had a first listed
'altitude'.  I think nowadays, we can write:

arr.loc[50:60, 1:5, 3:]

If we only reference dimensions by number not by name.  Under the "commas
separate keys from values" this would be difficult:

arr.loc[50:60, "lon", 1:5, "lat", 3:]

Yes, I can imagine a rule like "If it is a slice that wasn't preceded by a
string, treat it as positional, otherwise if it is a string treat it as a
key, but treat the next thing after a string as the slice value
corresponding to that key."

That seems more error prone and harder to grok than the potential:

arr.loc[50:60, lon=1:5, lat=3:]

Where you'd still just have to know that "axis 0 is altitude" of your
particular array.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XNJL45PE6DFYTMB6SUYESPG5HNY62A4I/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Use lambda in str.replace/re.sub

[David Mertz]

We have this in re.sub().  Not sure we need a second way to do it.

On Sun, Jul 19, 2020 at 12:57 PM artem6191  wrote:

> Example: '1 1 1 1'.replace('1', lambda char, index: str(index)+char) # '01
> 11 21 31'
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/7WBQW3YWNO5MODRQJJXAC2AW3QFGGCXB/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OUUIT4GRWQ7TWC6L7HGY2BU7IURIOUV7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz via Python-ideas]

On Fri, Jul 17, 2020 at 4:12 PM Todd  wrote:

> ds["empty"][lon=1:5, lat=6:] = 10
>

I agree that looks really nice. I think this is the first suggestion to
allow slices on the RHS of keyword indexing. That's the part that was
missing in me understanding how this would help xarray greatly. It's also
somewhat of an additional novelty added to the proposal that is somewhat
independent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E4RTBDZL6CHMZAMCNWLMJ3RHZILDM5CT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

Fwiw, I'm probably -0 on the feature itself. Someone suggested it could be
useful for xarray, but I'm not sure now what that would look like. If
someone had an example, I could easily be moved.

I'm not against the original suggested use with type annotations, but I
also don't really care about it. I don't think 'd[K(1, 2, 3, a=4, b=5)]' is
bad as an existing spelling.

On Fri, Jul 17, 2020, 12:08 PM David Mertz  wrote:

> On Fri, Jul 17, 2020, 8:16 AM Jonathan Fine  wrote:
>
>> Steve and I have different opinions, as to what the new behaviour of:
>> >>> d = dict()
>> >>> d[x=1, y=2] = 3
>> should be.
>>
>> He prefers that the assignment fail with
>> TypeError: dict subscripting takes no keyword arguments
>>
>> I prefer that the assignment succeed (and hence a new key-value pair is
>> added to 'd').
>>
>
> I definitely agree with Steven. It is obviously *possible* to create some
> brand new type of object that is "multi-assignment fragment." But why?!
>
> No clearly useful semantics comes to mind for this new object. Well, it
> would need to be hashable. Lots of things are though, so it's not like we
> have nothing to use as dict keys now.
>
> We don't lose anything if we add the feature but intake don't support it
> for dictionaries. If someone comes up with a really useful reason to have
> that MultiAssignmentType, is not usually considered a breaking change to go
> from "this raises am exception" to "this does something worthwhile" (but
> obviously, in all such cases, you can artificially construct code that will
> break without a certain exception).
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/62Y7TQXAMH2HCJPRCZOBHVHQJ76IBG4O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 -- Support for indexing with keyword arguments

[David Mertz]

On Fri, Jul 17, 2020, 8:16 AM Jonathan Fine  wrote:

> Steve and I have different opinions, as to what the new behaviour of:
> >>> d = dict()
> >>> d[x=1, y=2] = 3
> should be.
>
> He prefers that the assignment fail with
> TypeError: dict subscripting takes no keyword arguments
>
> I prefer that the assignment succeed (and hence a new key-value pair is
> added to 'd').
>

I definitely agree with Steven. It is obviously *possible* to create some
brand new type of object that is "multi-assignment fragment." But why?!

No clearly useful semantics comes to mind for this new object. Well, it
would need to be hashable. Lots of things are though, so it's not like we
have nothing to use as dict keys now.

We don't lose anything if we add the feature but intake don't support it
for dictionaries. If someone comes up with a really useful reason to have
that MultiAssignmentType, is not usually considered a breaking change to go
from "this raises am exception" to "this does something worthwhile" (but
obviously, in all such cases, you can artificially construct code that will
break without a certain exception).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MUR2S5NGWCNT52Q2HXLEKSVXCVA3RCE6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about Fast Python Engine and Python EveryWhere

[David Mertz]

On Thu, Jul 16, 2020, 2:57 PM  wrote:

> I know, I know that PyPy is fast as V8 but PyPy implement the whole
> library inside and it is not easy to embed it somewhere
>

I have no idea what you mean by that. PyPy doesn't need its standard
library to run. Just like in CPython, those are just a bunch of external
files that you can import or not, as desired.

>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BW55H6V4JE2A3XY5XRYJJNXKRIC52WEG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts about Fast Python Engine and Python EveryWhere

[David Mertz]

On Thu, Jul 16, 2020 at 12:59 PM  wrote:

> Recently I have been thinking about why `JavaScript` with it's horrible
> type system and lots of edge cases has supported so many platform and is
> very fast ...
> First answer is simple, because big companies such as Google, Facebook and
> so on evolve this language and run-time for it ...
>

Check out PyPy.  It is every bit as fast as the fastest Javascript engines
like V8.

So is Numba, for that matter, but it's not language wide, so it's a
slightly different beast.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6D6WUAUHDO6WWCYWBXDOZQ2PBVSHXBOG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add random.choice_iter for randomly selecting from an arbitrary iterable

[David Mertz]

If we get this function (which I would like), the version with k items
(default 1) is much better. Some iterators cannot be repeated at all, so
not only is it slower to call multiple times if you need k>1, it's
impossible.

On Mon, Jul 13, 2020, 8:37 AM David Mertz  wrote:

> This is an inefficient reservoir sampling. The optimized version does not
> need to call a random inclusion switch on every element, but can skip a
> geometrically ordered collection of (random) skip lengths to avoid most
> random inclusion decisions.
>
> Obviously, all items must be iterated over no matter what, but if
> randrange() is significant compared to the cost of next(), the
> skip-then-decide version is much preferred, especially as size grows.
>
> On Mon, Jul 13, 2020, 7:53 AM Oscar Benjamin 
> wrote:
>
>> I posted this in the thread about indexing dict items but it seems to
>> have been buried there so I'm starting a new thread.
>>
>> Maybe there could be a function in the random module for making a
>> random choice from an arbitrary (finite) iterable. This implementation
>> can choose a random element from an iterable by fully iterating over
>> it so is O(N) in terms of CPU operations but O(1) for memory usage:
>>
>> # rnd.py
>> from random import randrange
>>
>> def choice_iter(iterable):
>> iterator = iter(iterable)
>> try:
>> value = next(iterator)
>> except StopIteration:
>> raise ValueError("Empty iterable")
>> for n, candidate in enumerate(iterator, 2):
>> if not randrange(n):
>> value = candidate
>> return value
>>
>> You could use that to get a choice from a dict, set etc:
>>
>> >>> choice_iter({'q', 'w', 'e'})
>> 'e'
>> >>> choice_iter({'q', 'w', 'e'})
>> 'q'
>> >>> choice_iter({'q', 'w', 'e'})
>> 'e'
>> >>> choice_iter({'q', 'w', 'e'})
>> 'w'
>>
>> You can make a random choice from the keys, values or items of a dict.
>> For a large dict/set it will be slow compared to converting to a list
>> because it's calling randrange once per item. It can probably be made
>> a lot faster by doing something like calling randbits and extracting
>> the bits for multiple iterations of the loop.
>>
>> Although choice_iter is slower than choice it works for arbitrary
>> iterables and has constant memory requirements when used with a lazy
>> iterator. There are situations in which you would not want to build a
>> list and use random.choice and where the computational overhead is
>> insignificant. For example choice_iter can be used to randomly choose
>> a line from an arbitrarily large text file without loading it entirely
>> into memory:
>>
>> >>> from rnd import choice_iter
>> >>> with open('rnd.py') as fin:
>> ... line = choice_iter(fin)
>> ...
>> >>> line
>> 'except StopIteration:\n'
>>
>> When reading from a large text file this is roughly optimal in
>> performance terms because it does precisely the minimum amount of IO
>> (one pass) while having constant memory overhead. Of course if you
>> wanted to select multiple lines from the file then calling choice_iter
>> repeatedly is immediately suboptimal because it would read the file
>> more than once. It's not hard though to generalise the function above
>> to something that e.g. makes a selection of k values from an iterable
>> in a single pass. Here is a version that works without replacement:
>>
>> def sample_iter(population, k):
>> iterator = iter(population)
>> values = []
>> for _ in range(k):
>> try:
>> value = next(iterator)
>> except StopIteration:
>> raise ValueError("Too few items")
>> values.append(value)
>> for n, candidate in enumerate(iterator, k+1):
>> random_index = randrange(n)
>> if random_index < k:
>> values[random_index] = candidate
>> return values  # maybe shuffle first?
>>
>> The memory cost of this is O(k) regardless of the size of the input
>> iterable.
>>
>> --
>> Oscar
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/4OZTRD7FLXXZ6R6RU4BME6DYR3AXHOBD/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FLXVRXRPZCIAFYD4UIXV4Y3LSFBZZKTQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add random.choice_iter for randomly selecting from an arbitrary iterable

[David Mertz]

This is an inefficient reservoir sampling. The optimized version does not
need to call a random inclusion switch on every element, but can skip a
geometrically ordered collection of (random) skip lengths to avoid most
random inclusion decisions.

Obviously, all items must be iterated over no matter what, but if
randrange() is significant compared to the cost of next(), the
skip-then-decide version is much preferred, especially as size grows.

On Mon, Jul 13, 2020, 7:53 AM Oscar Benjamin 
wrote:

> I posted this in the thread about indexing dict items but it seems to
> have been buried there so I'm starting a new thread.
>
> Maybe there could be a function in the random module for making a
> random choice from an arbitrary (finite) iterable. This implementation
> can choose a random element from an iterable by fully iterating over
> it so is O(N) in terms of CPU operations but O(1) for memory usage:
>
> # rnd.py
> from random import randrange
>
> def choice_iter(iterable):
> iterator = iter(iterable)
> try:
> value = next(iterator)
> except StopIteration:
> raise ValueError("Empty iterable")
> for n, candidate in enumerate(iterator, 2):
> if not randrange(n):
> value = candidate
> return value
>
> You could use that to get a choice from a dict, set etc:
>
> >>> choice_iter({'q', 'w', 'e'})
> 'e'
> >>> choice_iter({'q', 'w', 'e'})
> 'q'
> >>> choice_iter({'q', 'w', 'e'})
> 'e'
> >>> choice_iter({'q', 'w', 'e'})
> 'w'
>
> You can make a random choice from the keys, values or items of a dict.
> For a large dict/set it will be slow compared to converting to a list
> because it's calling randrange once per item. It can probably be made
> a lot faster by doing something like calling randbits and extracting
> the bits for multiple iterations of the loop.
>
> Although choice_iter is slower than choice it works for arbitrary
> iterables and has constant memory requirements when used with a lazy
> iterator. There are situations in which you would not want to build a
> list and use random.choice and where the computational overhead is
> insignificant. For example choice_iter can be used to randomly choose
> a line from an arbitrarily large text file without loading it entirely
> into memory:
>
> >>> from rnd import choice_iter
> >>> with open('rnd.py') as fin:
> ... line = choice_iter(fin)
> ...
> >>> line
> 'except StopIteration:\n'
>
> When reading from a large text file this is roughly optimal in
> performance terms because it does precisely the minimum amount of IO
> (one pass) while having constant memory overhead. Of course if you
> wanted to select multiple lines from the file then calling choice_iter
> repeatedly is immediately suboptimal because it would read the file
> more than once. It's not hard though to generalise the function above
> to something that e.g. makes a selection of k values from an iterable
> in a single pass. Here is a version that works without replacement:
>
> def sample_iter(population, k):
> iterator = iter(population)
> values = []
> for _ in range(k):
> try:
> value = next(iterator)
> except StopIteration:
> raise ValueError("Too few items")
> values.append(value)
> for n, candidate in enumerate(iterator, k+1):
> random_index = randrange(n)
> if random_index < k:
> values[random_index] = candidate
> return values  # maybe shuffle first?
>
> The memory cost of this is O(k) regardless of the size of the input
> iterable.
>
> --
> Oscar
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/4OZTRD7FLXXZ6R6RU4BME6DYR3AXHOBD/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZJNJX6GUBABH3ETHES43X3PWXZIOZGJX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[David Mertz]

On Sat, Jul 11, 2020 at 4:24 PM Christopher Barker 
wrote:

> NOTE: I've wanted for ages to make a "PYSON" format / module for when JSON
> is not quite enough. e.g. distinction between lists and tuples, dict keys
> that aren't strings 
>

https://github.com/jsonpickle/jsonpickle

You're not the first one.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UKVDHKYFH2ZPX22XYAOMVCTOEYZFRIDX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[David Mertz]

On Sat, Jul 11, 2020 at 4:33 PM David Mertz  wrote:

> In any case, if "reservoir sampling" is the goal here, we should just add
> a function `random.reservoir_sample()` to accommodate using iterators
> rather than sequences (https://en.wikipedia.org/wiki/Reservoir_sampling)
>

Doh! I mean *iterables* not *iterators* of course.

This would actually be a reasonable and useful function to have, and
sampling from dict.items() would fall out transparently from that.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZNSL5SMUVRPNROFLWVLV56HCULP7CBGT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[David Mertz]

On Sat, Jul 11, 2020 at 3:45 PM Christopher Barker 
wrote:

> random.choice(the_dict.keys())
>
> is a little easier than:
>
> random.choice(list(the_dict.keys())
>

Ummm... don't you mean:

random.choice(list(the_dict))

If it's keys you care about I've saved you one character over your proposed
style, while also reading better to me.  It's only for .items() where it
doesn't work.  And honestly, just looking up the value from the random key
is not hard.

In any case, if "reservoir sampling" is the goal here, we should just add a
function `random.reservoir_sample()` to accommodate using iterators rather
than sequences (https://en.wikipedia.org/wiki/Reservoir_sampling)


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GIIGKOEDIJ3NLMUYJQ3HXFK5ZMI5IQX6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[David Mertz]

On Sat, Jul 11, 2020 at 3:45 PM Christopher Barker 
wrote:

> On Fri, Jul 10, 2020 at 12:45 PM David Mertz  wrote:
> > The strongest argument I've seen is: `list(d.items())` adds six
> characters.
>
> 1) Matching our mental model / usability: if I want the nth item (or a
> random item) from a dict, I want to ask for that -- I don't want to make a
> list, just to index it and throw it away. the list(d.items()) idiom is the
> right one if I actually need a list -- it's a bit awkward to have to make a
> list, just to throw it away. 2) Performance: making an entire list just to
> get one item out is a potentially expensive operation. Again, for the
> limited use cases, probably not a big deal, I'm having a really hard time
> imagining a application where that would be a bottleneck, but it is *a*
> reason, if not a compelling one.
>

For the single case of wanting JUST ONE random key/val pair from a
dictionary EVER, I agree random.choice(d.items()) looks a little better.  I
even agree that it's something that probably seems obvious to try the first
time.

However, once you want to get multiple items, the amortized costs starts to
matter.  I think that latter situation is likely to be VASTLY more common
in real code... but I'm just saying that without evidence.  Still, this is
such a narrow situation, I just don't see it as justifying a language
change (the "get an item, only once" case).

Code that I can easily imagine writing (although I'm not sure I actually
have):

items = dict.items()
while True:
k, v = random.choice(items)
if satisfies_need(k, v):
do_stuff(k, v)
break

The hypothetical view-sequence would be an attractive nuisance that would
hurt performance in all such similar code.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/57GW6PGVJ3WAF6A4NKMERIOHI2SVMSGI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[David Mertz]

Definitely -1 after reading all the discussion.

The strongest argument I've seen is: `list(d.items())` adds six characters.

So far, there's absolutely nothing described that cannot easily be
accomplished with list(). Moreover, even apart from the work of maintaining
the feature itself, the attractive nuisance of getting O(N) behavior rather
than O(1) seems like a strong anti-feature.

On Fri, Jul 10, 2020, 1:07 PM Stestagg  wrote:

>
> I too have sometimes proposed what I think of as "minor quality-of-life"
>> enhancements, and had them shot down. It stings a bit, and can be
>> frustrating, but remember it's not personal.
>>
>
> I don't mind the shooting down, as long as the arguments make sense :D.
>
> It seems like we're both in agreement that the cost of implementing &
> maintaining the change is non-zero.
> I'm asserting that the end benefit of this change is also non-zero, and in
> my opinion higher than the cost. But I also acknowledge that the benefit
> may not be enough to overcome the inertia behind getting a change made.
>
> The reason I'm persevering is to try and weed out the immaterial or
> incorrect reasons for not making this change, so hopefully we're left with
> a good understanding of the pros-cons.
>
>
>>
>> The difficulty is that our QOL enhancement is someone else's bloat.
>> Every new feature is something that has to be not just written once, but
>> maintained, documented, tested and learned. Every new feature steepens
>> the learning curve for the language; every new feature increases the
>> size of the language, increases the time it takes to build, increases
>> the time it takes for the tests to run.
>>
>
> Yeah, I can see that more code = more code overhead, and that's got to be
> justified.
> I don't believe that this feature would steepen the language learning
> curve however, but actually help to shallow it slightly (Explained more
> below)
>
>
>>
>> This one might only be one new method on three classes, but it all adds
>> up, and we can't add *everything*.
>
>
>> (I recently started writing what was intended to be a fairly small
>> class, and before I knew it I was up to six helper classes, nearly 200
>> methods, and approaching 1500 LOC, for what was conceptually intended to
>> be a *lightweight* object. I've put this aside to think about it for a
>> while, to decide whether to start again from scratch with a smaller API,
>> or just remove the word "lightweight" from the description :-)
>>
>
> Absolute method count is seldom a standalone indicator of a dead end.
> Often classes with many methods, (especially if they're accompanied by
> lots of helpers)
> are a side-effect of some abstraction failure.  Usually when I'm
> consulting on fixing projects
> with these characteristics, it's a case of the developers not correctly
> choosing their abstractions, or letting them leak.
> It sounds like you let that one get away from you, chalk it up to a
> learning experience.
>
> The "It walks like a zoo, squaks/lows/grunts/chitters like a zoo" problem
> is very real.
> This is more of a "It used to be a duck. Now it walks like a duck, but
> doesn't sound like a duck because it's a coot" problem
>
>>
>> So each new feature has to carry its own weight. Even if the weight in
>> effort to write, effort to learn, code, tests and documentation is
>> small, the benefit gained must be greater or it will likely be rejected.
>>
>> "Nice to have" is unlikely to be enough, unless you happen to be one of
>> the most senior core devs scratching your own itch, and sometimes not
>> even then.
>>
>>
>> > >>> import numpy as np
>> > >>> mapping_table = np.array(BIG_LOOKUP_DICT.items())
>> > [[1, 99],
>> >  [2, 23],
>> >  ...
>> > ]
>>
>> That worked in Python 2 by making a copy of the dict items into a list.
>> It will equally work in Python 3 by making a copy of the items into a
>> list.
>>
>> And I expect that even if dict.items() was indexable, numpy would
>> still have to copy the items. I don't know how numpy works in detail,
>> but I doubt that it will be able to use a view of a hash table internals
>> as a fast array without copying.
>>
>> Bottom line here is that adding indexing to dict views won't save you
>> either time or memory or avoid making a copy in this example. All it
>> will save you is writing an explicit call to `list`. And we know what
>> the Zen says about being explicit.
>>
>
> What making dict_* types a Sequence will do is make this code (as written)
> behave:
> 1. like it used to do
> 2. like most people seem to expect it to.
>
> Currently numpy does something that I consider unexpected (I'm sure, given
> your previous responses, you'd disagree with this, but from canvassing
> Python devs, I feel like many people share my opinions here)
> with that code.
>
>
>>
>>
>> > >>> import sqlite3
>> > >>> conn = sqlite3.connect(":memory:")
>> > >>> params = {'a': 1, 'b': 2}
>> > >>> placeholders = ', '.join(f':{p}' for p in params)
>> > >>> statement = f"select {placeholders}"
>> > 

[Python-ideas] Re: An alternative to using a clamp / clip / trim function

[David Mertz]

This class or closure to clamp at specific bounds is nice. But I want to
clamp based on runtime values for upper/lower fairly often. A function is
much better for that use.

On Wed, Jul 8, 2020, 11:22 AM Jonathan Fine  wrote:

> Hi All
>
> This is related to discussion
> https://mail.python.org/archives/list/python-ideas@python.org/thread/KWAOQFSV3YJYQV2Y5JXGXFCXHJ3WFLRS/#ZT3OBOPNIMXQ2MU7N5RFBL5AJSYRZJ6Q
>
> In Python, lists don't have a join method. Instead, it's strings that have
> the join method. Hence we have:
> >>> ', '.join('abcde')
> 'a, b, c, d, e'
>
> The intermediate method we can save and store and use again.
> >>> joiner = ', '.join
> >>> joiner('fghijk')
> 'f, g, h, i, j, k'
>
> We can do something similar when clamping, clipping or trimming a value.
> For example, suppose we want limits on the room temperature, that the
> thermostat cannot override.
> >>> aircon_clipper = Clamper(50, 80)
>
> >>> thermostat_temp = 40
> >>> target_temp = aircon_temp_clipper(thermostat_temp)
> >>> target_temp
> 50
>
> What I like best about this is that the name of the function --
> aircon_temp_clipper -- gives the reason for the clipping (or clamping). And
> this name can be chosen to suit the domain. I also like that it wraps the
> high and low values (50 and 80) as data used by a method.
>
> Here's an implementation of Clamper.
>
> class Clamper:
> def __init__(self, lo, hi):
>
> if not lo <= hi:
> raise ValueError
> self._lo = lo
> self._hi = hi
>
> def __call__(self, num):
> lo, hi = self._lo, self._hi
>
> at_least_lo = (lo <= num)
> at_most_hi = (hi >= num)
>
> if at_least_lo and at_most_hi:
> return num
> elif at_least_lo:
> return hi
> elif at_most_hi:
> return lo
> else:
> raise ValueError
>
> If performance is important to you, you could instead use a closure to
> create a function. This would also allow you to provide a docstring.
>
> I hope this helps at least some of the people some of the time.
>
> --
> Jonathan
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SXECQDXUJ7J5J4YOIFYJL3VNS757PTNI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IWVMIURULLW5IOZGFHIDNEIYBN5IJIWM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

On Sun, Jul 5, 2020, 10:25 PM Steven D'Aprano  wrote:

> > The standard library *does* seem to have taken pains to avoid
> "finite nans."
>
> I don't know what you mean by "finite nans". By definition, any NAN is not
> considered finite.
>

The scare quotes because I know that's not a real thing. Maybe "genetically
finite" ... I.e. do some operations on regular finite values that wind up
with NaN. I know, I know... That's not really right either, since finite
values can overflow to infinities on their own.

Do you mean, the stdlib has taken pains to avoid returning NANs ex novo,
> i.e. from purely finite arguments? Then I agree.
>

Yes. That's it. :-)


> min() and max() are notable, and unfortunate, exceptions in that their
> treatment of NANs depends on the order of argument. I would call that a bug
> except that the statistics module (which I wrote) has the same flaw, and
> I've argued in the past that this is not a bug :-)
>

Personal growth is healthy!

That's not the trig functions' fault, it's the fact that we cannot
> exactly represent pi/2 exactly. I'm not sure what you mean by pruning
> them, it is pretty much standard that tan(pi/2) doesn't fail:
>

Of course. math.pi is a number that isn't really pi, as any finite
representation must be. tan() is doing the right thing. I just meant that
if math.cot() were added to the standard library, I could pass an exact
zero as argument. None of the trig or hyperbolic functions that are
undefined at ZERO are included.

But yes, math.log(0) is another one where a NaN is avoided in favor of an
exception. A few different log bases, but same story for all.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZVBGV7UDCFC4OEFE5UFV7ZNUFK223ZIC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

On Sun, Jul 5, 2020 at 10:04 PM Christopher Barker 
wrote:

> Since you brought that up -- I recall a lot of debate about whether NaN's
> should be considered missing values or "poisoning" in the statistics module
> -- there are arguments for both, and neither was clear or obvious. So using
> NaN to mean "not specified" in this context would not be obvious to
> everyone, and when we have the perfectly good None instead, why do it?
>

Well, yes... I wrote a lot of that debate :-)

I even sort of re-discovered quick select on my own... then eventually
figured out that a bunch of people had benchmarked a better implementation
to potentially use in statistics.median() a year before I tried.  Python
sorted() is really fast!  But it's still the WRONG way to do this, or at
least there should be a switch to allow nan-poisoning  and/or nan-stripping.

Btw, definitely +1 on math.clamp(value, *, lower=None, upper=None) .
>>
>
> what about:
>
> math.clamp(value, *, lower=-math.inf, upper=math.inf) .
>

Oh sure.  That's fine.  But the implementation would still need to check
for None and convert it to the infinities.  Ordinary users just simply ARE
going to try:

math.clamp(x, lower=None, upper=99)

And expect that to mean "I don't care about lower bound."

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JMNU4L42AAGKFGF6CSUW3CWY574QUFIM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

On Sun, Jul 5, 2020 at 9:38 PM Steven D'Aprano  wrote:

> > >I agree with you that `clamp(lower=x, value=NAN, upper= x)` should
> > >return x.
>
> Sorry Greg, on this point at least the IEEE-754 standard is firm: if a
> function will return the same result for every non-NAN argument, then it
> must return the same result for NAN arguments too.
>
> clamp(value, x, x)
>
> will always return x for every finite and infinite value, so it must
> return x for NANs too.
>

I strongly agree with Steven here.  Also about order-dependence in results
of min() and max() being disturbing and contrary to IEEE-754.

... so, umm... Steven... statistics.median()?!

Btw, definitely +1 on math.clamp(value, *, lower=None, upper=None) .

-1 on built-in.  -0 on any other function signature.  Actually, I'm fine
with math.clip() as well, but clamp seems more popular.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7X64MLI6G4Q5ZSIZNBRCU3HAYWWWYOI3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

On Sun, Jul 5, 2020 at 8:57 PM Steven D'Aprano  wrote:

> On Sun, Jul 05, 2020 at 12:15:27PM -0400, David Mertz wrote:
> > This is a digression, but does anyone have a nice example IN PYTHON of
> > arriving at a NaN without going through infinity. I think Julia is right
> > and Python is wrong about '0/0', but as things are, that's not an
> example.
>
> I wouldn't expect one in Python, I think there is an unofficial policy
> of ensuring that Python builtins and the math library will not return
> NANs unless passed a NAN, or at least an INF, rather they will raise.
>
> > > >>> 1e1000-1e1000
> > > nan
> is just a funny way of writing INF - INF :-)
>

The standard library *does* seem to have taken pains to avoid "finite
nans."  It kinda weakens your case about worrying about doing clamp() right
in the face of NaNs :-).

I recognize there are funny ways of writing infinity.  But since Python
really doesn't quite follow IEEE-754 on 0/0, or math.fmod(x, 0.), or a few
other places it might arise in "natural" operations (i.e. it's easy not to
notice that your 'y' has become zero.

It also looks like the trig functions are pruned to those that don't have
undefined values for numbers I can type in.  I can *type*
`math.tan(math.pi/2)`, of course.  But math.pi is a little bit smaller than
the actual pi, so I just get a big number for an answer.  But I cannot try
the hypothetical:

>>> math.cot(0)
nan

For what we actually have:

>>> math.tan(math.pi/2)

1.633123935319537e+16

One ULP more:

>> math.tan(np.nextafter(math.pi/2, np.inf))

-6218431163823738.0


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P742DYSQAMWKAAKNPASMDSEXH6AGEXKO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

On Sun, Jul 5, 2020, 3:51 PM Chris Angelico  wrote:

> On Mon, Jul 6, 2020 at 2:15 AM David Mertz  wrote:
> >
> > This is a digression, but does anyone have a nice example IN PYTHON of
> arriving at a NaN without going through infinity. I think Julia is right
> and Python is wrong about '0/0', but as things are, that's not an example.
> >
>
> Not sure why "without going through infinity" is relevant, but you can
> always just use float("nan") to get one, and I'm sure there are other
> calculations that result in nan. It's just that 0/0 (like any other
> operation that involves division by zero, including 0**-1) immediately
> raises, rather than silently returning a nan.
>

Like I said, digression. I teach ieee-754 pretty often, or at least touch
on it. I want to demonstrate to students that they might have NaN values to
consider.

Constructing one purely artificially with 'float("nan")' doesn't make the
point well. Some operation that ends up with overflow infinities that are
divided or subtracted is OK. But it would be nice to show a clean example
where NANs arise without infinities arising first.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EUL4LL2WYRAH2TMYEAGAJMT3WTODWIX6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[David Mertz]

This is a digression, but does anyone have a nice example IN PYTHON of
arriving at a NaN without going through infinity. I think Julia is right
and Python is wrong about '0/0', but as things are, that's not an example.

On Sun, Jul 5, 2020, 12:05 PM Chris Angelico  wrote:

> On Mon, Jul 6, 2020 at 1:58 AM Steven D'Aprano 
> wrote:
> > Python makes it quite hard to get a NAN from the builtins, but other
> > languuages do not. Here's Julia:
> >
> > julia> 0/0
> > NaN
> >
> > So there's at least one NAN which means *there is no correct answer*.
>
> >>> 1e1000-1e1000
> nan
>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/TDIWUV4DCURDEKFMFVY2YZZWQ6ODWOFP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NZNMC3KSCKMBGIHXDJZPE5T4RI3TO3CE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Is possible some day add Uniform Function Call Syntax (UFCS) in Python like Dlang?

[David Mertz]

A large number of topics have come up that wish to make Python more
"fluent" (https://en.wikipedia.org/wiki/Fluent_interface).  This is an
example of the same general idea.  In every single case, the goal has been
almost universally rejected as opposite Python's attitude.

Converting fun(a) to a.fun() might be possible automatically, but if most
such fun() functions return None because they are mutating functions, this
does not get chaining of methods, which is the underlying goal.  While
technically, Python *could* usually return self from most method calls (and
libraries like Pandas do exactly that), the Python developers have very
deliberately decided that they don't want that.

On Sun, Jun 28, 2020 at 11:00 AM  wrote:

> Is possible some day add Uniform Function Call Syntax (UFCS) in Python
> like Dlang?
>
> Example:
> https://tour.dlang.org/tour/en/gems/uniform-function-call-syntax-ufcs
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UBGJ7K5YAYZSREN2QSTUEMLAS4JMYUCU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NPVTE7AH3BREPUGEKY64J3PYZAXTZVD6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[David Mertz]

On Sun, Jun 28, 2020, 10:19 AM Jeff Allen
> Now the commit message is the thing that causes me to write. It contains
a long justification for the change. It need only have said that we've
decided not to enforce S rules. It is somewhat offensive, since it
asserts as fact an opinion the community evidently does not hold
universally, which is that the recommendation to use a standard form of
English is "upholding relics of white supremacy".

The commit message is simply silly. It introduces numerous contentious and
false claims that have nothing whatsoever to do with the small wording
change. It misunderstands how language, culture, history, and indeed white
supremacism, work.

I would recommend amending the commit message.

The underlying change itself is reasonable, and to my mind a small
improvement. There was unnecessary specificity in using Strunk and White as
reference, and not, say, William Zinsser's _On Writing Well_, which is
almost as well known. In the concrete, it would be exceedingly rare for
these to provide conflicting advice on a specific code comment.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N7Z4HDYP7CV75GRCBLMZGSFEDZTFIVQF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[David Mertz]

My point is that _Elements of Style_ is not a set of rules. It's a nice
book with generally good advice; it's not a style guide in a formal sense.
If we wanted rules, _The Chicago Manual of Style_ or the_Associated Press
Style Guide_ would be much closer to that. But neither of those actually
removes subjectivity about what's clear either.

On Sat, Jun 27, 2020, 1:24 AM Steven D'Aprano  wrote:

> On Sat, Jun 27, 2020 at 12:37:35AM -0400, David Mertz wrote:
>
> > Objectively, could I have dropped "most"? Both "most famous"?
> >
> > Could you have dropped "still," objectively?
>
> Yes, we probably could have. What's your point? I'm not arguing in
> favour of S here on this mailing list, and I've explicitly said that I
> don't even like their rules.
>
> But having a set of rules as (1) guidance for those who need guidance,
> especially those who have difficulty telling what is *clear and
> understandable to others*, and (2) for use in disputes, is not a bad
> thing.
>
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/EN2RMKA5ZACAW5F4FWTKOYILIUAVHJOR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3MOFSENG4TYK3BYGRJT36ZG4IWT3ETRE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[David Mertz]

Objectively, could I have dropped "most"? Both "most famous"?

Could you have dropped "still," objectively?

On Sat, Jun 27, 2020, 12:34 AM Steven D'Aprano  wrote:

> On Fri, Jun 26, 2020 at 11:36:47PM -0400, David Mertz wrote:
> > On Fri, Jun 26, 2020, 8:40 PM Steven D'Aprano 
> wrote:
> >
> > > "Clear and easily understandable" is subjective. What is clear and
> > > understandable to me may be impenetrably confusing to others, or
> > > obnoxiously dumbed down.
> > >
> >
> > Strunk and White's most famous mandate of "omit needless words" is
> likewise
> > subjective.
>
> Take the word out of the sentence, and does the sentence still mean the
> same thing? Then the word was needless. That's an objective test.
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/SXUJNGR6EFCCPYJ7Y6RWKWMM72K33LPI/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DYPNEGXRLMXVIC5HT6DHRMH6IAGGREDI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[David Mertz]

On Fri, Jun 26, 2020, 8:40 PM Steven D'Aprano  wrote:

> "Clear and easily understandable" is subjective. What is clear and
> understandable to me may be impenetrably confusing to others, or
> obnoxiously dumbed down.
>

Strunk and White's most famous mandate of "omit needless words" is likewise
subjective.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KDFW46TNDBK7E3OOMCW5LY3VNKFDSVMG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Amend PEP-8 to require clear, understandable comments instead of Strunk & White Standard English comments

[David Mertz]

Your minor change is certainly an improvement.  _The Elements of Style_ (
https://en.wikipedia.org/wiki/The_Elements_of_Style) is certainly a good
text, but it's not even actually a style guide in the formal sense.

On Fri, Jun 26, 2020 at 5:26 PM Keara Berlin  wrote:

> Hi all, this is a very small change, but I thought I would field it here
> to see if anyone has suggestions or ideas. Instead of requiring that
> comments be written in Strunk & White Standard English, PEP-8 should
> require instead that English-language comments be clear and easily
> understandable by other English speakers. This accomplishes the same goal
> without alienating or putting up barriers for people (especially people of
> color) whose native dialect of English is not Standard English. This change
> is a simple way to correct that while maintaining the original intent of
> the requirement. This change may even make the requirement more clear to
> people who are not familiar with Strunk & White, since for programmers, the
> main relevant aspect of that standard is "be clear and concise;" simply
> saying that instead of referencing Strunk & White may communicate this more
> effectively.
> Here is the current line in PEP-8: "When writing English, follow Strunk
> and White."
> I propose changing this line to "When writing English, ensure that your
> comments are clear and easily understandable to other English speakers."
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/AE2M7KOIQR37K3XSQW7FSV5KO4LMYHWX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LRV7JKNXMKVWMX3VUQ7BRAEGL53YWHPL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extend the range functionality to allow infinite ranges

[David Mertz]

How on earth is "count()" not easily available?

On Fri, Jun 19, 2020 at 12:23 PM  wrote:

> Proposal:
> range(start, ..., step)
> should function like
> itertools.count(start, step)
>
> Reason:
> It's pretty common to see people do things where they increment a count
> within a while True loop, and it would be nice to have something easily
> available for people to use to replace this.
>
> Usefulness:
> There is definitely a use for this, as the type of code mentioned above is
> common, and itertools.count is commonly used as well.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/6WEV73JM4EARGAYT6JIFH35Y5MRLK3MX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6YQLTM34KL2J6HILHQKKHH7WL7OV6A46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: json library: non-standards-compliant by default, and what to do about it.

[David Mertz]

Apart from not making it clear that allow_nan=False is about RFC compliance
whether than for some other purpose, it also doesn't mean what it says. The
argument really means "allow_nan_and_infs."

On Wed, Jun 17, 2020, 8:18 AM Greg Ewing 
wrote:

> On 17/06/20 5:42 pm, David Mertz wrote:
> > I think the argument 'allow_nan' is poorly spelled. Spelling it 'strict'
> > would have been much better. Maybe 'conformant'.
>
> It should be called 'json_me_harder'.
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/J2ZXDXGEY2YHG2AIJLADYHJQLQGMP6UX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: json library: non-standards-compliant by default, and what to do about it.

[David Mertz]

On Wed, Jun 17, 2020, 1:30 AM Steven D'Aprano  wrote:

> What exactly is getting in the way here? Standards do change. One standard
> (JSON) is not capable of representing all values from another standard
> (IEEE-754). Removing NANs and INFs would break floating point software
> everywhere, and a lot of hardware too. Adding support for them to JSON
> would be an enhancement, not a breakage.


I can't speak for Guido, of course. But it seems to me that changing the
JSON standard used by many languages and libraries, would be a long uphill
walk. Not because it's perfect, but simply because there is a lot of
institutional inertia and varying interested parties.

That said, Python is far from alone in supporting "JSON+" as an informal
extension that adds +/-Infinity and NaN. Those balls are definitely work
having available.

I think the argument 'allow_nan' is poorly spelled. Spelling it 'strict'
would have been much better. Maybe 'conformant'. I'm not sure I hate the
spelling enough to propose a change with depreciation period, but I
certainly wouldn't oppose that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PZ56EUB3L4XLHESU74CNDDLQQMQFG62Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: json library: non-standards-compliant by default, and what to do about it.

[David Mertz]

On Tue, Jun 16, 2020 at 11:55 PM Christopher Barker 
wrote:

> Well, maybe not "Decimal", but it IS, in fact decimal -- i.e. base 10 --
> it can only exactly represent  values that can be exactly represented in
> base 10.
> Which is why, performance aside, it would be better if a JSON number
> mapped to a Python Decimal, rather than a float.Which I'd still like to see
> happen, at least optionally.
>

I strongly agree.  simplejson has `use_decimal=True`, which is
straightforward.  The standard library would require users painfully define
their own subclasses of JSONEncoder and JSONDecoder that have the desired
Decimal behavior (passing in the `cls=` argument to use the custom
classes).  It's doable, but it should *just work* with a switch instead.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ILNTYE7CY7HX4GW26SH4NTA2ORSA5DMO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: json library: non-standards-compliant by default, and what to do about it.

[David Mertz]

On Tue, Jun 16, 2020, 8:42 PM Steven D'Aprano  wrote:

> Please forgive the stupid question, but given that the JSON standard is so
> obviously broken (being unable to serialise valid values from a supported
> type, what on earth were they thinking???), wouldn't all this time and
> energy be better aimed at fixing the standard rather than making Python's
> JSON encoder broken by default?
>

I do not support changing the default in the Python library.

However, I do want to note that JSON is not "broken." The data that gets
represented in JSON may have the type "Number" (RFC 7159). That is, there
is no concept of IEEE-754 floating point, or integer, or Decimal, etc.
There is simply the type "Number" that a particular programming language
may approximate in one of its available types.

The type Number has no specific semantics, only a grammar for what
sequences of characters qualify. There is a large intersection with the
things representable in IEEE-754, but it's neither a subset not superset of
that. From the RFC:

A JSON number such as 1E400 or 3.141592653589793238462643383279

may indicate potential interoperability problems, since it

suggests that the software that created it expects receiving

software to have greater capabilities for numeric magnitude

and precision than is widely available.



On the other side, strings like -inf, Infinity, or NaN are not matched by
the grammar.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OAUGLXBVRRG5FA4KYC63G5AYMBZML25O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: approximate equality operator ("PEP 485 follow-up")

[David Mertz]

On Sun, Jun 14, 2020 at 7:49 PM Oscar Benjamin 
wrote:

> > > I've had occasion to use math.isclose(), np.isclose(), and
> np.allclose()
> > > quite often.
> > Can you elaborate a bit on the kinds of things you use them for?
>
> I can't elaborate on David's use but in my own experience these
> functions are mostly useful for interactive checking or for something
> like unit tests. They can be used extensively in the testing code for
> projects with a lot of floating point functions.


At times I have computations which *should be* the same mathematically, but
are carried out through a different sequence of specific computations.  One
common example is in parallel frameworks where the order of computation is
indeterminate because multiple workers/threads/processes are each
calculating portions to aggregate.

Another related case is when I call some library to do an operation, but I
did not write the library, nor do I understand its guts well.  For example,
the tensor libraries used in neural networks that will calculate a loss
function.  Occasionally I'd like to be able to replicate (within a
tolerance) a computation the library performs using something more general
like NumPy.  Having a few ulps difference is typical, but counts as
validating the "same" answer.

Another occasion I encounter it is with data measurements.  Some sort of
instrument collects measurements with a small jitter.  Two measurements
that cannot be distinguished based on the precision of the instrument might
nonetheless be stored as different floating point numbers.  In that case, I
probably want to be able to tweak the tolerances for the specific case.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VJSIJ3VHKVJQUZ6ZSXWY7VCBRXSOQKVJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: approximate equality operator ("PEP 485 follow-up")

[David Mertz]

On Sun, Jun 14, 2020, 10:22 AM Greg Ewing 
wrote:

> On 15/06/20 12:39 am, Sebastian M. Ernst wrote:
> > It's such a common problem when dealing with floating point numbers
>
> Is it really? I've done quite a lot of work with floating
> point numbers, and I've very rarely needed to compare two
> of them for almost-equality. When I do, I always want to
> be in control of the tolerance rather than have a default
> tolerance provided for me.
>

I've had occasion to use math.isclose(), np.isclose(), and np.allclose()
quite often. And most of the time, the default tolerances are good enough
for my purpose. Note that NumPy and math use different algorithms to define
closeness, moreover.

But it's more often than rare that I want to choose a different tolerance
(or switch between absolute and relative tolerance). Adding an operator
adds an impediment to refactoring to change tolerance. I'm more concerned
about that problem than I am with the few extra characters needed to call a
function.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PVKZZC2GFQDWPUHI2QU7LKP2SBLA7C2G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bringing the print statement back

[David Mertz]

On Wed, Jun 10, 2020, 8:04 PM Jonathan Crall  wrote:

> I wouldn't mind if this *only *worked for the specific characters
> "print".
>

I often swap out 'print' for 'pprint', depending on the presentation I'm
assuming for. Obviously, I need a 'from pprint import pprint' earlier. It
"just works" and is seamless.

It would become an impedance mismatch if it didn't include 'pprint'. But
what about my 'custom_print' I use occasionally...
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UBZDXBZJE4HCS6ROVRDUZC2X7YWVGKNR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bringing the print statement back

[David Mertz]

I think I have almost the same feeling as Naomi and Gregory. After 10+
years of print statement being an error, typing it as one remains my SECOND
most common syntax error.

My FIRST most common is omitting the colon that introduces blocks... Which
we also don't actually need. I've made that mistake for 20+ years. And I
never used 0.8 or whatever version didn't have colon, only since 1.4.

Despite my occasional typos, I don't think I want the inconsistency of
being able to make calls either with or without parens.

I've used TCL, REXX, Ruby, REBOL, Haskell, and bash with pleasure, and
enjoy their "arguments after a space." I've used Perl even more, and never
enjoyed it. :-) But different languages are pleasant in their different
consistency, and this feels out of place in modern Python.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JCQ55VAAFNWP2GXFFZ63OMZZNUJME3DI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Delayed computation

[David Mertz]

On Fri, May 29, 2020 at 5:06 PM Dominik Vilsmeier 
wrote:

> I'm still struggling to imagine a real use case which can't already be
> solved by generators. Usually the purpose of such computation graphs is to
> execute on some specialized hardware or because you want to backtrack
> through the graph (e.g. Tensorflow, PyTorch, etc). Dask seems to be similar
> in a sense that the user can choose different execution models for the
> graph.
>

If you look for Dask presentations, you can find lots of examples, many of
real world use.  You want those that use dask.delayed specifically, since a
lot of Dask users just use the higher-levels like dask.dataframe.  But off
the cuff, here's one that I don't think you can formulate as generators
alone:

def lazy_tree(stream, btree=BTree(), new_data=None):
"""Store a b-tree of results from computation and the inputs

For example, a modular  form of large integers, or a PRG
"""
for num in stream:
# We expect a few million numbers to arrive
result = delayed expensive_calculation(num)
if result == btree.result:
if new_data is None:
btree.result = new_data.result
btree.inits = [new_data.init]
else:
# Add to the inits that might produce result
btree.inits.append(num)
elif result < btree.value.result:
btree.left = delayed lazy_tree(stream, btree.left,
InitResultPair(num, result))
elif result > btree.value.result:
btree.right = delayed lazy_tree(stream, btree.right,
InitResultPair(num, result))
return btree

I probably have some logic wrong in there since I'm doing it off the cuff
without testing.  And the classes BTree and InitResultPair are hypothetical.

But the idea is we are partitioning on the difficult to calculate result.
Multiple inits might produce that same result, but you don't know without
the computation.  So yes, parallelism is potentially beneficial.  But also
partial computation without expanding the entire tree.  That could let us
run a line like this to only realize a small part of the tree:

btree = lazy_tree(mystream)
my_node = btree.path("left/left/left/right/left/right/right/right")

Obviously it is not impossible to write this without new Python semantics
around delayed computation.  But the scaffolding would be much more
extensive.  And it certainly couldn't be expressed just as a sequence of
generator comprehensions.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KNFDVPYHRUKAQ4PN4O6RUUKDXBYN6S46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Delayed computation

[David Mertz]

On Fri, May 29, 2020 at 1:56 PM Rhodri James  wrote:

> Presumably "delayed" is something that would be automatically applied to
> the actual parameter given, otherwise your call graphs might or might
> not actually be call graphs depending on how the function was called.
> What happens if I call "foo(y=0)" for instance?
>

I am slightly hijacking the thread.  I think the "solution" to the narrow
"problem" of mutable default arguments is not at all worth having.  So yes,
if that was the only, or even main, purpose of a hypothetical 'delayed'
keyword and 'DelayedType', it would absolutely not be worthwhile.  It would
just happen to solve that problem as a side effect.

Where I think it is valuable is the idea of letting all the normal
operations work on EITHER a DelayedType or whatever type the operation
would otherwise operate on.  So no, `foo(y=0)` would pass in a concrete
type and do greedy calculations, nothing delayed, no in-memory call graph
(other than whatever is implicit in the bytecode).

However, I don't really love the 'concretize' operation, which you
correctly identify as a source of new bugs.  I think it is necessary
occasionally, and actually there is no reason it cannot be a plain old
function rather than a keyword.  The DelayedType object is simply some way
of storing a call graph, which is has some kind of underlying
representation.  So the only slightly special function concretize() could
just walk the graph.

I think it would be better if MANY operations simply *implied* the need to
concretize the call tree into a result value.  The problem is, I'm really
not sure exactly which operations, and probably a lot of them could have
arguments in either direction.  My thinking here is inspired by several
data frame libraries that address this issue.

* Pandas is always eager.  Call a method, it does the computation right
away, and usually returns the mutated data frame object (or some new
similar object) to chain methods in a "fluent style."
* Dask.dataframe is always lazy.  It implements 95% of the Pandas API (by
putting Pandas "under the hood").  You can still chain methods, but
everything returns a call graph that the next method uses to build a larger
call graph.  The ONLY time any computation is done when you explicitly call
`df.compute()`
* Vaex is another data frame library that is very interesting.  It is
*almost always* lazy, and likewise allows chaining methods.  You can easily
build up call graphs behind the scenes (either by chaining or with
intermediate names for "computations").  However, the RIGHT collection of
methods know they need to concretize... and they just do the right thing.

Of these, I've used Vaex the least by far.  However, my impression is that
it makes the right decisions, and users rarely need to think about what is
going on behind the scenes (other than that operations on big data is darn
fast).  Still, "things with data frames" is more specialized than
"everything one can do in Python."  I'm not sure what the right answers are.

For example, my first intuition is that `print(x)` is a good occasion to
implicitly concretize.  If you really want the underlying delayed, maybe
you'd write `print(x.callgraph)` or something similar.  But then, sticking
in a debugging print might wind up being an expensive thing to do in that
case, so...


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G7ZYBSFC6L3FFPFODPZWMHV72MLIQPMU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Fri, May 29, 2020 at 1:12 PM Alex Hall  wrote:

> def foo(a=17, b=42,, x=delayed randint(0,9), y=delayed randrange(1,100)):
>
>> if something:
>> # The simple case is realizing a direct delayed
>> val = concretize x
>> elif something_else:
>> # This line creates a call graph, not a computation
>> z = ((y + 3) * x)**10
>> # Still call graph land
>> w = a / z
>> # Only now do computation (and decide randoms)
>> val = concretize w - b
>>
>
> But if I understand correctly, a delayed value is concretized once, then
> the value is cached and remains concrete. So if we still have early
> binding, then x will only have one random value, unlike Stephen's lambda
> which generates a new value each time it's called.
>

I don't think that's required by the example code I wrote.  The two things
that are concretized, 'x' and 'w-b', are assigned to a different name.  I'm
taking that as meaning "walk the call graph to produce a value" rather than
"transform the underlying object".  So in the code, x and y would stay
permanently as a special delayed object.

This is a lot like a lambda, I recognize.  The difference is that the
intermediate computations would look through the right-hand side and see if
anything was of DeferredType. If so, the computation would become "generate
the derived call graph" rather than "do numeric operations."  So, for
example, 'w' also is DeferredType (and always will be, although it will
fall out of scope when the function ends).  In some cases, generating a
call graph can be arbitrarily cheaper than the actual computation; passing
around the call graph rather than the result can hence be worthwhile.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M2PWLQXCT63YDOBKH7AY4SYTK72GWW2V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

> > Where do we draw the line? How much of the behaviour of the
> > function do we want to move into the header?
>
> Well obviously we don't draw the line until we've fallen down the
> slippery slope and the entire body of the function is part of the
> header!
>

Sure, what's the problem? :-)

def hof(*, op=lambda F: lambda x: F(x), fn=lambda x: x**2, val=None):
# Default is squaring, do what you want.
return op(fn)(val)

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LUD723DCIWI7D3XR54JALXG5MSMQP5J2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Fri, May 29, 2020 at 3:19 AM Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> # Just too ugly for me!
> def foo(x=lambda: random.randint(0,9)):
>
> x = x()
> # ...
>

I think this is a perfect example of where my desired "delayed" (or
"deferred") construct would be great.  There are lots of behaviors that I
have not thought through, and do not specify here.  But for example:

def foo(a=17, b=42,, x=delayed randint(0,9), y=delayed randrange(1,100)):
if something:
# The simple case is realizing a direct delayed
val = concretize x
elif something_else:
# This line creates a call graph, not a computation
z = ((y + 3) * x)**10
# Still call graph land
w = a / z
# Only now do computation (and decide randoms)
val = concretize w - b

I do not find this particularly ugly, and I think the intent is pretty
clear.  I chose two relatively long words, but some sort of shorter
punctuation would be possible, but less intuitive to my mind.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UOXBYGFPJL6EK3EVFYT3LJJHACYEAQ43/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python __main__ function

[David Mertz]

I agree with Paul's sentiment. We do not need to bless just one way of
writing scripts. Code review or organization style guides, sure. But not at
language level.

I also write scripts with no explicit __main__. But I rarely name them as
.py. In the style of Unix commands, they have no extension, but have a
pound-bang at the start. This has the side effect of making them much less
straightforward to import at all.

On Fri, May 29, 2020, 3:29 AM Paul Moore  wrote:

> Also, I routinely write scripts that have no `if __name__ ==
> '__main__'` line at all, they just run - no-one should ever import
> them, so it makes no difference. And I exit (in multiple places) using
> `raise SystemExit("reason")`.
>
> My point being that yes, there are *lots* of ways of writing Python
> scripts/programs. Why "bless" one of them as being somehow superior?
>
> Paul
>
> On Fri, 29 May 2020 at 02:02, Greg Ewing 
> wrote:
> >
> > On 29/05/20 8:05 am, tritium-l...@sdamon.com wrote:
> >
> > > People write main entry points that are not exactly this?
> > >
> > > If __name__ == '__main__':
> > >  sys.exit(main(sys.argv[1:]))
> >
> > It's not clear that exiting with the return value of main() is
> > the most Pythonic thing to do -- it's more of a C idiom that
> > doesn't seem so useful when exceptions exist.
> >
> > --
> > Greg
> > ___
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UEFQEB3YZIB45KWAHPBWSZLFDBSA672Y/
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/JMRF6PUCOMMLAAYWZKZR2E22EM4YIUS4/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NKHWLTFXEHFTLYEFON2LXI6YU25IIYD3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python __main__ function

[David Mertz]

On Thu, May 28, 2020, 3:06 PM Chris Angelico  wrote:

> There aren't multiple entry points, though. There would be multiple
> blocks of code that are skipped if the module is imported, but
> executed if it's run as a script. Remember, Python code is NOT
> declarative. That 'def' statement is an actual executable bit of code.
>

I take back my comment. I think I may have used the multiple __main__
blocks occasionally, for reasons Chris notes. Like I want something
configured BEFORE a function is defined (maybe a global that only makes
sense when run as script).

However, I haven't done it for a long while. I think when I was younger I
didn't value encapsulation sufficiently. That said, I don't want the
capability to change. There are lots of things I don't do stylistically,
but I don't want striken from the language.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FPDFDFM66BZSHPYM3ZNYHEMZ4PTR5T3L/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python __main__ function

[David Mertz]

On Thu, May 28, 2020, 12:17 PM  wrote:

> The OP is proposing as a possibility: "we could require user to have only
> one if __name__ == '__main__':". In that case, functionality will be
> reduced, won't it?
>

I don't support the proposal. However, I've also never written a script
with multiple 'if __name__ == __main__' lines, and I think if I saw one I'd
complain in code review.

It's hard for me to imagine how multiple entry point to a script can be a
good thing.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ARMVJL5BXEVZAEZRBWWGMRGZ2IAOPTMR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Tue, May 26, 2020 at 8:30 PM Greg Ewing 
wrote:

> "Did the user supply a value for this optional argument?" is
> a simple and reasonable question to ask. It deserves to have a
> simple and direct way of answering it that always works.
>

It does so very little that I would definitely not want syntax ONLY for
that.  The example of lru_cache() is god here... None is a sentinel, but it
is NOT the default value.  So there is a value of 128 supplied, even if the
user doesn't supply one.  Changing syntax for something that doesn't even
actually save a line, isn't worth it.  I'm assuming you mean something
along the lines of:

def foo(a, b, mode=):
if undef mode:
mode = whatever

It's really just a sentinel, no better than None.  If we want to save more,
we're back to PEP 505 and None-coalescing.  I'm feeling more sympathetic to
that than I did in earlier discussion.  But I'm still a bit horrified by
some of the use cases that were presented at that time (not directly in the
PEP).  E.g.:

data = json.load(fh)
needle ?= data?.child?.grandchild?[key]?[index]?.attr

When it looks like that, I get visions of APL or Perl unreadability.  So I
guess what I'm sympathetic with is about 1/3 of PEP 505.  Like only:

stuff = this ?? that

Yes, there are still rare non-None sentinels to deal with differently, but
that is 90% of the issue being discussed.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XBA4VCCRP4LLVZ3S7JYBZMDADF3C4ZPS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Tue, May 26, 2020 at 12:53 PM Steven D'Aprano 
wrote:

> >>> len(Undef)
> TypeError: len() takes exactly one argument (0 given)
>
> since len takes no default values. But if Undef is considered to be an
> actual object, like any other object, we ought to get this:
>

JavaScript and R, for example, do have a special pseudo-values for "even
more missing".  In JS, it is null vs undefined.  In R, it is... well,
actually NULL vs. NA vs. NaN.  E.g.:

> c(NULL, NA, NaN, 0, "")
[1] NA"NaN" "0"   ""

The NULL is a syntactic placeholder, but it's not a value, even a sentinel
(i.e. it doesn't get in the array).  Python *could* do that.  But it would
require very big changes in many corners of the language, for no
significant benefit I can see.

In other words, I agree with Steven.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2TMKZH3HMWBVQRLDF6WIN5GFENSRAAYG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Tue, May 26, 2020 at 12:02 PM Dominik Vilsmeier 
wrote:

> The NumPy, deque, and lru_cache cases are all ones where None is a perfect
>> sentinel and the hypothetical 'undef' syntax would have zero value.
>
> For both `deque` and `lru_cache` None is a sensible argument so it can't
> act as a sentinel. It just happens that these two cases don't need to check
> if an argument was supplied or not, so they don't need a sentinel.
>
I think getting caught up in the specific functions is a bit of a rabbit
hole.  I think the general point stands that None is usually fine as
sentinel, and in the rare cases it's not, it's easy to define your own.

But lru_cache() and deque() seem kinda obvious here.  The default cache
size is a finite number, but we can pass in any other positive integer for
maxsize.  Therefore, nothing in the domain of positive integers can carry
the sentinel meaning "unbounded" or "infinite."  We need a special value to
signal different behavior (in other words, a sentinel).

In C/C++, we'd probably use -1. It's an integer, but not one that makes
sense as a size.  I suppose we might use float('inf') as a name for
unbounded, but then there are issues of converting that float to an int...
or really just doing what currently happens of taking a different code
path.  For every practical purpose, sys.maxsize would be fine.  It is not
technically infinite, but it's far larger than the amount of memory you can
possibly cache.  I do that relative often myself... "really big" is enough
like "infinite" for most practical purposes (I could make an even bigger
integer in Python, of course, but that it mnemonic and plenty big).

Or we could use a string like '"UNBOUNDED"'. Or an enumeration. Or a module
constant.  But since there is just one special state/code path to signal,
None is a perfect sentinel.

If `keepdims` is not supplied, it won't be passed on to sub-classes; if it
> is set to None then the sub-class receives `keepdims=None` as well:


Yeah, OK, there's a slight different if you subclass ndarray. I've never
felt an urge to do that, and never seen code that did... but it's possible.


> For `np.concatenate` None is a meaningful argument to `axis` since it will
> flatten the arrays before concatenation.
>

This is again very similar to the sentinel in lru_cache().  It means "use a
different approach" to the algorithm.  I'm not sure what the C code does,
but in concept it's basically:

sentinel = None
if axis is sentinel:
a = a.flatten()
b = b.flatten()
axis = 0
... rest of logic ...


> I was wondering if anyone would mention Pandas, which is great, but in
> many ways and abuse of Pythonic programming. There None in an initializing
> collection (often) gets converted to NaN, both of which mean "missing",
> which is something different. This is kind of an abuse of both None and
> NaN... which they know, and introduced an experimental pd.NA for exactly
> that reason... Unfortunately, so far, actually using of.NA is cumbersome,
> but hopefully that gets better next version.
>
> I wouldn't say it's an abuse, it's an interpretation of these values.
> Using NaN has the clear advantage that it fits into a float array so it's
> memory efficient.
>

I know why they did it.  But as a data scientist, I sometimes (even often)
care about the difference between "this computation went wonky" and "this
data was never collected."  NaN is being used for both meanings, but they
are actually importantly different cases.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E5L6OXTLS773CZ3UADJIG3WAYW35XVCZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

All of those uses, including those where you say otherwise, treat None as a
sentinel. In the iter() case, the optional seconds argument is *called*
'sentinel'. Guido recently mentioned that he had forgotten the two argument
form of iter(), which is indeed funny... But useful.

Well, ok functions.reduce() really does make it's own sentinel in order to
show NONE as a "plain value". So I'll grant that one case is slightly
helped by a hypothetical 'undef'.

The NumPy, deque, and lru_cache cases are all ones where None is a perfect
sentinel and the hypothetical 'undef' syntax would have zero value.

I was wondering if anyone would mention Pandas, which is great, but in many
ways and abuse of Pythonic programming. There None in an initializing
collection (often) gets converted to NaN, both of which mean "missing",
which is something different. This is kind of an abuse of both None and
NaN... which they know, and introduced an experimental pd.NA for exactly
that reason... Unfortunately, so far, actually using of.NA is cumbersome,
but hopefully that gets better next version.

Within actual Pandas and function parameters, None is always a sentinel.

On Tue, May 26, 2020, 4:48 AM Dominik Vilsmeier 
wrote:

> On 26.05.20 06:03, David Mertz wrote:
>
> On Mon, May 25, 2020, 11:56 PM Christopher Barker
>
>> well, yes and no. this conversation was in the context of "None" works
>> fine most of the time.
>>
>
> How many functions take None as a non-sentinel value?! How many of that
> tiny numbers do so only because they are poorly designed.
>
> None already is an excellent sentinel. We really don't need others. In the
> rare case where we really need to distinguish None from "some other
> sentinel" we should create our own special one.
>
> The only functions I can think of where None is appropriately non-sentinel
> are print(), id(), type(), and maybe a couple other oddball special ones.
>
> Seriously, can you name a function from the standard library or another
> popular library where None doesn't have a sentinel role as a function
> argument (default or not)?
>
> * From the builtins there is `iter` which accepts a sentinel as second
> argument (including None).
> * `dataclasses.field` can receive `default=None` so it needs a sentinel.
> * `functools.reduce` accepts None for its `initial` parameter (
> https://github.com/python/cpython/blob/3.8/Lib/functools.py#L232).
> * There is also [`sched.scheduler.enterabs`](
> https://github.com/python/cpython/blob/v3.8.3/Lib/sched.py#L65) where
> `kwargs=None` will be passed on to the underlying `Event`.
>
> For the following ones None could be a sentinel but it's still a valid
> (meaningful) argument (different from the default):
>
> * `functools.lru_cache` -- `maxsize=None` means no bounds for the cache
> (default is 128).
> * `collections.deque` -- `maxlen=None` means no bounds for the deque
> (though this is the default).
>
> Other example functions from Numpy:
>
> * [`numpy.concatenate`](
> https://numpy.org/doc/1.18/reference/generated/numpy.concatenate.html) --
> here `axis=None` means to flatten the arrays before concatenation (the
> default is `axis=0`).
> * Any function performing a reduction, e.g. [`np.sum`](
> https://numpy.org/doc/1.18/reference/generated/numpy.sum.html) -- here if
> `keepdims=` is provided (including None) then it will passed to the `sum`
> method of ndarray-sub-classes, otherwise not.
> * [`np.diff`](
> https://numpy.org/doc/1.18/reference/generated/numpy.diff.html) supports
> prepending / appending values prior to the computation, including None
> (though that application is probably rare).
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/U4U7X36COLQ776LRB4O6O4BEXDXFWHJK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CITAZRZXLAUAHBFPOKMFQZEBJ34OWJBS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

You mean the sentinel is called 'undef'? With undef-coallescing operators?

On Tue, May 26, 2020, 2:14 AM Greg Ewing 
wrote:

> Wild idea: Instead of sentinels, have a way of declaring optional
> arguments with no default, and a way of conditionally assigning a
> value to them if they are not bound.
>
> E.g.
>
>  def order(eggs = 4, spam =):
>  spam ?= Spam()
>
> Here the '?=' means "if spam is not bound, then evaluate the
> rhs and assign it, otherwise do nothing."
>
> --
> Greg
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/4C2WQRFTLBXXQ32IJTWJA4QQKYKBISRC/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XWBH67GJU2BZ5SKOG4F7UN5SO32564JR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Mon, May 25, 2020, 11:56 PM Christopher Barker

> well, yes and no. this conversation was in the context of "None" works
> fine most of the time.
>

How many functions take None as a non-sentinel value?! How many of that
tiny numbers do so only because they are poorly designed.

None already is an excellent sentinel. We really don't need others. In the
rare case where we really need to distinguish None from "some other
sentinel" we should create our own special one.

The only functions I can think of where None is appropriately non-sentinel
are print(), id(), type(), and maybe a couple other oddball special ones.

Seriously, can you name a function from the standard library or another
popular library where None doesn't have a sentinel role as a function
argument (default or not)?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HRHERKG7LQABU6C3CCQ4ASRAX4NLRTAP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python JIT Compilation Thoughts

[David Mertz]

On Mon, May 25, 2020, 5:33 PM Christopher Barker

> see numba:  http://numba.pydata.org/
>
> Also, I"m pretty sure one of the efforts to make a faster python was based
> on LLVM, but it didn't work out as well as hoped -- unladen swallow maybe?
>

Numba is great. Very easy and very fast. Pypy experimented with an LLVM
target but decided it's own JIT was better for various reasons.

>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZEWOLQ7EX6SFDYO6BOTF77VXMMA6CZW6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Delayed Execution

[David Mertz]

Subject changed for tangent.

On Sun, May 24, 2020 at 4:14 PM Dominik Vilsmeier 
wrote:

> output = []
> for x in data:
> a = delayed inc(x)
> b = delayed double(x)
> c = delayed add(a, b)
> output.append(c)
> total = sum(outputs)  # concrete answer here.
>
> Obviously the simple example of adding scalars isn't worth the delay
> thing.  But if those were expensive operations that built up a call graph,
> it could be useful laziness.
>
> Do you have an example which can't be solved by using generator
> expressions and itertools? As far as I understand the Dask docs the purpose
> of this is to execute in parallel which wouldn't be the case for pure
> Python I suppose? The above example can be written as:
>
> a = (inc(x) for x in data)
> b = (double(x) for x in data)
> c = (add(x, y) for x, y in zip(a, b))
> total = sum(c)
>
Obviously delayed execution CAN be done in Python, since Dask is a
pure-Python library that does it.  For the narrow example I took from the
start of the dask.delayed docs, your version look equivalent.  But there
are many, not very complicated cases, where you cannot make the call graph
as simple as a sequence of generator comprehensions.

I could make some contrived example. Or with a little more work, I could
make an actual useful example.  For  example, think of creating different
delayed objects within conditional branches inside the loop.  Yes, some
could be expressed with an if in the comprehensions, but many cannot.

It's true that Dask is most useful for parallel execution, whether in
multiple threads, multiple processes, or multiple worker nodes.  That
doesn't mean it would be a bad thing for language level capabilities to
make similar libraries easier.  Kinda like the way we have asyncio, uvloop,
and curio all built on the same primitives.

But another really nice thing in delayed execution is that we do not
necessarily want the *final* computation. Or indeed, the DAG might not have
only one "final state."  Building a DAG of delayed operations is almost
free.  We might build one with thousands or millions of different
operations involved (and Dask users really do that).  But imaging that
different paths through the DAG lead to the states/values "final1",
"final2", "final3" that share many, but not all of the same computation
steps.  After building the DAG, we can make a decision which computations
to perform:

if some_condition():
x = concretize final1
elif other_condition():
x = concretize final2
else:
x = concretize final3

If we avoid 2/3, or even 1/3 of the computation by having that approach,
that is a nice win where we are compute bound.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XGZMRIYPOIZKLQOG47M5CF6HQGYUCDWA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

On Sun, May 24, 2020, 9:03 PM Rob Cliffe via Python-ideas

> In fact it's *more* intuitive than what we do now (evaluation is strictly
> left-to-right),  as can be seen with composed functions:
>
>  ((path)str)len


Forth was created in 1970. The idea isn't brand new.

C is two years newer (1972). But Algol is 12 years older (1958).

Most languages of the last 30 years seem to be influenced most by C, or at
least the Algol tradition for basic syntax. That doesn't mean it's better,
but it won out for whatever reason.

Smalltalk (1972) is interestingly different from either the Forth or Algol
style. But closer to Forth.

Fwiw, I don't think I'd have much difficulty adjusting to the postfix
calling style, but Python isn't going to do that, even in version 17.3 in
2100.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PP3C5TWPBSWA5ZY2LD5ABGIO3MI66U7O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

On Sun, May 24, 2020 at 6:56 PM Steven D'Aprano  wrote:

> > I use bash a lot, and writing something like this is common:
> > cat data | sort | cut -d; -f6 | grep ^foo | sort -r | uniq -c
>
> And today's "Useless Use Of cat Award" goes to... :-)
>
> sort data | ...
>
> (What is it specifically about cat that is so attractive? I almost
> certainly would have done exactly what you did, even knowing that sort
> will take a file argument.)
>

This is probably going afield since it is a bash thing, not a Python
thing.  But I can actually answer this quite specifically.

When I write a pipeline like that, I usually do not do it in one pass.  I
write a couple of the stages, look at what I have, and then add some more
stages until I get it right.  Many of the commands in the pipeline can take
a file argument (not just sort, also cut, also grep, also uniq...
everything I used in the example).

But I find fairly often that I need to add a step BEFORE what I initially
thought was first processing step.  And then I have to remove the filename
as an argument of that no-longer-first step.  Rinse and repeat.  With `cat`
I know it does nothing, and I won't have to change it later (well, OK,
sometimes I want -n or -s).  So it is a completely generic "data" object
... sort of like how I would write "fluent programming" starting with a
Pandas DataFrame, for example, and calling chains of methods..

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7DRNTPAWCU5SA666E6ZLEXZQVUYCS7VN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

Tiimmy:

> Best I know, f@g applies g first in every language that implements a
>>
> composition operator, and in mathematics. While that may be arbitrary,
>> it's easy to remember:  (f@g)(x)  "looks a heck of a lot more like"
>> f(g(x)) than g(f(x))
>>
>
On Sun, May 24, 2020 at 5:39 PM Guido van Rossum  wrote:

> Dang, you're right. It works as you'd expect without overthinking it. :-)
>

I think one thing that pulls in different directions is that both
composition and piping are useful, and do something closely related.  But
in one the data "goes" forward and in the other the data "goes backward."

I use bash a lot, and writing something like this is common:

cat data | sort | cut -d; -f6 | grep ^foo | sort -r | uniq -c

My data is moving left to right through the operations.  In contrast, I
might write in some hypothetical programming language:

myfilter = uniq[count] ∘ sort[reverse] ∘ grep[^foo] ∘ cut[;,f6] ∘ sort
result = myfilter(data)

(In my just-now-invented PL, the square brackets are some kind of partial
application operators)

Here my data is moving right to left within the composition.  Haskell
spells pipes approximately `>>`, F# as `|>`, R as `%>%`, and so on.  But
some of the same languages also have composition (point-free style) that
reverses the order of operations (using different symbols).

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VBFQU6IBJ5RTF4TLHTS45BAZC6CEATZ6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

On Sun, May 24, 2020, 5:11 PM Alex Hall

> But when you *read* a call to filter(), it's generally pretty obvious
> which argument is which, even if you don't remember the signature. You just
> need to see which one's callable or which one's iterable (few things are
> both). You can probably guess just from the variable names. If you read `f@g`,
> you can only guess if one of `f(g(x))` or `g(f(x))` is nonsense, which I
> think is typically less obvious.
>

I definitely agree. My way of remembering which way filter goes was to try
the wrong one in a shell and get an exception.

In contrast, quite likely both f(g(h(x))) and h(g(f(x))) produce a value,
but only one is the one I want. If x is  string, or a list, or a number, or
a NumPy array, most functions I'd call return something of the same type.
Mutations are usually not order independent.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JKW6HJPPIUSS4IHBTWUGLL4HMMFLOM6T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

On Sun, May 24, 2020, 3:43 PM Guido van Rossum  wrote:

> I’ve never been able to remember whether (f@g)(x) means f(g(x)) or
> g(f(x)). That pretty much kills the idea for me.
>

Well, it means whichever one the designers decide it should mean. But
obviously it's a thing to remember, and one that could sensibly go the
other way.

On the other hand, when I showed an example using filter() a couple days
ago, I had to try it to remember whether the predicate or the iterable came
first. Lots of such decisions are pretty arbitrary.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FWU4J4QKH3WDHQ6FZLMOXXQ35MA5MGU6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[David Mertz]

On Sun, May 24, 2020 at 12:36 PM Alex Hall  wrote:

> OK, let's forget the colon. The point is just to have some kind of
> 'modifier' on the default value to say 'this is evaluated on each function
> call', while still having something that looks like `arg=`. Maybe
> something like:
>
>  def func(options=from {}):
>

I worry so very little about this issue of mutable defaults that this
discussion has trouble interesting me much.  It's a speed bump for
beginners, sure, but it's also sometimes a sort of nice (but admittedly
hackish) way of adding something statefull to a function without making a
class.  On the other hand, I mostly thought it was cool before generators
even existed; nowadays, a generator is more often a useful way to have a
"stateful function" (yes, I know there are some differences; but a lot of
overlap).

The pattern:

def fun(..., option=None):
if option is None:
option = something_else

Becomes second nature very quickly.  Once you learn it, you know it.  A
line of two of code isn't a big deal.

But this discussion DOES remind me of something much more general that I've
wanted for a long time, and that has had long discussion threads at various
times.  A general `deferred` or `delayed` (or other spellign) construct for
language-wide delayed computation would be cool. It would also require
rethinking a whole lot of corners.  I think it would address this mutable
default, but it would also do a thousand other useful things.

Much of the inspiration comes from Dask.  There we can write code like this
simple one from its documentation:

output = []for x in data:
a = delayed(inc)(x)
b = delayed(double)(x)
c = delayed(add)(a, b)
output.append(c)
total = delayed(sum)(output)

However, in that library, we need to do a final `total.compute()` to get
back to actual evaluation.  That feels like a minor wart, although within
that library it has a purpose.  What I'd rather in a hypothetical future
Python is that "normal" operations without this new `delayed` keyword would
implicitly call the .compute().  But things like operators or most function
calls wouldn't raise an exception when they try to combine DelayedType
objects with concrete things, but rather concretize them and then decide if
the types were right.

As syntax, I presume this would be something like:

output = []
for x in data:
a = delayed inc(x)
b = delayed double(x)
c = delayed add(a, b)
output.append(c)

total = sum(outputs)  # concrete answer here.

Obviously the simple example of adding scalars isn't worth the delay
thing.  But if those were expensive operations that built up a call graph,
it could be useful laziness.  Or for example:

total = sum(outputs[:1_000_000)  # larger list, don't compute everything

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IDRLUHRALKZOWCDZPQGB4PGPZU5ECN5G/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Function composition

[David Mertz]

On Sun, May 24, 2020 at 11:21 AM Steven D'Aprano 
wrote:

> > But how would you go about getting a .__matmul__ attribute onto all
> > functions. For ones you write yourselves, you could decorate them at
> > definition. What about all the other functions though.
>
> As a functional programming fan, you might not know about this, but we
> object oriented programming people have this concept called
> "inheritance" where we would add the attribute to FunctionType once, and
> just like magic every function would support it!
>

I think I heard of that inheritance thing somewhere! :-)

My thought is really that there's no way we're going to get .__matmul__
added (with the meaning "compose") to FunctionType.  So I was thinking
about what one could do with a separate module.  I.e. something that could
live on PyPI... before, perhaps someday, being added to standard library.

Is there an evil way to monkey patch FunctionType? I.e. without actually
recompiling a fork of the interpreter.  Alex' forbiddenfruit.curse is
pretty cool looking.  But it does require explicitly cursing each
individual function that might be composed.

Alex Hall:

> But seriously, I don't see that much point to this idea. It's just
> slightly more concise while not being particularly readable or beginner
> friendly.
>
> sorted(paths, key=len @ str)
> sorted(paths, key=lambda p: len(str(p)))
>

I think once you compose three or more functions, lambda starts to look
pretty bad.  This is only slightly mitigated by one of those proposals for
"a more concise lambda" that occur intermittently. And if you actually save
a composed function under a meaningful name for later use, that much more
so.  Of course, I'm really not all that bothered by `my_op = compose(fun1,
fun2, fun3, fun4)`.  The @ might be cool, but writing my own HoF compose()
isn't hard.

I've been playing more lately with R Tidyverse.  It's pipe with currying of
first argument is actually really nice.  The pipe operator is god-awful
ugly.  But other than that it works nicely.  For example:

iris %>%  group_by(Species) %>%  summarize_if(is.numeric, mean) %>%
ungroup() %>%  gather(measure, value, -Species) %>%  arrange(value)


It's not abstract composition since it always starts with a concrete object
the several operations work on.  But it is some of the same feel.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/X5N4UWZJYWMFCRSXLJW36THUHAJSOHXI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Function composition

[David Mertz]

Changed subject line. This is far from original topic.

On Sun, May 24, 2020, 9:35 AM Ram Rachum  wrote:

> What's wrong with using @? If I understand correctly, it's used for matrix
> multiplication, which is far enough from function composition to avoid
> confusion. And it's slightly similar visually to a circle.
>

I like @. Function composition is kinda-sorta enough like for product that
I can even make sense of the same operator.

But how would you go about getting a .__matmul__ attribute onto all
functions. For ones you write yourselves, you could decorate them at
definition. What about all the other functions though. I suppose this is
possible:

@compasable
def foo(x):
return x + "foo"

len = composable(len)

newfun = len @ foo @ str.upper
nbar = newfun("bar")  # 6

I deliberately didn't decorate str.upper in the example since it shouldn't
matter for a left-associative operator.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RZLALB7SIIJTJPY55RYWKA644UYTRLBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: len(path) == len(str(path))

[David Mertz]

Hi Steven,

On Sun, May 24, 2020, 8:14 AM Steven D'Aprano

> sorted(paths, key=len∘str, reverse=True)
> *semi-wink*
>

Do you have an evil twin with whom you share email.

Yesterday you were arguing against functional programming style on the
grounds that filtering based on a predicate was too obscure represented as
'filter(pred, ...))'

Today you want full function composition.

Fwiw, I like function composition. This example makes a good case for it.
Too bad that symbol isn't on most keyboards. I don't think 'compose(len,
str)' looks so terrible, but it's longer.


>
> > So I implemented `PurePath.__len__` as `str(len(path))`.
> >
> > Serhiy and Remi objected, because it might not be obvious that the length
> > of the path would be the length of string.
>
> It isn't obvious to me that the length of a path is the length of the
> string representation of that path.
>
> I thought that the length of the path would be the number of path
> components:
>
> /tmp/file  # length 2
>
> I wouldn't have predicted that was the length of the string
> representation of the path.
>
>
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/RHOPEZHLIPN5OVHNQGGT55VF5OBG4RJX/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SS2SD6COHAZ7ILSA7XW3DMI3QSSQSNNL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: len(path) == len(str(path))

[David Mertz]

Like several other commenters, I would expect len(path) to be its depth,
not the length of its string representation.

On Sun, May 24, 2020, 7:30 AM Ram Rachum  wrote:

> Hi everyone,
>
> I submitted a PR today, and Serhiy decided it needs a discussion on
> python-ideas and agreement from core developers before it could go forward.
>
> BPO: https://bugs.python.org/issue40752
> PR: https://github.com/python/cpython/pull/20348
>
> Today I wrote a script and did this:
>
> sorted(paths, key=lambda path: len(str(path)), reverse=True)
>
> But it would have been nicer if I could do this:
>
> sorted(paths, key=len, reverse=True)
>
> So I implemented `PurePath.__len__` as `str(len(path))`.
>
> Serhiy and Remi objected, because it might not be obvious that the length
> of the path would be the length of string.
>
> What do you think? Can I get some +1s and -1s for this change?
>
>
> Thanks,
> Ram.
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/2CD4QYG3TDRCMNHRWZK3N2ZLHYLGESFK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LWHVZZQTDJQBF57PJCSCJC3VAZKFE3GY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhance list.index with a comparison function

[David Mertz]

On Sat, May 23, 2020, 10:54 AM Rob Cliffe via Python-ideas

> index_of(needle, haystack, key=func)
>
> Sounds like a list comprehension: [ needle for needle in haystack if
> func(needle) ]
>

The times it doesn't sound like a list comprehension is when you have a
million items in the list, 100k of which match the predicate, but you only
care about the first one... Or even just the first ten.

Still, generator comprehension are great. And next() is an excellent
function.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2ZQR3HMRLIPSOAAK7GJQBJX6MQLUEIKD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhance list.index with a comparison function

[David Mertz]

On Sat, May 23, 2020, 12:26 AM Steven D'Aprano

> Obviously not all such key functions are that simple and you may need to
> write a helper function, but the same applies to filter.
>

I like the key function much better than the predicate. In large part
that's because as soon as you say predicate, I think filter. Particularly
if I care about laziness in not looking through extra items.

If you wanted everything matching a predicate, using a comprehension just
seems more obvious. It could be lazy with a generator comprehension, not
them you need next() to get the first.

Key= has the obvious parallel with sorted() and friends. Even then, a
function similar to sorted() feels more general than a method on lists only.

I.e.

index_of(needle, haystack, key=func)

I'm not sure where that would live exactly, maybe itertools. I think you
were on my side in believing an import from standard library is not a huge
burden.

I think if such a function existed, I'd want another argument to match n !=
1 results. But then it gets ugly because one index is an integer, and
multiple indices are some sort of collection. So maybe one is best...
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W24RTDL4VHKNDMLVMRO6CB3V2AT544XP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhance list.index with a comparison function

[David Mertz]

On Fri, May 22, 2020 at 8:59 PM Steven D'Aprano  wrote:

> > Why not just this (by object, not by its index, but that seems simpler):
> > >>> do_something(next(filter(pred, somelist)))
>
> Sure, that works too. But have you ever written it for real? I haven't.
> And having seen it, I'll probably forget all about it within an hour.
>

Yes, I've written that, or something very similar for real.  More than once
even.  But I do think in functional terms fairly easily.

However, even if you don't do it that way, changing the signature on method
on `list` specifically feels really clunky.  What if I want to do something
to the first item in a tuple that meets a condition?  Or the first from a
deque. Or from a set.  Or even the first matching key from a dictionary. Or
the first match from some third-party collection?

Even if next(filter(...)) isn't how you think, an imperatively written
function can be much more generic. For example (untested):

def process_first(it, pred=lambda _: True, func=lambda _: _, sentinel=None):
for item in it:
if pred(item):
return func(item)
return sentinel

Just stick that somewhere, and it does everything you want and more.  I use
a sentinel rather than raise an exception if nothing matches.  That feels
much more natural to me, but you can change that if such makes more sense
to you.  Likewise, if you want to return the "index", it's a simple enough
change (but what is the index of an unsized iterable?  I mean, of course
you can enumerate() or i+=1... but it might not really be an index.



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FTKB5JBIJZAWGXIR4MSKVQOTSEPQOZE5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhance list.index with a comparison function

[David Mertz]

>
> After I answered that question, it dawned on me that I have probably
>
written something like that loop, or variations of it, a thousand times:
>
> for obj in somelist:
> if comparison(obj, needle):
> do_something(obj)
> break
>

Why not just this (by object, not by its index, but that seems simpler):

>>> do_something(next(filter(pred, somelist)))
Something about 55
>>> somelist
[3, 4, 29, 23, 46, 55, 90, 81]
>>> pred


My style works on general iterables, including infinite ones (that
hopefully eventually have something fulfilling pred()).

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P6SZC2PBIEDBHGCAI46YFDD6T4NIA2ZZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: type hints : I'd like to suggest allowing unicode → as an alternative to ->

[David Mertz]

As is... my editor looks like this.  I don't type all those special things,
except once in a configuration file.  But the "prettification" is handled
transparently when I type some ASCII sequences.
[image: Python-arrow.png]



> that’s nice ! it’s a real shame though, and a bit of a waste honestly,
> that everybody needs to cook their own brew of an editor to get there
> and primarily all I’m trying to say is that, one day, this will be a legal
> piece of code, so that we can share and enjoy exactly like this
> instead of in poor work-it-out-yourself ascii
>

The first screenshot (of completely pointless code I just typed to use a
few symbols or sequences) used vim conceal plugin; also my particular
configuration of what to substitute visually on screen.  That approach lets
entire patterns get replaced by... whatever you want.  E.g. I replace
`set()` with the empty-set symbol, but no one else is required to by the
plugin if they don't want to (or the script Z/R for int/float, likewise).

For comparison, I changed my system text editor to use JetBrains Mono
Medium.  The editor is called "Text Editor" on my system, but it's a
rebranded gEdit, which is a pretty reasonable code editor.  Someone else in
the thread pointed to that font which uses ligatures to make composed
characters.  So it's not a substitution but simply another way of drawing
some letter pairs.  The result is in the attached screenshot.

This approach is somewhat less flexible in that it gets exactly those
ligatures that JetBrains decides you want.  In concept, there is no reason
this couldn't create a 'float' ligature that renders as a script R, for
example.  But that is not a combo JetBrains decided to use.

The main point of this, is that the code is still just plain ASCII, it just
happens to look "fancier."  It requires no plugins, and it does not require
a JetBrains IDE or editor.  I haven't tried every editor, but I believe
that most or all that use GUI fonts will work fine.  Not sure about
ligatures and Linux terminal, someone can try it.



-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/25AQCLEWJXK6IQ23KSIKQW3E5IUOGP43/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: type hints : I'd like to suggest allowing unicode → as an alternative to ->

[David Mertz]

On Thu, May 21, 2020, 5:51 PM Thierry Parmentelat <
thierry.parmente...@inria.fr> wrote:

> You are SERIOUSLY suggesting that typing 'Ctrl-Shift-U 2 1 9 2 ' is
> easier for me than typing '->' as I do now!?
>
> that’s not how I’d do it; and I dont think I said or suggested anything to
> that effect
> this thread seems to be obsessing on entering with the keyboard; that is
> not my point at all
>

I don't know how'd you'd do it on you particular OS, certain, operating
system, editor, etc. But I very much doubt that's a key on your keyboard,
so you'd have to do something akin to this. Maybe you have an old APL
keyboard, but most people don't.

The reason everyone is "obsessing" about how someone would actually make
your symbol appear is that it won't happen magically by thinking it. So you
are proposing and huge hit to ergonomics for everyone for no discernable
benefit.

that’s nice ! it’s a real shame though, and a bit of a waste honestly, that
> everybody needs to cook their own brew of an editor to get there
>

I'm more than happy to share my vim conceal config. There plugin comes with
something similar, but I've customized it to my liking. But other people
would probably prefer something different... Which they can have.

and primarily all I’m trying to say is that, one day, this will be a legal
> piece of code,
>

I very much hope that will never happen. I'm 99% sure my hope will be
realized. I will never be able to use a keyboard with 100k keys on it...
Admittedly, I don't really understand Chinese character entry methods.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YYL4YW4A3CJ52DDCOQGZGCL4JTYTAINJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: type hints : I'd like to suggest allowing unicode → as an alternative to ->

[David Mertz]

You are SERIOUSLY suggesting that typing 'Ctrl-Shift-U 2 1 9 2 ' is
easier for me than typing '->' as I do now!?

And it remains easier if I use a different computer where I have to figure
out or remember some different way of getting the Unicode code point into
the editor?

The goal of this being to wind up with source code that will render gray
boxes for unknown character if I don't have the right font or the right
display context?

Even if *I* don't start using that special character (or others), I wind up
having to edit and view code that other people have written, which means I
need to deal with a tool stack to handle things that are hard to enter on
most keyboards.

As is... my editor looks like this.  I don't type all those special things,
except once in a configuration file.  But the "prettification" is handled
transparently when I type some ASCII sequences.




-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WQVN7JN6VID73UOAFOFROUTYJ26PWLRG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: str.strip: argument maxstrip

[David Mertz]

On Tue, May 19, 2020 at 8:43 PM Cameron Simpson  wrote:

> >salt = salt.rstrip("=", maxstrip=2)
> >assert not salt.endswith("=")
>
> Reiterating the Python 3.9 suggestion, what about:
>
>   salt2 = salt.cutsuffix(('==', '='))
>

You'd have to call cutsuffix twice.  Valid base64 might end in one, two, or
zero '=', but the result wants to have none.  Actually, no... even two
calls won't always do the right thing.  I'm happy to have the new method
.cutsuffix() , but I don't think it addresses this case.

I was going to suggest an example about dynamically built path components
that may or may not end in '/'.  However:

A. I couldn't find an example in my own code easily, even though I know
I've fiddled with that
B. Someone would scold me for playing with strings rather than using
Pathlib (they'd be right, I realize... but I am sinful in my quick-script
ways).

I'll let someone else, like the OP, advance the case more.  I kinda like
it, but it's not a big deal for me.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/365KHHD2XJHCQVT7HU6SRBED5GNXIZDP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: str.strip: argument maxstrip

[David Mertz]

On Tue, May 19, 2020 at 3:50 PM Alex Hall  wrote:

> Anyway, you could write:
> assert not salt.endswith("=" * 3)  # at most 2 ='s allowed
> salt = salt.rstrip("=")
>

Yes, I *could* write that.  It feels a bit more  cryptic than the version I
mentioned.  But it's fine.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LPRFI4KS7FFK2MOJV7LZUCGWTKSH7TSQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: str.strip: argument maxstrip

[David Mertz]

>
> I may be misunderstanding, but it sounds like = is not acceptable in the
> final result, so it's not enough to remove only 2 of 4 ='s. You want to
> make sure nothing messed up your string. So if the code existed, what you'd
> want is:


> ```
> assert salt.count("=") <= 2
> salt = salt.rstrip("=", "")
> assert "=" not in salt
> ```
>

I think the code I'd want, if the new parameter existed, would be:

salt = salt.rstrip("=", maxstrip=2)
assert not salt.endswith("=")

That feels *slightly* better than your version.  This version would allow
there to be '=' in the middle of the string (albeit, such would not be
base64, but some other kind of thing).  Obviously, I've managed to program
Python without this switch for 20+ years.  But I think I'd use it
occasionally.

Currently, I'd probably program my intention like this.  Let's assume this
is something where the '=' is allowed in the middle.

if salt.endswith("=="):
salt = salt[-2:]
elif salt.endswith("="):
salt = salt[-1:]
assert not salt.endswith("=")

The version you suggested, as mentioned, would not handle a format where
'=' was permitted in the middle.

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7PAEOO7CHS3NM2GUEDBXGLJL4L3SSJ2J/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: str.strip: argument maxstrip

[David Mertz]

I did a couple greps of my git/ directory to see if I found examples.
Given that there are a couple ways one might achieve the effect now, I
don't necessarily find everything.  But here's something in software I did
not write myself.

This is from ./JohnTheRipper/run/sspr2john.py.  I found another example in
a different file, but looking at it I'm pretty sure it is actually a
potential bug (it has a comment similar to "Is this safe?" which I won't
bother showing.

elif fmt == "PBKDF2_SHA256":
h = base64.b64encode(base64.b64decode(text)[:32])
# a terrible hack follows, use "adapted base64" alphabet (using
. instead of + and with no padding)
h = h.rstrip("=").replace("+", ".")
salt = base64.b64encode(salt)
salt = salt.rstrip("=").replace("+", ".")

We actually know that base64 code should only produce at most 2 '='s as
padding.  In this instance, the encoding comes immediately before the
stripping.  However, perhaps some code would pass the encoded string and
you wouldn't be as confident locally that extra '='s hadn't snuck in.

If it existed, I think these lines would be good candidates for 'maxstrip'.

On Tue, May 19, 2020 at 2:07 PM Henk-Jaap Wagenaar <
wagenaarhenkj...@gmail.com> wrote:

> David (or somebody else) could you give us some, as real as possible,
> examples? This will strengthen the case for it!
>
> I am confident they exist and are pretty plentiful but I myself am coming
> up blank thinking about it for a few minutes and documenting them would be
> good for discussion.
>

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ULBSV3U2SDUPXY7NBA6XO4PLE3N4UWUS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: str.strip: argument maxstrip

[David Mertz]

I think this would be useful, and doesn't break any backward
compatibility.  I would have use for it from time to time myself.

On Tue, May 19, 2020 at 9:01 AM computermaster360 . <
computermaster...@gmail.com> wrote:

> I often find myself in a need of stripping only a specified amount of
> characters from a string (most commonly a single character). I have been
> implementing this in an ad-hoc manner, which is quite inelegant:
>
> def rstrip1(txt, chars):
>   if txt is None:
> return None
>   elif any(txt.endswith(c) for c in chars):
> return txt[:-1]
>   else:
> return txt
>
> I would appreciate if str.split, str.lstrip, str.rsplit functions had a
> `maxstrip` argument, similar to the `maxsplit` argument of `str.split`,
> which would specify the maximum count of characters to be removed from the
> string. In case of `str.split` this maximum would apply to each side of the
> string separately.
>
> Am I the only one who is surprised such functionality is not implemented
> already??
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/NSUFM4YA6E65ASDPJZPSJWBS2XEDRDFU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZRR3MEKM336G3P3744V6JLFFNPWW7A7N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: PEP 618: Add Optional Length-Checking To zip

[David Mertz]

On Sun, May 17, 2020 at 2:09 PM Nathan Schneider  wrote:

> If you want a better metaphor: Some door handles include locks, others do
>> not.  "Strict" ones have locks.  So yes, it's possible to leave the lock in
>> the unlocked position, and then it functions pretty much the same as one
>> without a lock.  But likewise, it's possible to leave the door in the
>> locked position when you don't have the key on you, and you face a
>> significant inconvenience that serves no purpose.
>>
>> For some of us, strictness is a property that users often want when they
> use zip(), whether they are properly enforcing it or not—so giving them a
> really easy way to opt into it would help avoid bugs. (Personally, I cannot
> remember the last time I actually wanted non-strict zip.)
> I think David is saying that he more often wants non-strict zip, and it
> would be tempting and dangerous to make strict-zip too easy to use for
> those who aren't thinking carefully about their code, so it would be better
> to bury strict-zip in itertools for the power users who really know they
> need it. (Is that a fair summary?)
>

The API matter is really orthogonal to this.  My point here is that Nathan
and some other discussants are operating under the assumption that:
"Everyone really wants strict-zip but they just haven't had a way to
express it conveniently.  They all assume their iterables are the same
length already, so this just adds a check."

I disagree strongly with that assumption.  I think that the actual majority
of my uses of zip are non-strict.  Admittedly, I have not scanned my code
to count that; for that matter, most of the code I have written is no
longer accessible to me, being written for companies I no longer work for
(and not open source).  But whatever the actual percentages might be, I
COMMONLY want a non-strict zip by actual specific intention, not because
I've made a wrong assumption about the nature of the iteratables I use.  Of
course, I also commonly use zip with the implicit assumption that my
iterables are the same length... I have most certainly written many lines
where I would appropriately choose strict-zip if it existed (whichever API).

To me, itertools is not some hidden vault only accessible after performing
Herculean labors.  I believe boolean mode switches are usually a bad design
for Python.  Not always, there are exceptions like open().  And I think
Guido made the good point that one of the things that makes mode switches
bad is when they change the return type, which the `strict=True` API would
not do (but it would create a new kind of exception to worry about).

In fact, itertools is pretty much the only module where I occasionally
write `from itertools import *`.  There are many good things in that module
that are general purpose.  But namespaces, after all, are a honkin' good
idea.  I think if `zip()` were proposed today as a brand new function that
hadn't existed, I would advocate strongly for putting it inside itertools.
Probably `map()` and `filter()` similarly.  So I don't want zip_strict() to
join built-ins, but it's not because I think it is a niche case, but rather
because I think we already have more in built-ins than we should, and some
of it could very reasonably live in a more descriptive namespace.  I would
certainly not mind if we added `zip_shortest()` as a synonym for the
current zip to itertools.


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VP72ECM5JCHVYPRA55M6EL3YLJJQQRJL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: PEP 618: Add Optional Length-Checking To zip

[David Mertz]

On Sun, May 17, 2020 at 12:22 PM Nathan Schneider 
wrote:

> Let me attempt a metaphor, which won't be perfect but may help:
>
> The safety one gets from strictness is a bit like driving a car wearing a
> seat belt. It is not fundamentally different from driving a car without a
> seat belt, and in most cases you hope the seat belt will not come into
> play. But it is a precaution that is worth taking in *most* circumstances
> (not all, e.g. for infants a standard seat belt won't work).
>

That's a really terrible analogy. :-(

I never drive without a seat belt.  And a never want the seat belt to
actually matter, of course.  Everyone who want a zip_strict behavior
(including me) wants to be able either to catch the exception explicitly or
to have the program fail-fast/fail-hard because of it.

In contrast, as I've said, more than half of the times that *I* use zip()
it would be BROKEN by using zip_strict() instead (or zip(..., strict=True),
or whichever spelling).  Raising an exception for something I want to
succeed, and I want to work exactly as it does (e.g. leave some iterators
unconsumed) is not a "harmless safety precaution".

If you want a better metaphor: Some door handles include locks, others do
not.  "Strict" ones have locks.  So yes, it's possible to leave the lock in
the unlocked position, and then it functions pretty much the same as one
without a lock.  But likewise, it's possible to leave the door in the
locked position when you don't have the key on you, and you face a
significant inconvenience that serves no purpose.

I have some doors with locks, and some other doors without locks.  I have
both for a good reason, but the reasons are different, and depend on
various things like whether a particular room is private or contains
valuables.  In truth though, I don't lock my outside doors because I live
in a community of "consenting adults" (occasionally I do lock the bathroom
door for privacy, for a short while... no-locks is definitely strongly my
default mode, as is no-strict when I zip).


> --
>
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FQHSMQO6URCXGBPNIMD4NW2EW33LNR7R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Documenting iterators vs. iterables [was: Adding slice Iterator ...]

[David Mertz]

On Thu, May 14, 2020, 11:20 PM Stephen J. Turnbull

>  > I can teach a child why a glass will break permanently when you hit
>  > it while a lake won’t by using the words “solid” and “liquid”.
>
> Terrible example, since a glass is just a geologically slow liquid. ;-)
>

It isn't though. I used to believe the old urban legend about glass being a
(very slow) liquid too. It makes for good "impress your friends" chatter at
a certain point. But it's not true, glass is an amorphous solid, and old
windows are thicker at the bottom because they hung them that way.

https://www.scientificamerican.com/article/fact-fiction-glass-liquid/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/J3XVCONCHZRC4NPN567J6WG26SSXCMYY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve handling of Unicode quotes and hyphens

[David Mertz]

A third-party module on PyPI for "fix-the-horrible-things-Outlook-does"
could be useful.  There is no way the standard library can or should keep
up with the newest mangling techniques mail handlers employ in this week's
version.

I don't understand what you mean by the current interpreter not telling you
which character is bad.  It puts a pointer right under the problem
character in the error message.  Nothing is being hidden, even if the
code-manglers can do pretty insidious things... different in every version,
for every sender, and for every recipient.

On Mon, May 11, 2020 at 3:24 AM Steve Barnes  wrote:

>
>
> -Original Message-
> From: Steven D'Aprano 
> Sent: 11 May 2020 06:02
> To: python-ideas@python.org
> Subject: [Python-ideas] Re: Improve handling of Unicode quotes and hyphens
>
> On Mon, May 11, 2020 at 04:28:38AM +, Steve Barnes wrote:
>
> > So we currently have a situation where not only does whether code
> > works or not depends on who typed it, in what environment, with what
> > settings but also on the same factors for who received it
>
> You say "currently", but that has always been the case, and the further
> back you go, the worse it was.
> [Steve Barnes] True - apart from the older email clients being less
> "helpful".
>
> > - so I could
> > use Outlook or Word to send a code fragment to 100 people and 5 say
> > that is great it works and the other 95 end up thinking that there is
> > something wrong with their installation.
>
> While the actual problem is that you are sending code via a non-WYSIWYG
> medium that may modify what you type.
>
> If you want to send code via email, the most reliable method is to
> attach it as a .py file.
>
> [Steve Barnes] My employer mandates the communication method(s) (as do
> many others). They also block unsafe "attachments" such as .py & .bat files
> (even when zipped). In any case I am often trying to tell the end user how
> to invoke existing code that they already have from the command line,
> (especially when they have stored files in folders with spaces, ampersands,
> hyphens, etc., in the path).
>
> > Personally I don't think this
> > fits in with the pythonic way of thinking!
>
> The Zen says differently:
>
> "Errors should never pass silently."
>
> It's not Python's responsibility to make up for lossy transmission
> methods such as mail servers which strip the high bit off every octet,
> or faulty hardware that randomly corrupts one byte in a thousand.
>
> And nor is it the interpreter's responsibility to guess what the user
> intended to type, or *should have* intended to type, in the face of
> editors which substitute characters.
>
> [Steve Barnes] What I would prefer is an error message that actually tells
> the end user which specific character(s) they have got wrong rather than
> something is wrong after all "Explicit is better than implicit" would
> suggest that "illegal character x (Unicode u)" would be more pythonic than
> the current error message.
> --
> Steven
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3AO3MPWD7YRCQSHU5UIUSDENCO526PTD/
> Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/UC6YNOSVVIDCQDMB5T73RIJALPUVPDLZ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WIKRK47WPMHVHYDKJSCI7MOL2MDVAYSR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve handling of Unicode quotes and hyphens

[David Mertz]

On Sun, May 10, 2020 at 9:05 PM Steven D'Aprano  wrote:

> On Sun, May 10, 2020 at 01:17:17PM -0700, Andrew Barnert via Python-ideas
> wrote:
> > (By the way, the reason I used -f rather than —fix is that I can’t
> > figure out how to get the iPhone Mail.app to not replace double
> > hyphens with an em-dash, or even how to fix it when it does.
>


> I don't know anything about Mail.app, but in LibreOffice you can Ctrl-Z
> to underdo autocorrect without undoing the characters you typed.
>

I don't use iPhone, but I'm pretty sure it doesn't have a Ctrl-Z key on it
:-)

-- 
The dead increasingly dominate and strangle both the living and the
not-yet born.  Vampiric capital and undead corporate persons abuse
the lives and control the thoughts of homo faber. Ideas, once born,
become abortifacients against new conceptions.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/S7GMB4M3API5TO6N2THM3RZPTZVULM4C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve handling of Unicode quotes and hyphens

[David Mertz]

I think it is a bad idea to  change the design of Python because some
college professor teaching intro to programming very poorly might require
inappropriate tooling.

On Sun, May 10, 2020, 8:39 PM Jonathan Goble  wrote:

> On Sun, May 10, 2020 at 8:08 PM David Mertz  wrote:
>
>> Why would you use Word? LaTeX exists, and will export to PDF. Heck, the
>> PDF export from Jupyter is quite good (by way of LaTeX).
>>
>> On Sun, May 10, 2020, 7:40 PM Jonathan Goble <
>> Several of the questions required us to write Bash scripts or Python
>> functions, and we were required to write all of that code, along with all
>> of our other exam answers, several of which were essay questions, in
>> Microsoft Word and then export the whole mess as a single PDF file to be
>> uploaded. We were not allowed to submit multiple files, or zip files, or
>> anything at all except one single PDF file containing all of our answers
>> and code.
>>
>
> This exam format was dropped on us with short notice before the midterm
> and we didn't have time to set up and learn brand-new tooling. Also, this
> was a freshman-level course, and I highly doubt more than a tiny fraction
> of the class had even heard of LaTeX, much less knew how to use it (I've
> definitely heard of it, but haven't the slightest clue how to use it
> myself). And as the course was many students' first introduction to Python
> (we only covered the bare basics of the language), Jupyter was again
> something most students in the class had probably never heard of.
>
> The course's primary focus was on introducing students to the usage of the
> command line interface and to the usage of Linux in general, and Python was
> a one-week side note to that. Most of the course centered on Bash and on
> general OS concepts (processes, virtual memory, etc.).
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VVL3CVT76PCGAP27YWCZTHFAW5VYM6SF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve handling of Unicode quotes and hyphens

[David Mertz]

Why would you use Word? LaTeX exists, and will export to PDF. Heck, the PDF
export from Jupyter is quite good (by way of LaTeX).

On Sun, May 10, 2020, 7:40 PM Jonathan Goble <
Several of the questions required us to write Bash scripts or Python
functions, and we were required to write all of that code, along with all
of our other exam answers, several of which were essay questions, in
Microsoft Word and then export the whole mess as a single PDF file to be
uploaded. We were not allowed to submit multiple files, or zip files, or
anything at all except one single PDF file containing all of our answers
and code.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7AHWDMOABTI3S7NWTA2T57RKGNACK2EC/
Code of Conduct: http://python.org/psf/codeofconduct/