[Python-ideas] Re: @lazy decorator an alternative to functools.partial ?

[Lucas Wiman]

Seems fine, but a pypi library seems better than the standard library.
Separately, I think this is usually called "currying", and there are
already libraries which implement this functionality, eg toolz:
https://toolz.readthedocs.io/en/latest/curry.html

Best wishes,
Lucas Wiman
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CDDCE3I5CRVLITN5ASC2LRNNBFVYCK7C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto dedent -c arguments.

[Lucas Wiman]

On Tue, Apr 4, 2023 at 7:19 AM Jonathan Crall  wrote:

> Would there be any downside to the Python CLI automatically dedenting the
> input string given to -c? I can't think of any case off the top of my head
> where it would make a previously valid program invalid. Unless I'm missing
> something this would strictly make previously invalid strings valid.
>
> Thoughts?
>

Very strong +1 to this. That would be useful and it doesn't seem like
there's a downside. I often make bash functions that pipe files or database
queries to Python for post-processing. I also sometimes resort to Ruby
because it's easy to write one-liners in Ruby and annoying to write
one-liners in python/bash.

I suppose there's some ambiguity in the contents of multi-line
"""strings""". Should indentation be stripped at all in that case? E.g.
python -c "
'''
some text

"

But it seems simpler and easier to understand/document if you pre-process
the input like using an algorithm like this:

* If the first nonempty line has indentation, and all subsequent lines
either start with the same indentation characters or are empty, then remove
that prefix from those lines.

I think that handles cases where editors strip trailing spaces or the first
line is blank. So e.g.:
python -c "
some_code_here()
"
Then python receives  something like "\nsome_code_here\n"

python -c "
some_code here()

if some_some_other_code():
still_more_code()
"
Then python receives something like "\nsome_code_here\n\nif ..."

This wouldn't handle cases where indentation is mixed and there is a first
line, e.g.:
python -c "first_thing()
if second_thing():
third_thing()
"
That seems simple enough to avoid, and raising a syntax error is reasonable
in that case.

Best wishes,
Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YQNYNGO4LI2537HU6SET4LIIINNDHHUO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Restricting access to sensitive APIs with a permission model like Deno

[Lucas Wiman]

On Sun, Feb 26, 2023 at 7:31 AM python--- via Python-ideas <
python-ideas@python.org> wrote:

> Supply chain attacks are becoming a pressing concern in software
> development due to the large number of dependencies and multiple attack
> vectors. Using third party modules (libraries, packages etc)  is always a
> risk but the true potential of these attacks is now being weaponized. One
> way to deal with the risk is by limiting access to sensitive APIs like
> filesystem, shell, network and ffi so that packages which aren't explicitly
> granted permissions cannot use them, reducing their ability to do damage.
>

I agree with the other commenters that creating a sandbox within a
particular python process is going to be extremely difficult or impossible.
But I commend you in trying to do work in this area.

One way this could be implemented is by providing some primitives for
sandboxing subprocesses. E.g. in the requirements file add an optional
section for sandbox directives that will cause the import to be executed in
a subprocess with a restricted set of OS-level permissions (e.g. no writing
to the filesystem other than a particular socket for communicating with the
parent process).

An incremental and independently useful place to start on that might be
adding sandboxing primitives to the subprocess and/or multiprocessing
module. I have tried to do this manually on a linux web service using calls
to Imagemagick and didn't see a way to do it aside from calling a `docker
run` command. Is there a cross-platform way of doing this? I did a bit of
googling and found some "experimental" libraries for the purpose, though
nothing that wanted to call itself production-ready. (Gaol
https://github.com/servo/gaol and Boxfort https://github.com/Snaipe/BoxFort)
Presumably web browsers like Chrome have some prior art as well.

Best wishes,
Lucas Wiman
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JPZFA2TSA6ULNS6QTK4EZOM4XDCK4UOE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Call functools.update_wrapper on Mock(wraps=func)

[Lucas Wiman]

For code that uses annotations / signature, wrapping a callable in a Mock
object does not work correctly:

>>> import typing, inspect, functools
>>> from unittest.mock import Mock
>>>
>>> def incr(i: int) -> int:
... return i+1
...
>>> wrapped = Mock(wraps=incr)
>>> inspect.signature(wrapped)

>>> typing.get_type_hints(wrapped)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/local/lib/python3.11/typing.py", line 2330, in get_type_hints
raise TypeError('{!r} is not a module, class, method, '
TypeError:  is not a module, class, method, or
function.
>>> functools.update_wrapper(wrapped, incr)

>>> inspect.signature(wrapped)
 int>
>>> typing.get_type_hints(wrapped)
{'i': , 'return': }

It's relatively simple to call update_wrapper on your own, but it seems
like it would be better if the mock "just worked" for code that tries to
use signature and annotations at runtime. I think all that's required would
be adding `if wraps is not None: functools.update_wrapper(self, wraps)`
after the line here:
https://github.com/python/cpython/blob/f4fcfdf8c593611f98b9358cc0c5604c15306465/Lib/unittest/mock.py#L1107

Does anyone see any problems with implementing this?

Thanks & best wishes,
Lucas Wiman
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/25EGYHN3OYUFDJYZLEUZPLZUTMYWW5R5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

[Lucas Wiman]

On Tue, Dec 20, 2022 at 5:38 PM Christopher Barker 
wrote:

> But collections.UserString does exist -- so if you want to subclass, and
> performance isn't critical, then use that. Steven A pointed out that
> UserStrings are not instances of str though. I think THAT is a bug. And
> it's probably that way because with the magic of duck typing, no one cared
> -- but with all the static type hinting going on now, that is a bigger
> liability than it used to be. Also basue when it was written, you couldn't
> subclass str.
>
> Though I will note that run-time type checking of string is relatively
> common compared to other types, due to the whole a-str-is-a-sequence-of-str
> issue making the distinction between a sequence of strings and a string
> itself is sometimes needed. And str is rarely duck typed.
>

Note that UserString does break some built-in functionality, like you can't
apply regular expressions to a UserString:
>>> class FooString(UserString):
... pass
...
>>> re.compile(r"asdf").match(FooString("asdf"))
Traceback (most recent call last):
  File "", line 1, in 
TypeError: expected string or bytes-like object, got 'FooString'


There is more discussion in this thread (
https://stackoverflow.com/questions/59756050/python3-when-userstring-does-not-behave-as-a-string),
including a link to a very old bug (https://bugs.python.org/issue232493).
There is a related issue with json.dump etc, though it can be worked around
since there is a python-only json implementation.

I have run into this in practice at a previous job, with a runtime "taint"
tracker for logging access to certain database fields in a Django
application. Many views would select all fields from a table, then not
actually use the fields I needed to log access to, which generated false
positives. (Obviously the "correct" design is to only select data that is
relevant for the given code, but I was instrumenting a legacy codebase with
updated compliance requirements.) So I think there is some legitimate use
for this, though object proxies can be made to work around most of the
issues.

- Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DZQNWKTSUTKRMA4WFXVERW46AMPYDEAX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP Idea: native f-string support as a match pattern

[Lucas Wiman]

On Sat, Aug 13, 2022 at 10:23 AM Tushar Sadhwani <
tushar.sadhwani...@gmail.com> wrote:

> Stephen J. Turnbull wrote:
> > I think this is a pretty unconvincing example.  While people seem to
> > love to hate on regular expressions, it's hard to see how that beats
> > def unquote(string: str) -> str:
> > m = re.match(r"^(?:"(.*)"|'(.*)'|(?Pvalue3))$", string)
>
> RegEx feels overkill for this. Certainly takes longer to read, understand
> and test.
>

f-strings are inherently ambiguous. You've chosen cases with unambiguous
and mutually exclusive prefixes, but that won't generally be the case.
You'd need implicit rules about greediness, which could easily lead to some
buggy code looking correct.

For what it's worth, Raymond Hettinger has some code examples that make
using regexes in match statements much easier:  see
https://www.dropbox.com/s/w1bs8ckekki9ype/PyITPatternMatchingTalk.pdf?dl=0
and https://twitter.com/i/web/status/1533369943764488192

Best wishes,
Lucas Wiman
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SJGI7GRUWE44HO7JL4GXRDWMYRWNQKG6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Void type

[Lucas Wiman]

On Wed, Jul 27, 2022 at 1:07 PM Chris Angelico  wrote:

> with all the variants of keyword and positional args. Being able to
> say "this function has the same signature as that one, plus it accepts
> consumed_arg" or "same, but without added_arg" would be extremely
> useful, and short of some fairly detailed messing around, not easy at
> the moment.
>

I would suggest checking out python-makefun, which can do many
transformations on the signature of the method:
https://smarie.github.io/python-makefun/ The wrapt library also does some
of this, though the process for adding or removing fields is more
convoluted than with makefun.

It might be interesting to put this in the standard library (e.g. expanding
functools.wraps), though there are third party libraries that do a
reasonable job.


> So rather than proposals for weird magic objects that do weird things
> as function arguments, I'd much rather see proposals to fix the
> reported signature, which would largely solve the problem without
> magic. (Or technically, without *more* magic; the only magic needed is
> "if the function has a __wrapped__ attribute, use the signature from
> that", which already exists in many tools.)
>

Note that per PEP 362, tools should also respect the __signature__ magic
attribute (and that should override the signature of .__wrapped__ when both
are present).

Best wishes,
Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RIWRVBOPBI33DIVEGCCGAG2I6YQTZ6VT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Void type

[Lucas Wiman]

Михаил Крупенков:
> Yes, I want that when Void is received in a function parameter it is not
processed:
>
> func(Void) == func() == "default"

Mathew Elman:
> I believe this is a rebirth of a request that has come up many times
before, which is to have something like javascript's `undefined` where it
means "use the default value" if passed to a function that has a default
value or "value not provided" (slightly different to "None").

I think this can be achieved with a decorator.
>>> import inspect, functools
>>>
>>> class Void:
... pass
...
>>> def handle_defaults(f, undefined=Void):
... sig = inspect.signature(f)
... @functools.wraps(f)
... def defaults_handled(*args, **kwargs):
... bound = sig.bind(*args, **kwargs)
... bound.apply_defaults()
... for name, value in list(bound.arguments.items()):
... if value is undefined:
... field = sig.parameters[name]
... if field.default is not inspect._empty:
... bound.arguments[name] = field.default
... return f(*bound.args, **bound.kwargs)
... return defaults_handled
...
>>>
>>> @handle_defaults
... def func(val="default"):
... return val
...
>>> func(Void)
'default'
>>>
>>> def wrapper(val=Void):
... return func(val)
...
>>>
>>> wrapper()
'default'

(In case formatting is screwed up in the above example, see
https://gist.github.com/lucaswiman/995f18763d08847a74864254ac0b)

I would recommend using Ellipsis (...) rather than Void for this purpose so
you have a convenient syntax.

Best wishes,
Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WCLWTRKOAYHYNVMEHBOHLPEX523CSFGC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bare wildcard in de-structuring to ignore remainder and stop iterating (restart)

[Lucas Wiman]

Some background. PEP 3132 (https://peps.python.org/pep-3132/) lists the
following:
>
> Possible changes discussed were:
>
>
>- Only allow a starred expression as the last item in the exprlist.
>This would simplify the unpacking code a bit and allow for the starred
>expression to be assigned an iterator. This behavior was rejected because
>it would be too surprising.
>
> This seems to be a reference to this message:
https://mail.python.org/pipermail/python-3000/2007-May/007299.html

Guido van Rossum said the following (
https://mail.python.org/pipermail/python-3000/2007-May/007378.html):

> The important use case in Python for the proposed semantics is when
> you have a variable-length record, the first few items of which are
> interesting, and the rest of which is less so, but not unimportant.
> (If you wanted to throw the rest away, you'd just write a, b, c =
> x[:3] instead of a, b, c, *d = x.)
>

There was also discussion about retaining the type of the object on the
RHS, e.g.:
c, *rest = "chair"  # c="r", rest="hair"
it = iter(range(10))
x, *rest = it  # c=1, rest is a reference to `it`

That proposal was rejected because the types were too confusing, e.g.:
header, *lines = open("some-file", "r")  # lines is an iterator
header, *lines, footer = open("some-file", "r")  # lines is a list

van Rossum later said (
https://mail.python.org/pipermail/python-3000/2007-May/007391.html):

> From an implementation POV, if you have an unknown object on
> the RHS, you have to try slicing it before you try iterating over it;
> this may cause problems e.g. if the object happens to be a defaultdict
> -- since x[3:] is implemented as x[slice(None, 3, None)], the
> defaultdict will give you its default value. I'd much rather define
> this in terms of iterating over the object until it is exhausted,
> which can be optimized for certain known types like lists and tuples.
>

It seems like these objections don't apply in this case, if we define a
syntax that explicitly says not to assign anything. There is no
inconsistency in the types there. E.g. in the proposal here:

header, *... = open("some-file", "r")
header, *..., footer = open("some-file", "r")

It's clear that to compute what the footer is, you would need to iterate
over the whole file, whereas you don't in the first one.
So historically, the idea here was discussed and rejected, but for a reason
which does not apply in this case.

===

Regarding utility, there are many sort of ugly ways of doing this with
method calls, especially from itertools. I tend to like syntax over methods
for handling basic data types. This is partly because it's more readable:
almost any method which takes more than one positional argument introduces
cognitive load because you have to remember what the order of the arguments
are and what they mean. You can add keyword arguments to improve
readability, but then it's more characters and you have to remember the
name or have it autocompleted. So if there is a simple way to support a use
case with simple built-in syntax, it can improve the utility of the
language.

Like honestly, does anyone remember the arguments to `islice`? I'm fairly
sure I've had to look it up every single time I've ever used it. For
iterator-heavy code, this might be multiple times on the same day. For the
`next(iterator, [default], count=1)` proposal, it's very easy to write
incorrect code that might look correct, e.g. `next(iterator, 3)`. Does 3
refer to the count or the default? If you've written python for years, it's
clear, but less clear to a novice.

There are efficiency arguments too: method calls are expensive, whereas
bytecode calls can be much more optimized. If you're already using
iterators, efficiency is probably relevant:
>>> import dis
>>> from itertools import islice
>>> def first_two_islice(it):
... return tuple(islice(it, 2))
...
>>> def first_two_destructuring(it):
... x, y, *rest = it
... return x, y
...
>>> dis.dis(first_two_islice)
  2   0 LOAD_GLOBAL  0 (tuple)
  2 LOAD_GLOBAL  1 (islice)
  4 LOAD_FAST0 (it)
  6 LOAD_CONST   1 (2)
  8 CALL_FUNCTION2
 10 CALL_FUNCTION1
 12 RETURN_VALUE
>>> dis.dis(first_two_destructuring)
  2   0 LOAD_FAST0 (it)
  2 UNPACK_EX2
  4 STORE_FAST   1 (x)
  6 STORE_FAST   2 (y)
  8 STORE_FAST   3 (rest)

  3  10 LOAD_FAST1 (x)
 12 LOAD_FAST2 (y)
 14 BUILD_TUPLE  2
 16 RETURN_VALUE

The latter requires no expensive CALL_FUNCTION operations, though it does
currently allocate rest pointlessly.

Personally, I think the main use case would be for handling large lists in
a memory efficient and readable manner. Currently using *_ mean

[Python-ideas] Add tz argument to date.today()

[Lucas Wiman]

Since "today" depends on the time zone, it should be an optional argument
to date.today(). The interface should be the same as datetime.now(tz=None),
with date.today() returning the date in the system time zone.

Rationale: It is common for processes to run in different timezones than
the relevant end user. The most common case is probably a server running in
UTC, which needs to do date calculations in timezones elsewhere. As a
contrived example, you might do something like:

tomorrow = date.today() + timedelta(days=1)
return render_template(..., ship_by_date=tomorrow)

But then if fulfillment is operating e.g. in US/Pacific time, the ship by
date suddenly shows two days hence after 5 or 6pm when UTC midnight happens.

In my particular use case, we get some collection of business rules about
when a particular record is considered "stale" or prioritized, and naively
using date.today() can lead to off-by-one errors.

The way to "fix" the code above is to reference "now":

tomorrow = datetime.now(tz=...).date() + timedelta(days=1)
return render_template(..., ship_by_date=tomorrow)

While this is not terrible, it seems like the API between now() and today()
should be consistent: in reality, they need the same set of inputs. Calling
date.today(...) is more explicit and specific than calling
datetime.now(...).date().

Best wishes,
Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OJI3GBPNNGVZL22EAD723TYIFNF6OWAH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bare wildcard in de-structuring to ignore remainder and stop iterating (restart)

[Lucas Wiman]

>
> "Self-explanatory". This is how we got Perl and APL o_O


What I mean is that if you already know the destructuring syntax, then it’s
pretty clear what it means. If you already know existing syntax, / isn’t
suggesting of anything. The only related syntax is for declaring
positional-only arguments, which is very uncommon and I usually have to
look up.

Using arbitrary unrelated symbols because they happen to be available is
how we got Perl.


What do we need the star for?
>
> x, *... = items
> x, ... = items
>
> Convince me that adding another meaning for the star symbol is a
> good idea


I don’t see it as another meaning for the star symbol. It’s an extension to
an existing meaning. The only argument I can give is based on intuition
having written Python for years.

>From a code reader’s perspective, you often guess meaning based on the rest
of the language and the context rather than memorizing the entire
grammar.  Consider
the following:

x, *remainder = items  # 1
x, *_ = items  # 2
x, *… = items  # 3
x, … = items  # 4

(1) and (2) are valid syntax. They are how one would currently write the
concept under discussion, other than the optimizations of not allocating a
list or advancing the iterator.

(3) has a pretty clear analogy to (2). If you basically know Python 3.10
syntax and the interpreter is telling you that (3) is valid Python code,
there are two thing this could possibly mean:

(a) [wrong] … is an identifier that can be assigned to, similar to _. Its
value will be the rest of the list/iterator, but given what … means in
English and other Python code, you’re being told by the code author that
you don’t care. The main difference with (2) is that _ is used in
internationalization, which makes assigning to it a bad idea in some code.
(b) [correct] You recall that … is actually a singleton object that cannot
be assigned to. This suggests the intended meaning in this thread. Since …
cannot be assigned to, the rest of the list/iterator is probably not being
used for anything.

What’s notable about this is that the wrong interpretation is still mostly
correct, eg if this syntax is used on a list. With (4), you could have the
same misunderstanding about whether … is a special-case identifier name,
but then your interpretation becomes completely wrong as opposed to
slightly wrong.

In other words, * is telling you that this is variable-length object
destructuring syntax, and it has a similar meaning to (1) and (2).

So I think your question is backwards. Omitting the * means you are
genuinely introducing a new syntax for something that is extremely similar
to existing syntax, as opposed to slightly altering existing syntax for a
slightly different meaning.

Best wishes,
Lucas
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GJWREIR3N5SU7F6Q2BHUCDZWH3LSHBQ3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bare wildcard in de-structuring to ignore remainder and stop iterating (restart)

[Lucas Wiman]

Using either * or / could lead to some odd inconsistencies where a missing
space is very consequential, eg:
x, / = foo  # fine
x, /= foo  # syntax error?
x / = foo  # syntax error
x /= foo  # fine, but totally different from the first example.

That said, the * syntax feels intuitive in a way that / doesn’t. I’d
suggest:
x, *… = foo
This seems unambiguous and fairly self-explanatory.

- Lucas

On Sat, Jun 18, 2022 at 11:23 PM Steven D'Aprano 
wrote:

> On Fri, Jun 17, 2022 at 11:32:09AM -, Steve Jorgensen wrote:
>
> > That leads me to want to change the proposal to say that we give the
> > same meaning to "_" in ordinary destructuring that it has in
> > structural pattern matching, and then, I believe that a final "*_" in
> > the expression on the left would end up with exactly the same meaning
> > that I originally proposed for the bare "*".
> >
> > Although that would be a breaking change, it is already conventional
> > to use "_" as a variable name only when we specifically don't care
> > what it contains following its assignment, so for any code to be
> > affected by the change would be highly unusual.
>
> Not so: it is very common to use `_()` as a function in
> internationalisation.
>
>
> https://stackoverflow.com/questions/3077227/mercurial-python-what-does-the-underscore-function-do
>
> If we are bike-shedding symbols for this feature, I am a bit dubious
> about the asterisk. It already gets used in so many places, and it can
> be confused for `a, b, *x` with the name x lost.
>
> What do people think about
>
> first, second, / = items
>
> where / stands for "don't advance the iterator"?
>
> I like it because it reminds me of the slash in "No Smoking" signs, and
> similar. As in "No (more) iteration".
>
>
> --
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/B2IIGM5TMC7YWVXHQPD6HKT4IMHHSJDP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/S4ZPX4XT6Y5PJYND53I2Q5NWPNTOUUW6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a line_offsets() method to str

[Lucas Wiman]

I'm a little confused by the benchmark. Using re looks pretty competitive
in terms of speed, and should be much more memory efficient.

# https://www.gutenberg.org/cache/epub/100/pg100.txt (5.7mb; ~170K lines)
with open('/tmp/shakespeare.txt', 'r') as f:
text = f.read()
import re
from itertools import *
line_re = re.compile(r"\n")

Then when I run it:
In [25]: %timeit _ = list(accumulate(chain([0], map(len,
text.splitlines(True)
30.4 ms ± 705 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [26]: %timeit _ = [m.start() for m in line_re.finditer(text)]
29 ms ± 457 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

This is on 3.10.3 on an Intel 2.3gz i9 Macbook. (Note that the regex is
off-by-one from the splitlines implementation.)

What benchmark shows the regex to be significantly slower?

That said, str.indexes(char) sounds like a reasonable addition.

Best wishes,
Lucas Wiman

On Fri, Jun 17, 2022 at 1:12 PM Jonathan Slenders 
wrote:

> Hi everyone,
>
> Today was the 3rd time I came across a situation where it was needed to
> retrieve all the positions of the line endings (or beginnings) in a very
> long python string as efficiently as possible. First time, it was needed in
> prompt_toolkit, where I spent a crazy amount of time looking for the most
> performant solution. Second time was in a commercial project where
> performance was very critical too. Third time is for the Rich/Textual
> project from Will McGugan. (See:
> https://twitter.com/willmcgugan/status/1537782771137011715 )
>
> The problem is that the `str` type doesn't expose any API to efficiently
> find all \n positions. Every Python implementation is either calling
> `.index()` in a loop and collecting the results or running a regex over the
> string and collecting all positions.
>
> For long strings, depending on the implementation, this results in a lot
> of overhead due to either:
> - calling Python functions (or any other Python instruction) for every \n
> character in the input. The amount of executed Python instructions is O(n)
> here.
> - Copying string data into new strings.
>
> The fastest solution I've been using for some time, does this (simplified):
> `accumulate(chain([0], map(len, text.splitlines(True`. The
> performance is great here, because the amount of Python instructions is
> O(1). Everything is chained in C-code thanks to itertools. Because of that,
> it can outperform the regex solution with a factor of ~2.5. (Regex isn't
> slow, but iterating over the results is.)
>
> The bad things about this solution is however:
> - Very cumbersome syntax.
> - We call `splitlines()` which internally allocates a huge amount of
> strings, only to use their lengths. That is still much more overhead then a
> simple for-loop in C would be.
>
> Performance matters here, because for these kind of problems, the list of
> integers that gets produced is typically used as an index to quickly find
> character offsets in the original string, depending on which line is
> displayed/processed. The bisect library helps too to quickly convert any
> index position of that string into a line number. The point is, that for
> big inputs, the amount of Python instructions executed is not O(n), but
> O(1). Of course, some of the C code remains O(n).
>
> So, my ask here.
> Would it make sense to add a `line_offsets()` method to `str`?
> Or even `character_offsets(character)` if we want to do that for any
> character?
> Or `indexes(...)/indices(...)` if we would allow substrings of arbitrary
> lengths?
>
> Thanks,
> Jonathan
>
>
>
>
>
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/6WAMKYXOYA3SKL5HIRZP4WARMYYKXI3Q/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JZP5L57KORNOHILH7W3WZMCWRHGKTPQK/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add MutableSet.update?

[Lucas Wiman]

On Fri, Mar 9, 2018 at 6:24 PM, MRAB  wrote:

> On 2018-03-10 01:15, Guido van Rossum wrote:
>
>> Yes, you can use the |= operator instead.
>>
>> |= is not quite the same as .update because it rebinds, so if the name on
> the LHS isn't local it'll raise NameError.
>
> Does that matter?


Not for my use case; that should work great. Thanks Guido!

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add MutableSet.update?

[Lucas Wiman]

I was trying to use `collections.abc.MutableSet` today, and noticed that it
does not define an `update` method. This is a very useful method that is
present on the builtin `set` class, and seems to fit naturally with the
other methods.

Was omitting this method intentional?

Thanks & best wishes,
Lucas Wiman
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add a module itertools.recipes

[Lucas Wiman]

On Fri, Oct 13, 2017 at 11:35 AM, Chris Angelico  wrote:

> On Sat, Oct 14, 2017 at 5:16 AM, Antoine Rozo 
> wrote:
> [...]

> Can we consider making itertools a package and adding a module
> > itertools.recipes that implements all these utilility functions?
>
> Check out more-itertools on PyPI - maybe that's what you want?
>

toolz is another good collection of itertools-related recipes:
http://toolz.readthedocs.io/en/latest/api.html

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Hexadecimal floating literals

[Lucas Wiman]

On Thu, Sep 21, 2017 at 8:23 AM, Victor Stinner 
wrote:

> While I was first in favor of extending the Python syntax, I changed
> my mind. Float constants written in hexadecimal is a (very?) rare use
> case, and there is already float.fromhex() available.
>
> A new syntax is something more to learn when you learn Python. Is it
> worth it? I don't think so. Very few people need to write hexadecimal
> constants in their code.
>

It is inconsistent that you can write hexadecimal integers but not floating
point numbers. Consistency in syntax is *fewer* things to learn, not more.
That said, I agree it's a rare use case, so it probably doesn't matter much
either way.

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] LOAD_NAME/LOAD_GLOBAL should be use getattr()

[Lucas Wiman]

On Wed, Sep 13, 2017 at 11:55 AM, Serhiy Storchaka 
wrote:

> [...] Calling __getattr__() will slow down the access to builtins. And
> there is a recursion problem if module's __getattr__() uses builtins.
>

 The first point is totally valid, but the recursion problem doesn't seem
like a strong argument. There are already lots of recursion problems when
defining custom __getattr__ or __getattribute__ methods, but on balance
they're a very useful part of the language.

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "any" and "all" support multiple arguments

[Lucas Wiman]

On Tue, Aug 1, 2017 at 6:01 AM, Louie Lu  wrote:

> [...]
> I'm not sure if this is discuss before, but can "any" and "all"
> support like min_max "arg1, arg2, *args" style?
>

Can this be done consistently? For example consider x=[[]]. Then all(x)
where x is interpreted as an iterable should be False, but all(x) where x
is interpreted as a single argument should be True. This inconsistency
already exists for max:

>>> max({1, 2})
2
>>> max({1, 2}, {1})
set([1, 2])

However, it doesn't seem like there's a good reason to add an inconsistency
to the API for any/all.

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Lucas Wiman]

On Thu, Jul 20, 2017 at 5:19 AM, Victor Stinner 
wrote:

> For me, namedtuple was first used to upgrade an old API from returning a
> tuple to a "named" tuple. There was a hard requirement on backward
> compatibility: namedtuple API is a superset of the tuple API.
>
> For new code, there is no such backward compatibility issue. If you don't
> need a type, types.Namespace is a good choice.
>
> Using ns=types.Namespace, you can replace (x=0, y=1) with ns(x=0, y=1). It
> already works, no syntax change.
>
> *If* someone really wants (x=0, y=1) syntax sugar, I would prefer to get a
> namespace (no indexed (tuple) API).
>

It's a minor point, but the main reason I use namedtuple is because it's
far easier to get a hashable object than writing one yourself. Namespaces
are not hashable. If the (x=0, y=1) sugar is accepted, IMO it should
immutable and hashable like tuples/namedtuples.

Best,
Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime types vs static types

[Lucas Wiman]

On Sun, Jul 2, 2017 at 4:54 AM, Steven D'Aprano  wrote:

> I think that the current status is that the MyPy folks, including Guido,
> consider that it *is* reasonable to ask these questions for the purpose
> of introspection, but issubclass and isinstance are not the way to do
> it.
>

That may have once been the viewpoint of the developers of typing/MyPy,
though it seems to have changed. The current view is that this should be
implemented in a third party library. Further discussion is here
.

- Lucas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime types vs static types

[Lucas Wiman]

After rereading Koos' OP, I can now see that he was referring to a
different kind of runtime type checking than I am interested in. There was
a distinction I was unaware the core typing devs make between "typing-style
types" and "classes" that I'll discuss further in the typing repo itself.
Apologies for the digression.

- Lucas

On Sun, Jun 25, 2017 at 11:13 AM, Koos Zevenhoven  wrote:

> On Sun, Jun 25, 2017 at 8:47 PM, Lucas Wiman 
> wrote:
>
>> > ​Yes, but then `isinstance(tuple(range(100)), Tuple[int, ...])`
>> would be a difficult case.
>>
>> Yes, many methods on million-element tuples are slow. :-)
>>
>> Python usually chooses the more intuitive behavior over the faster
>> behavior when there is a choice. IIUC, the objections don't have anything
>> to do with speed.
>>
>>
> ​Sure, performance cannot dictate everything. But if you look at, for
> example, that multidispatch example I wrote in the OP, it would not be wise
> to do that expensive check every time. Also, people don't expect an
> isinstance check to be expensive. But looking at annotations, it is often
> possible to get the desired answer without checking all elements in a tuple
> (or especially without consuming generators etc. to check the types).
>
> -- Koos
> ​
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime types vs static types

[Lucas Wiman]

> ​Yes, but then `isinstance(tuple(range(100)), Tuple[int, ...])` would
be a difficult case.

Yes, many methods on million-element tuples are slow. :-)

Python usually chooses the more intuitive behavior over the faster behavior
when there is a choice. IIUC, the objections don't have anything to do with
speed.

- Lucas

On Sun, Jun 25, 2017 at 10:11 AM, Koos Zevenhoven  wrote:

> On Sun, Jun 25, 2017 at 7:13 PM, Lucas Wiman 
> wrote:
> [...]
>
>> >>> from typing import *
>> >>> isinstance(0, Union[int, float])
>> Traceback (most recent call last):
>>   File "", line 1, in 
>>   File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py",
>> line 767, in __instancecheck__
>> raise TypeError("Unions cannot be used with isinstance().")
>> TypeError: Unions cannot be used with isinstance().
>>
>> I think the natural reaction of naive users of the library is "That's
>> annoying. Why? What is this library good for?", not "Ah, I've sagely
>> learned a valuable lesson about the subtle-and-important-though-unmentioned
>> distinction between types and classes!" The restriction against runtime
>> type checking makes `typing` pretty much *only* useful when used with
>> the external library `mypy` (or when writing a library with the same
>> purpose as `mypy`), which is a pretty unusual situation for a standard
>> library module.
>>
>> Mark Shannon's example also specifically does not apply to the types I'm
>> thinking of for the reasons I mentioned:
>>
>>> For example,
>>> List[int] and List[str] and mutually incompatible types, yet
>>> isinstance([], List[int]) and isinstance([], List[str))
>>> both return true.
>>>
>>> There is no corresponding objection for `Union`; I can't think of any*
>> inconsistencies or runtime type changes that would result from defining
>> `_Union.__instancecheck__` as `any(isinstance(obj, t) for t in
>> self.__args__`.
>>
>
> ​One thing is that, as long as not all types support isinstance, then also
> isinstance(obj, Union[x, y, z])​ will fail for some x, y, z. From some
> perspective, that may not be an issue, but on the other hand, it may invite
> people to think that all types do support isinstance.
>
> For `Tuple`, it's true that `()` would be an instance of `Tuple[X, ...]`
>> for all types X. However, the objection for the `List` case (IIUC;
>> extrapolating slightly) is that the type of the object could change
>> depending on what's added to it. That's not true for tuples since they're
>> immutable, so it's not *inconsistent* to say that `()` is an instance of
>> `Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible
>> definition to the base case of an empty tuple.
>>
>>
> ​Yes, but then `isinstance(tuple(range(100)), Tuple[int, ...])` would
> be a difficult case.
>
> I think it all comes down to how much isinstance should pretend to be able
> to do with *types* (as opposed to only classes). Maybe isinstance is not
> the future of runtime type checking ;). Or should isinstance have a third
> return value like `Possibly`?
>
> -- Koos
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Runtime types vs static types

[Lucas Wiman]

>
> For some background on the removal of __instancecheck__, check the linked
> issues here:
>

Thanks for the reference (the most relevant discussion starts here
<https://github.com/python/typing/issues/136#issuecomment-104698674>).
That said, I think I totally disagree with the underlying philosophy of
throwing away a useful and intuitive feature (having `is_instance(foo,
Union[Bar, Baz])` just work as you'd naively expect) in the name of making
sure that people *understand* there's a distinction between types and
classes.

This seems opposed to the "zen" of python that there should be exactly one
obvious way to do it, since (1) there isn't a way to do it without a third
party library, and (2) the obvious way to do it is with `isinstance` and
`issubclass`. Indeed, the current implementation makes it somewhat
nonobvious even how to implement this functionality yourself in a
third-party library (see this gist
<https://gist.github.com/lucaswiman/21373bea33ccd2c5e868ec52b6eff412>).

One of the first things I did when playing around with the `typing` module
was to fire up the REPL, and try runtime typechecks:

>>> from typing import *
>>> isinstance(0, Union[int, float])
Traceback (most recent call last):
  File "", line 1, in 
  File "/Users/lucaswiman/.pyenv/versions/3.6/lib/python3.6/typing.py",
line 767, in __instancecheck__
raise TypeError("Unions cannot be used with isinstance().")
TypeError: Unions cannot be used with isinstance().

I think the natural reaction of naive users of the library is "That's
annoying. Why? What is this library good for?", not "Ah, I've sagely
learned a valuable lesson about the subtle-and-important-though-unmentioned
distinction between types and classes!" The restriction against runtime
type checking makes `typing` pretty much *only* useful when used with the
external library `mypy` (or when writing a library with the same purpose as
`mypy`), which is a pretty unusual situation for a standard library module.

Mark Shannon's example also specifically does not apply to the types I'm
thinking of for the reasons I mentioned:

> For example,
> List[int] and List[str] and mutually incompatible types, yet
> isinstance([], List[int]) and isinstance([], List[str))
> both return true.
>
> There is no corresponding objection for `Union`; I can't think of any*
inconsistencies or runtime type changes that would result from defining
`_Union.__instancecheck__` as `any(isinstance(obj, t) for t in
self.__args__`. For `Tuple`, it's true that `()` would be an instance of
`Tuple[X, ...]` for all types X. However, the objection for the `List` case
(IIUC; extrapolating slightly) is that the type of the object could change
depending on what's added to it. That's not true for tuples since they're
immutable, so it's not *inconsistent* to say that `()` is an instance of
`Tuple[int, ...]` and `Tuple[str, ...]`, it's just applying a sensible
definition to the base case of an empty tuple.

That said, it sounds like the decision has already been made, and this
would be quite useful functionality to have in *some* form. What do people
think about implementing `__contains__` (for `__instancecheck__`) and
`__lt__` (for `__subclasscheck__`) for these cases? Then there would still
be a convenient syntax for doing runtime type checking/analysis, but
wouldn't violate Mark Shannon's objections.

Best,
Lucas

* Counterexamples welcomed, of course!
<http://www.dictionary.com/browse/more-things-in-heaven-and-earth--horatio>


On Sun, Jun 25, 2017 at 6:21 AM, rym...@gmail.com  wrote:

> For some background on the removal of __instancecheck__, check the linked
> issues here:
>
>
> https://github.com/python/typing/issues/135
>
>
> --
> Ryan (ライアン)
> Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone 
> elsehttp://refi64.com
>
> On Jun 25, 2017 at 8:11 AM, > wrote:
>
> On Sat, Jun 24, 2017 at 11:30 PM, Lucas Wiman 
> wrote:
>
>> ​
>> On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven 
>>  wrote:
>>
>>> There has been some discussion here and there concerning the differences
>>> between runtime types and static types (mypy etc.). What I write below is
>>> not really an idea or proposal---just a perspective, or a topic that people
>>> may want to discuss. Since the discussion on this is currently very fuzzy
>>> and scattered and not really happening either AFAICT (I've probably missed
>>> many discussions, though). Anyway, I thought I'd give it a shot:
>>>
>>>
> ​[...]​
>
>
>
>> Regarding runtime types and isinstance, let's look at the Iterable[int]
>>> example. For this case, there are a few

Re: [Python-ideas] Runtime types vs static types

[Lucas Wiman]

>
> And what about PEP544 (protocols), which is being drafted? The PEP seems
> to aim for having type objects that represent duck-typing
> protocols/interfaces. Checking whether a protocol is implemented by an
> object or type is clearly a useful thing to do at runtime, but it is not
> really clear if isinstance would be a guaranteed feature for PEP544
> Protocols.
>
> So one question is, is it possible to draw the lines between what works
> with isinstance and what doesn't, and between what details are checked by
> isinstance and what aren't? -- Or should insinstance be reserved for a more
> limited purpose, and add another check function, say `implements(...)`,
> which would perhaps guarantee some answer for all combinations of object
> and type?
>

I'm guessing to implement PEP 544, many of the `__instancecheck__` and
`__subclasscheck__` methods in `typing.py` would need to be updated to
check the `__annotations__` of the class of the object it's passed against
its own definition, (covered in this section

of the PEP).

I've been somewhat surprised that many of the `__instancecheck__`
implementations do not work at runtime, even when the implementation would
be trivial (e.g. for `Union`), or would not have subtle edge cases due to
immutability (e.g. for `Tuple`, which cannot be used for checking
parameterized instances). This seems like counterintuitive behavior that
would be straightforward to fix, unless there are subtleties & edge cases
I'm missing.

If people are amenable to updating those cases, I'd be interested in
submitting a patch to that effect.

Best,
Lucas

On Sat, Jun 24, 2017 at 12:42 PM, Koos Zevenhoven  wrote:

> There has been some discussion here and there concerning the differences
> between runtime types and static types (mypy etc.). What I write below is
> not really an idea or proposal---just a perspective, or a topic that people
> may want to discuss. Since the discussion on this is currently very fuzzy
> and scattered and not really happening either AFAICT (I've probably missed
> many discussions, though). Anyway, I thought I'd give it a shot:
>
> Clearly, there needs to be some sort of distinction between runtime
> classes/types and static types, because static types can be more precise
> than Python's dynamic runtime semantics. For example, Iterable[int] is an
> iterable that contains integers. For a static type checker, it is clear
> what this means. But at runtime, it may be impossible to figure out whether
> an iterable is really of this type without consuming the whole iterable and
> checking whether each yielded element is an integer. Even that is not
> possible if the iterable is infinite. Even Sequence[int] is problematic,
> because checking the types of all elements of the sequence could take a
> long time.
>
> Since things like isinstance(it, Iterable[int]) cannot guarantee a proper
> answer, one easily arrives at the conclusion that static types and runtime
> classes are just two separate things and that one cannot require that all
> types support something like isinstance at runtime.
>
> On the other hand, there are many runtime things that can or could be done
> using (type) annotations, for example:
>
> Multidispatch (example with hypothetical syntax below):
>
> @overload
> def concatenate(parts: Iterable[str]) -> str:
> return "".join(parts)
>
> @overload
> def concatenate(parts: Iterable[bytes]) -> bytes:
> return b"".join(parts)
>
> @overload
> def concatenate(parts: Iterable[Iterable]) -> Iterable:
> return itertools.chain(*parts)
>
> or runtime type checking:
>
> @check_types
> def load_from_file(filename: Union[os.PathLike, str, bytes]):
> with open(filename) as f:
> return do_stuff_with(f.read())
>
> which would automatically give a nice error message if, say, a file object
> is given as argument instead of a path to a file.
>
> However useful (and efficient) these things might be, the runtime type
> checks are problematic, as discussed above.
>
> Furthermore, other differences between runtime and static typing may
> emerge (or have emerged), which will complicate the matter further. For
> instance, the runtime __annotations__ of classes, modules and functions may
> in some cases contain something completely different from what a type
> checker thinks the type should be.
>
> These and other incompatibilities between runtime and static typing will
> create two (or more) different kinds of type-annotated Python:
> runtime-oriented Python and Python with static type checking. These may be
> incompatible in both directions: a static type checker may complain about
> code that is perfectly valid for the runtime folks, and code written for
> static type checking may not be able to use new Python techniques that make
> use of type hints at runtime. There may not even be a fully functional
> subset of the two "languages". Different libraries will ad

Re: [Python-ideas] Dictionary destructing and unpacking.

[Lucas Wiman]

>
> It's still true.  In Python, if I need those things in variables
> *frequently*, I write a destructuring function that returns a sequence
> and use sequence unpacking.
>

Yes, you frequently need to write destructuring functions. Of course there
are ways to write them within Python, but they're often sort of ugly to
write, and this syntax would make writing them nicer. I was disagreeing
with your assertion that this is an uncommon use case: it's not, by your
own admission. Of course, it is business logic which can be factored into a
separate method, it's just that it would often be nice to not need to write
such a method or define a NamedTuple class.

If this feature does get added, I think it would make more sense to add it
as part of a general framework of unification and pattern matching. This
was previously discussed in this thread:
https://mail.python.org/pipermail/python-ideas/2015-April/thread.html#32907

- Lucas

On Thu, Jun 8, 2017 at 12:15 AM, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Lucas Wiman writes:
>
>  > > Maps with a known, fixed set of keys are relatively uncommon
>  > > in Python, though.
>  >
>  > This is false in interacting with HTTP services, where frequently you're
>  > working with deserialized JSON dictionaries you expect to be in a
> precise
>  > format (and fail if not).
>
> It's still true.  In Python, if I need those things in variables
> *frequently*, I write a destructuring function that returns a sequence
> and use sequence unpacking.  If I don't need the values in a
> particular use case, I use the "a, _, c = second_item_ignorable()"
> idiom.
>
> Or, in a larger or more permanent program, I write a class that is
> initialized with such a dictionary.
>
> If you like this feature, and wish it were in Python, I genuinely wish
> you good luck getting it in.  My point is just that in precisely that
> use case I wouldn't be passing dictionaries that need destructuring
> around.  I believe that to be the case for most Pythonistas.
> (Although several have posted in favor of some way to destructure
> dictionaries, typically those in favor of the status quo don't speak
> up until it looks like there will be a change.)
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dictionary destructing and unpacking.

[Lucas Wiman]

>
> Maps with a known, fixed set of keys are relatively uncommon
> in Python, though.


This is false in interacting with HTTP services, where frequently you're
working with deserialized JSON dictionaries you expect to be in a precise
format (and fail if not).

On Wed, Jun 7, 2017 at 11:32 PM, Greg Ewing 
wrote:

> C Anthony Risinger wrote:
>
>> Incredibly useful and intuitive, and for me again, way more generally
>> applicable than iterable unpacking. Maps are ubiquitous.
>>
>
> Maps with a known, fixed set of keys are relatively uncommon
> in Python, though. Such an object is more likely to be an
> object with named attributes.
>
> --
> Greg
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defer Statement

[Lucas Wiman]

I agree that the stated use cases are better handled with ExitStack.  One
area where `defer` might be useful is in lazy-evaluating global constants.
For example in a genomics library used at my work, one module involves
compiling a large number of regular expressions, and setting them as global
contants in the module, like:

FOO1_re = re.compile(r'...')
FOO_TO_BAR_re = {foo: complicated_computation_of_regex(foo) for foo in
LONG_LIST_OF_THINGS}
...

This utility module is imported in a lot of places in the codebase, which
meant that importing almost anything from our codebase involved
precompiling all these regular expressions, which took around 500ms to to
run the anything (the test runner, manually testing code in the shell,
etc.) It would be ideal to only do these computations if/when they are
needed. This is a more general issue than this specific example, e.g. for
libraries which parse large data sources like pycountry (see my PR

for one possible ugly solution using proxy objects; the author instead went
with the simpler, less general solution of manually deciding when the data
is needed). See also django.utils.functional.lazy
,
which is used extensively in the framework.

A statement like: `defer FOO = lengthy_computation_of_foo()` which deferred
the lengthy computation until it is used for something would be useful to
allow easily fixing these issues without writing ugly hacks like proxy
objects or refactoring code into high-overhead cached properties or the
like.

Alternatively, if there are less ugly or bespoke ways to handle this kind
of issue, I'd be interested in hearing them.

Best,
Lucas



On Sat, Jun 3, 2017 at 11:38 PM, Nick Coghlan  wrote:

> On 3 June 2017 at 22:24, Nick Coghlan  wrote:
> > So while I'm definitely sympathetic to the use case (otherwise
> > ExitStack wouldn't have a callback() method), "this would be useful"
> > isn't a sufficient argument in this particular case - what's needed is
> > a justification that this pattern of resource management is common
> > enough to justify giving functions an optional implicit ExitStack
> > instance and assigning a dedicated keyword for adding entries to it.
>
> It occurred to me that I should elaborate a bit further here, and
> point out explicitly that one of the main benefits of ExitStack (and,
> indeed, the main reason it exists) is that it allows resource
> lifecycle management to be deterministic, *without* necessarily tying
> it to function calls or with statements.
>
> The behave BDD test framework, for example, defines hooks that run
> before and after each feature and scenario, as well as before and
> after the entire test run. I use those to set up "scenario_cleanup",
> "_feature_cleanup" and "_global_cleanup" stacks as part of the testing
> context: https://github.com/leapp-to/prototype/blob/master/
> integration-tests/features/environment.py#L49
>
> If a test step implementor allocates a resource that needs to be
> cleaned up, they register it with "context.scenario_cleanup", and then
> the "after scenario" hook takes care of closing the ExitStack instance
> and cleaning everything up appropriately.
>
> For me, that kind of situation is when I'm most inclined to reach for
> ExitStack, whereas when the cleanup needs align with the function call
> stack, I'm more likely to reach for contexlib.contextmanager or an
> explicit try/finally.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] π = math.pi

[Lucas Wiman]

I like the aesthetic of the idea, but it seems like this would be a better
fit in a library namespace like sympy or jupyter.

On Thu, Jun 1, 2017 at 12:19 AM, Sven R. Kunze  wrote:

> On 01.06.2017 08:47, Serhiy Storchaka wrote:
>
> What you are think about adding Unicode aliases for some mathematic names
> in the math module? ;-)
>
> math.π = math.pi
> math.τ = math.tau
> math.Γ = math.gamma
> math.ℯ = math.e
>
> Unfortunately we can't use ∞, ∑ and √ as identifiers. :-(
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> No, thank you. :)
>
> One way to do it is perfectly fine.
>
> Regards,
> Sven
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/