[Python-ideas] Re: join() could add separators at start or end
Currently you can write term.join(xs) + term if you want 1, 1, 2, 3, ... terminators when xs has 0, 1, 2, 3, ... elements, and term.join([*xs, '']) # or b'' if you want 0, 1, 2, 3, ... terminators, neither of which is prohibitively annoying. What I don't like about the status quo is that I'm never sure that the people who wrote "sep.join(xs) + sep" really want the separator when xs is empty. Unless it's obvious from surrounding code that xs can't be empty, I always worry it's a bug. In your PIL example, arr = [] for elt in (description, cpright): if elt: arr.append(elt) return "\r\n\r\n".join(arr) + "\r\n\r\n" do they really want four newlines when the description and copyright are both empty? I suspect not but I don't know. There's no clarifying comment. In email.contentmanager they call splitlines on some text, then rejoin it with '\n'.join(lines) + '\n'. It looks like the input can be empty since there is some special-case code for that. Do they know that they're increasing the number of newlines in that case? There's no clarifying comment. As for term.join([*xs, '']), while it seems less likely to be a bug, it's not very natural. You aren't adding an extra blank thing at the end, you're just terminating the things you already had. So I think it would be nice to have a way to say explicitly and concisely what you want to happen with an empty list. I suppose this idea will fail for the usual reason (no good syntax), but here's an attempt: term.joinlines(xs)in place of term.join([*xs, '']) term.joinlines(xs) or termin place of term.join(xs) + term The second one is less concise than before, but it doesn't give me that uneasy feeling. Adding a separator before the first element doesn't seem important enough to me to justify the complexity of adding an option for it. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZZ4MNCP2AFWKLAAT4QZFOM7LBWNPES7B/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Deprecate misleading escapes in strings
On Thu, Feb 16, 2023 at 8:09 AM Barry wrote: > This is valid and does not match your rules. ‘\x9b’ that is the ANSI CSI > in 8-bit. > In 7-bit it is ‘\x1b[‘. > Shouldn't that be b‘\x9b’? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ACL4IZXVS3DWMXTGPS7RUSGARTRXY6NW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Generalized deferred computation in Python
On Wed, Jun 22, 2022 at 6:36 PM Joao S. O. Bueno wrote: > implement "all possible" > dunder methods, and proxy those to the underlying object, for a "future > type" that was > calculated off-process, and did not need any ".value()" or ".result()" > methods to be called. > Here's a package on PyPI that seems to do that: https://pypi.org/project/lazy-object-proxy/ It's written partly in C, so it may be fast. I haven't tested it. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CUEVSLDF5AAGN4MFIXU4TQ47Z35AXXAC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Generalized deferred computation in Python
>
> On Wed, Jun 22, 2022 at 02:30:14PM -0400, David Mertz, Ph.D. wrote:
> >But basically, think about `x = (later expensive1() + later expensive2())
> /
> >later expensive3()`. How can we make `x` itself be a zero argument
> >lambda? [... see below ...]
>
x = lambda: (expensive1() + expensive2()) / expensive3()
What am I missing?
I don't understand what you meant by saying that zero-argument lambdas are
"isolated". It sounds like a way that you happen to think about lambdas,
and not an objective property of lambdas.
This proposal is like lambda on the definition end, but on the invocation
end the call happens implicitly. In effect you have to explicitly mark
everywhere that you *don't* want it to be called instead of everywhere that
you *do* want it to be called. It isn't clear to me that that's better,
much less enough better to justify changing the semantics of what I suppose
is the single most common operation in Python ("getting a value").
On Wed, Jun 22, 2022 at 2:53 PM Martin Di Paola
wrote:
> # Using zero-argument nested lambdas
> x = lambda: ((lambda: expensive1())() + (lambda: expensive2())()) /
> (lambda: expensive3())()
>
Why not just expensive1() instead of (lambda: expensive1())()?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/FXE5U2JBWZCBSHE5Z2DU3POMBW5K6JKM/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Addition to fnmatch.py
This calls the predicate once per element: def partition(pred, iterable): t1, t2 = tee((pred(x), x) for x in iterable) return (x for b, x in t1 if not b), (x for b, x in t2 if b) It's kind of inefficient though. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DQBFYD7KYRG6OOMXIY7IH45S25U6SXTK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Expand the try-expect syntax to support conditional expect block
On Wed, May 18, 2022 at 12:24 PM Serhiy Storchaka wrote: > try: > expression block > expect Exception if condition else (): > expression block > That's an interesting idea I haven't seen before, but it doesn't work if the condition you want to test depends on the exception object, which in my experience it often does. Some of OP's examples are of that form: for instance, in setuptools/sandbox.py: try: ... except SystemExit as v: if v.args and v.args[0]: raise # Normal exit, just return In pip/_internal/utils/misc.py: try: os.makedirs(path) except OSError as e: # Windows can raise spurious ENOTEMPTY errors. See #6426. if e.errno != errno.EEXIST and e.errno != errno.ENOTEMPTY: raise I suppose the right way to implement it would be to support PEP 622 patterns in except clauses. except case SystemExit(args=args) if not args or not args[0]: pass # Normal exit, just return except case OSError(errno = errno.EEXIST | errno.ENOTEMPTY): # Windows can raise spurious ENOTEMPTY errors. See #6426. pass ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/JIJQFAXIS4TGURIMAZOGCBMEBYM6FXOD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: New Tool Proposal
Cython seems to me rather different from what the original post was talking about. If I put class Foo: def bar(self): pass in a .pyx file and run it through Cython, the result is a 5600-line, 200-kilobyte lump of C that clearly isn't meant to be understood or modified by human beings. It is mostly boilerplate, but not the sort of boilerplate that could serve as the starting point for an extension module written (entirely) in C. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CWFDES62ZIJR32XLDU2GF43SGQMCYSTU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Native support for units [was: custom literals]
On Sun, Apr 3, 2022 at 10:10 PM Chris Angelico wrote: > On Mon, 4 Apr 2022 at 14:13, Ricky Teachey wrote: > > What does that idea bring other than being able to say 5.0m > [...] instead of 5.0*m [...]? > > A large amount of clarity, readability, and namespacing (you don't have to > pollute your global namespace with a large number of single-letter names, > since these tokens will ONLY have meaning when immediately following an int > or float literal). > I feel like one of the biggest sticking points in this thread is that people are arguing for a new kind of global scope just for units, and the sole reason seems to be that they want short names for them. The register_numeric_suffix idea would create a true global namespace, independent of the module system. That seems like a bad idea: libraries should be able to use their own units internally without potentially breaking other libraries. Units should be local to each module. You need a way to import them into your module's unit namespace. You might want to import only some of the units exported by a unit provider... There is already a solution to all of these problems. All you have to do to be able to use that solution as-is, instead of introducing a large amount of new complexity to the language, is give your units names like "ampere" instead of "A". I don't think that would be much of a problem. How often will explicit unit names show up in a program? Maybe you'll multiply by 1 meter in one place when reading a CSV file, and divide by 1 meter when writing. You probably won't write 1.380649e-23 J/K inline, even if you only use it once; you'll assign it to k_B or something. Or just import it from a physics-constants module. If you're doing a-la-carte computations at the interactive prompt, you can "from units import *" for convenience; the official docs already advise "from math import *" in that situation. in theory, we could just have a variable called "j" (or "i" if you prefer) > which you multiply by something and add something, and that's your > complex number. But it's cleaner to write "3+4j". > I would like to see examples of how complex literals are used in the wild. I feel like most will look like (4086.184383622179764082821 - 3003.003538923749396546871j) (an actual example from mpmath.functions.elliptic), or are just ±1j. There's practically no situation in which you'd want a literal like 3+4j. Even crazy constants like those from mpmath are unlikely to appear in your code because most people aren't numeric analysts who write math libraries. I feel like the biggest benefit of the suffix j syntax is that it's recognizable as a foldable compile-time constant, so you can put foo+barj in an inner loop cheaply. mpmath has "3.1415926535897932j" in a couple places, which I suppose is for speed; it certainly isn't for readability. Python should have some generic solution for this problem, like C++ constexpr, but I don't know how to do it, and it's a different discussion. >>> 3 * ureg.meter + 4 * ureg.cm Same problem here: I don't believe that anyone would write that in a real program. How are libraries like pint actually used? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VO6JTW742ENSAVO4U4LO6V5N5IZCDGB2/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: A modulo operator consistent with euclidean division.
On Fri, Mar 18, 2022 at 8:30 PM MRAB wrote: > Wikipedia describes Euclidean division. > > Basically, the modulo is non-negative: > > a == b * q + r where 0 <= r < abs(b) That convention in the Wikipedia article dates back to a 2004 edit by an anonymous (IP) editor. The only reference in that version was to a course handout that only considered positive denominators. There's nothing wrong with the convention, but I'm suspicious of the idea that it's a widespread standard of some sort. I've never heard of it before. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MG7J6KWWBWZPMWVZYSMAXF4WVG2WL42A/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: A modulo operator consistent with euclidean division.
On Fri, Mar 18, 2022 at 8:31 AM Chris Angelico wrote: > if y < 0: return -(x // -y), x % y I think that should be if y < 0: return -(x // -y), x % -y ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WDWITXUG2U7FHNRHJ7HJY6IIODQELCBF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: repeat until
On Tue, Mar 1, 2022 at 4:51 PM Steven D'Aprano wrote: > Then Python is the wrong language for you, because it uses exceptions to > direct control flow *wink* > > The iteration protocol uses StopIteration to end iteration. The older > sequence protocol uses IndexError for the same purpose. > I think it's better to say that exceptions are used in these cases to cover for the lack of disjoint-union types. Many functions return "a value or no value". When the values they can return are limited, a value they can't return is often used as a stand-in for "no value", like None or NotImplemented, or -1 for find(). When that isn't the case, an exception tends to be used, although letting the caller choose a sentinel is also popular. Haskell tends to use Maybe in all of these cases, because it's a disjoint union: the no-value return can't overlap with the yes-value returns, so there's never a need for anything else. (And it forces you to unwrap the returned value, so you can't forget to check for the no-value case.) The MyBreak proposal doesn't really fit that pattern. I've never seen it used, and I feel like it would be rejected in most places that have style guides and code review, but perhaps I'm wrong. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/PFJS2MLN5BRFTC53BZDFJ2Z3L7O7LWIJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: repeat until
On Tue, Mar 1, 2022 at 2:23 PM Steven D'Aprano wrote: > try: > do_this() > if condition: raise MyBreak > do_that() > if condition: raise MyBreak > do_next_step() > if condition: raise MyBreak > do_last_step() > except MyBreak: > pass > Why not: while True: [loop body with ordinary breaks] break It's less to write and almost free of charge*, and it also supports redo (continue). MyBreak has the advantage that you can raise it in nested loops and subfunctions, although that could be a disadvantage since it makes the control flow confusing. This goes for repeat-until also: I always write it while True: ... if cond: break. I just grepped through some personal Python projects and about 40% of my while-loop conditions are True. * The break at the end is free, but Python 3.10.2 inserts a NOP at the location of the while True, and the peephole optimizers of earlier Python versions have done other things. It looks like there's been a lot of work on bytecode efficiency in 3.11, so maybe it will become truly free, but I haven't checked. 3.11 has range-based exception handling (issue 40222), so the try: approach is free if you don't raise MyBreak (but expensive if you do). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EBNBHLKGNPGF73QKASXM4JF535AMA2UU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make 10**400 / 1e200 work?
On Mon, Feb 21, 2022 at 1:39 PM Tim Peters wrote: > [Mark Dickinson ] > > It Would Be Nice If > > `-1 * complex(inf, 0.0)` gave `complex(-inf, -0.0)` instead > > of the current result of `complex(-inf, nan)`. > Except replacing -1 with "-1.0" or "complex(-1)" would > presumably _still_ return complex(-inf, nan), despite that > > >>> -1 == -1.0 == complex(-1) > True > I think Python should do what C99 and C++ do, which is define complex(a, b) * c to mean complex(a * c, b * c). Python's complex constructor accepts two complex arguments, so that definition works even in the complex * complex case, though in practice you'd want to optimize for common argument types and defer to __rmul__ if the second argument is of unknown type. It does have the consequence that values that compare equal can have different arithmetic behavior. That happens in IEEE arithmetic with ±0, and it's a good thing in certain cases. IEEE arithmetic deliberately preserves the sign of 0 where it can. It's not an accident of the representation. There is a difference between a float and a complex with ±0 imaginary part that justifies the different answers in this case: the former is real by construction, while the latter may have underflowed. It's unfortunate that underflowed values compare equal to true zero, but there's a reason they say you shouldn't compare floating-point numbers for equality. If that's wanted, better for complex.__mul__ > to detect on its own whether component parts are 0, > and use a simpler multiplication implementation if so. > I think it's better not to do that for the reason in the previous paragraph. For example, this similar surprise has nothing to do with type promotion: > > >>> 1j * complex(math.inf, -0.0) > (nan+infj) > 1j is, in effect, being prematurely promoted to complex because Python lacks an imaginary type. C99 has _Imaginary for this reason. Another consequence of the missing imaginary type is that you can't write negative zeros in complex literals consistently. C++ doesn't have std::imaginary, possibly because it has no imaginary literals. That, plus I'm still waiting for a plausible use case ;-) > Why are complex numbers in core Python in the first place? I'm not sure, but I think it's for the same reason as Ellipsis and binary @ and the third argument to slice: to support numpy, since numpy can't define its own syntax. The core devs wouldn't normally add syntax just for some third-party library, but numpy is so important that they bend the rules. People use numpy to do heavy-duty real-world number crunching. The weird IEEE corner cases actually affect the stability of these calculations; that's why the IEEE standard tried to pin down their behavior. I think that improving Python's built-in numerics would have benefits for numpy users in the form of fewer failed computations (mysteriously failed for the many that don't have the numeric-analysis chops to work out what went wrong). I think it would have strategic value. It's less noticeable than adding syntax, but also easier. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5UTUOQ73XOIMDHQZM3GSQH3XMLPEN7XA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make 10**400 / 1e200 work?
On Mon, Feb 21, 2022 at 12:24 PM Mark Dickinson wrote:
> e.g., It Would Be Nice If `-1 * complex(inf, 0.0)` gave `complex(-inf,
> -0.0)` instead of the current result of `complex(-inf, nan)`. But the price
> in added complexity - both conceptual complexity and implementation
> complexity - seems too high.
>
There are some heterogeneous binary operations implemented already. For
example 10**400 < float('inf') correctly returns True even though there's
no type in Python to which both values could be losslessly coerced to
perform that comparison.
I don't think adding a few more cases like that implies an obligation to
fix every case. It's a "perfect is the enemy of the good" situation.
I disagree with your notion of conceptual complexity. The conceptually
simplest thing to do is to perform the requested operation exactly and then
round if necessary. Only practical difficulties prevent that from happening
in all cases. It's not conceptually simple to convert int to float and
float to complex blindly regardless of the consequences. C99 gets complex *
real multiplication right, despite C being the poster child for numeric
promotions, and so does C++. It isn't hard to get it right, and programmers
would be happier if Python got it right. Programmers don't expect or want
consistent promotion to complex even when it breaks things.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/5IOGT4XDKK2346C75QTFCU2SCTX4G373/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make 10**400 / 1e200 work?
On Sun, Feb 20, 2022 at 9:41 AM Tim Peters wrote: > It's a slippery slope, of scant discernible benefit, and still hackish. > For example, 10**600 / 1e200 / 1e200. > That's how IEEE arithmetic works: a/b is computed to infinite precision then properly rounded, but (a/b)/c isn't. Yes, it's not ideal, but it was doable, and was better than not even having consistent behavior for a/b, which was the state of things before the standard. Python follows the IEEE rounding model for float/float and int/int. Following it for float/int and int/float would be pretty easy since the hard work has already been done to support int/int. Let's just do it, and not worry about whether it's hypocritical to fix this without fixing the bigger problem. Once you start using slippery-slope arguments, pretty soon you're using them for everything, and progress grinds to a halt... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UJQWFDWOOGYQDWWDT5CVLF4HN3ML7NBU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make 10**400 / 1e200 work?
On Saturday, February 19, 2022, Stefan Pochmann
wrote:
> >>> 1e200.is_integer()
> True
>
> So that could losslessly be converted to int, and then the division would
> succeed
If the float argument isn't an integer, you can multiply both sides by a
power of 2 that makes it one (if it's finite, obviously), so this would
still work.
Currently, int/int and float/float are always properly rounded. It would be
nice if you could just say that a/b is properly rounded whenever a and b
are int or float, and forget about conversion issues. So I'm +1 on this.
It would make certain divisions more expensive than they currently are, but
only those where one argument is float and the other is an int that isn't
exactly representable as a float (meaning, in particular, its absolute
value is greater than 2**53). It seems unlikely to me that anyone is doing
a lot of divisions of that kind and needs the speed, but if they are and
do, casting to float beforehand is an easy workaround.
Also, 10**400/float('inf') should return +0.0 instead of raising
OverflowError, and 10**400/float('nan') should be NaN. The division should
behave as though both arguments are promoted to arbitrary-precision
floating point (but the result is rounded to 64-bit float). I wouldn't
suggest that if it was hard to implement, but I think it's pretty easy
when long_true_divide already exists.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/6GP4CN6DGNJ4WCYOPYE7BXDAL5NFDREH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Regex timeouts
On Tue, Feb 15, 2022 at 6:13 PM Chris Angelico wrote:
> Once upon a time, a "regular expression" was a regular grammar. That is no
> longer the case.
>
I use "regex" for the weird backtracking minilanguages and deliberately
never call them "regular expressions". (I was under the impression that the
Perl documentation observed the same convention but that doesn't seem to be
true.)
Once upon a time, a regular expression could be broadly compatible with
> multiple different parser engines.
>
I think there never was such a time, at least not if you wanted syntactic
compatibility.
Is there any sort of standardization of regexp syntax and semantics[...]?
>
I'm not sure there needs to be. There is no standardization of
programming-language syntax in general, unless you count conventions like
{...} for blocks which Python ignores.
The problem as I see it isn't that the syntax isn't standardized. It's that
the syntax, to the extent it is standardized, is terrible. The only
traditional Unix tool whose regular expression syntax isn't godawful is
lex, and unfortunately that isn't the one that caught on.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/GSYIG5EKDFPD6NOJCS3LXQJVFFGNBMED/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: "frozen" operator Re: Revisiting a frozenset display literal
There seem to be two different reasons people want a generic freeze syntax: 1. Making a hashable copy of an arbitrary object 2. Avoiding O(n) rebuilding of literals on every use (a constant for bytecode, as you put it) In both 1 and 2, not only the object but all of its children need to be immutable. For 2, that's the status quo, but for 1 it seems like a bigger problem. There is already a solution of sorts for 1: pickle. It may even be more efficient than a subobject-by-subobject deep freeze since it stores the result contiguously in RAM. On the other hand it can't share storage with already-hashable objects. For the second one, I would rather have an "inline static" syntax (evaluates to the value of an anonymous global variable that is initialized on first use), since it would be more broadly useful, and the disadvantages seem minor. (The disadvantages I see are that it's built per run instead of per compile, it's theoretically mutable (but mutation would be evident at the sole use site), and it may use more heap space depending on how constants are implemented which I don't know.) As for the syntax... well, backticks, obviously... ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YM3Z2NMM6AZNSZ67KET6QBIBEFRD3WYW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Revisiting a frozenset display literal
My preferred syntax for a frozenset literal would be something like
{1, 2, 3}.freeze()
This requires no new syntax, and can be safely optimized at compile time
(as far as I can tell).
set.freeze would be a new method of sets which could also be used at run
time. It would return a new frozenset object and wouldn't alter the set
object (so perhaps the name I suggested isn't ideal). Of course
frozenset.freeze would just return itself.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/LYGTXEHMUVVPC3DAHUDLMKLP2574GX5L/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: List comprehension operators
If you didn't know, range objects already support most non-mutating list methods: >>> fakelist = range(1, 101) >>> fakelist[-1] 100 >>> fakelist[-10:] range(91, 101) >>> 50 in fakelist True >>> fakelist.index(50) 49 Range objects are more efficient than lists since they use O(1) memory instead of O(n), and many of the methods take O(1) time instead of O(n). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VTRVLDZU35OZEMWITGZU64B6HNQZGWV4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults
On Sat, Oct 23, 2021 at 5:16 PM Chris Angelico wrote: > # Very common: Use None and replace it in the function > def bisect_right(a, x, lo=0, hi=None, *, key=None): > if hi is None: > hi = len(a) > Note that if this is changed to def bisect_right(a, x, lo=0, hi[whatever]len(a), *, key=None): then you'll lose the ability to pass hi=None to mean the end of the array. For this argument of bisect_right, None actually makes sense since it's used for that purpose in slices, and it would probably be a backward-compatibility problem to drop support for it also. So you will still have to write def bisect_right(a, x, lo=0, hi[whatever]len(a), *, key=None): if hi is None: hi = len(a) Self-referential expressions will result in UnboundLocalError:: > > def spam(eggs=>eggs): # Nope > That seems like not a rule of its own, but a special case of this rule: deferred arguments that haven't been assigned to yet are unbound (and not, say, set to some implementation-specific value analogous to _USE_GLOBAL_DEFAULT). > def bisect(a, hi=>len(a)): > That looks like an anonymous function that takes a single parameter named hi, ignores it, and returns the length of a free variable named a. It may even mean that in Python someday in expression contexts. But it's definitely not what it means here. hi<=len(a) would make a bit more sense. > def bisect(a, hi=:len(a)): > This one annoys me the least, though I can't say why. def bisect(a, hi?=len(a)): > ? seems reasonable for omitted/defaulted arguments, but that's not what this PEP is about. I can't see why call-time evaluation would have a ? and def-time evaluation wouldn't. def bisect(a, hi!=len(a)): > Same basic problem as => (and <=). Is hi equal to len(a), or not? def bisect(a, hi=`len(a)`): > It looks like the backticks are part of the expression rather than the function-parameter syntax, but they aren't, and never could be - even if there was a deferred-evaluation syntax at the expression level, it couldn't be used here, because the lexical environment would be wrong. > Since default arguments behave largely the same whether they're early or > late > bound, They seem very different to me. They're evaluated in a different lexical scope, and many use cases for this extension depend on the changed scoping. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/22DEQDF35SOBW6OC2Z2GGBVAQKIKHMPM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Syntax for late-bound arguments
On Sat, Oct 23, 2021 at 11:53 PM Steven D'Aprano wrote: > If we had thunks, that would give us late binding for free: > > def bisect(a, x, lo=0, hi=thunk len(a), *, key=None) > I'm unclear on exactly what the semantics of a thunk would be, but I don't see how it could do what you want here. In an ordinary default value, the "a" in "len(a)" refers to a variable of that name in the enclosing scope, not the argument of bisect. A generic delayed-evaluation mechanism wouldn't (shouldn't) change that. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TUOGJLRRYHBGRJRTYVT4YXPLNY2JJW6U/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: We should have an explicit concept of emptiness for collections
On Tue, Aug 24, 2021 at 9:08 AM Jon Kiparsky wrote: > > Numbers in general are useful concepts that help us with real-world > > problems, but it's really hard to pin them down. > > Not that hard, really. A number is just a hackenbush game :) > RIP Berlekamp, Conway and Guy. They all died within about a year of each other. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VULTYS2LPVNN3XCOOWHTQPK7FG47PAZA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Deprecate sum of lists
On Thu, Jun 17, 2021 at 3:09 PM Steven D'Aprano wrote:
> On Thu, Jun 17, 2021 at 02:22:29PM -0700, Ben Rudiak-Gould wrote:
> > [*chunk for chunk in list_of_lists]
>
> What would that do?
The difference between chunk and *chunk in the expression of a list
comprehension would be the same as the difference between them in the
expressions of a starred_list.
The only thing I can guess it would do is the
> equivalent of:
>
> result = []
> for chunk in list_of_lists:
> result.append(*chunk)
>
> which is a long and obfuscated way of saying `raise TypeError` :-)
>
It would be reasonable to allow list.append to take any number of arguments
to be appended to the list, as though its definition was
def append(self, *args):
self.extend(args)
If it did, then that translation would work and do the right thing.
Some similar functions do accept multiple arguments as a convenience,
though it's not very consistent:
myset.add(1, 2) # no
myset.update([1, 2], [3, 4]) # ok
mylist.append(1, 2) # no
mylist.extend([1, 2], [3, 4]) # no
mydict.update({'a': 1}, b=2, c=3) # ok
mydict.update({'a': 1}, {'b': 2}, c=3) # no
Well, there is this:
>
> result = []
> for chunk in list_of_lists:
> *temp, = chunk
> result.append(temp)
>
> which would make it an obfuscated way to spell `list(chunk)`.
>
Unpacking would be useless in every context if you interpreted it like that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/NNZKPJUD73WIQPUHZ3QNWPHQSM6BYUDP/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Deprecate sum of lists
On Thu, Jun 17, 2021 at 12:37 AM Serhiy Storchaka wrote: > And it is equivalent to pure Python code > > [x for chunk in list_of_lists for x in chunk] > Okay, slightly off-topic, but can we *please* allow [*chunk for chunk in list_of_lists] some day. I think it was left out because some discussion concluded it would be too confusing, which is ridiculous. I assumed it would work and was confused to find that it didn't. It's blatantly inconsistent. It's not even the performance I care about, it's the unreadability of having an extra "for i_have_to_think_of_yet_another_variable_name in whatever" at the end of the list comprehension (at maximum distance from where the variable is used). I've wished for this feature ridiculously many times. > It would be > possible to make the compiler recognizing such pattern and generating > more efficient bytecode (LIST_EXTEND instead of an internal loop with > LIST_APPEND), but I am not sure that this case is common enough to > complicate the compiler. > In my personal experience it's very common. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YH7COSZIV5BWQ3EOEA5MQMBHMB3OQNND/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Looking for people interested in a Python register virtual machine project
On Sun, Mar 21, 2021 at 11:10 PM Chris Angelico wrote: > At what point does the process_objects list cease to be referenced? > After the last visible use of it, or at the end of the function? In Python as it stands, at the end of the function, as you say. Skip Montanaro's PEP suggested that in his register machine, locals would be dereferenced after their last visible use. I don't think that's intrinsically a bad idea, but it's not backward compatible. The thing with the process objects was just an example of currently working code that would break. The example has nothing to do with PyQt5 really. I just happen to know that QProcess objects kill the controlled process when they're collected. I think it's a bad design, but that's the way it is. Another example would be something like td = tempfile.TemporaryDirectory() p = subprocess.Popen([..., td.name, ...], ...) p.wait() where the temporary directory will hang around until the process exits with current semantics, but not if td is deleted after the second line. Of course you should use a with statement in this kind of situation, but there's probably a lot of code that doesn't. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/EQ372ZPJWQW2GKCLPGXX2A6VKMRZRB36/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Looking for people interested in a Python register virtual machine project
In the "Object Lifetime" section you say "registers should be cleared upon last reference". That isn't safe, since there can be hidden dependencies on side effects of __del__, e.g.: process_objects = create_pipeline() output_process = process_objects[-1] return output_process.wait() If the process class terminates the process in __del__ (PyQt5's QProcess does), then implicitly deleting process_objects after the second line will break the code. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TZRDILZIQXOEZLMK5HOH2HBUG5JY2ZMZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Itertools generator injection
On Fri, Mar 19, 2021 at 8:59 PM Dennis Sweeney wrote: > I think these don't generally fit with the "feel" of the itertools module > though since almost everything there accepts iterables and returns iterators itertools.product as written barely seems to fit. It might as well accept only sequences, since it immediately converts everything into a sequence anyway. It's also documented as "equivalent to a nested for-loop" which is plainly untrue for several different reasons. A simple improvement would be to keep the first argument as an uncached iterator if repeat=1. Not only would that improve memory usage in many cases, but it would allow for things like product(count(1), ...) which ought to work in my opinion. Another option is to accept iterables that repeatedly get iter() called on > them, but that's hard to make work with generators That variant of itertools.product (one which is truly equivalent to a nested for loop) would have to be a new function, since it wouldn't be backward compatible. The lack of a convenient way to make it work with generators feels like a hole in the standard library that should be fixed: class ReiterableGenerator: # defined in itertools def __init__(self, f, *args, **kwargs): self._f, self._args, self._kwargs = f, args, kwargs def __iter__(self): return self._f(*self._args, **self._kwargs) primes = ReiterableGenerator(lambda: (n for n in count(2) if isprime(n))) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HQMXRFHEKEEMWHYPPQZKZXXBGKWDEI2C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: SimpleNamespace vs object
On Wed, Feb 17, 2021 at 6:12 PM Chris Angelico wrote: > But if object() could get arbitrary attributes, then __slots__ wouldn't > work. > It seems to me that all you'd have to do is add a line or two to the add_dict logic in type_new so that instances of object get a dict. Then instances of object would get a dict, and nothing else would change. In languages like C++ where an instance of a class contains actual in-memory instances of all of its superclasses, that wouldn't work. In Python, where instances of different classes have a priori nothing to do with each other, I think it would work. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2R4FMO5UH6MFTS25BRNJJQB34DKMAZHI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add a couple of options to open()'s mode parameter to deal with common text encodings
On Thu, Feb 4, 2021 at 3:29 PM Chris Angelico wrote: > With "t", it takes/gives Unicode objects, but with "b" it uses bytes. Sure, in Python 3, but not in Python 2, or C. Anyway, moral correctness is beside the point. People in point of fact don't write encoding='utf-8' when they should, because it's so much to type. If you had to write binary=True to enable binary mode, fewer people would have bothered to use it in the Python 2 era, and there would have been more portability (and Python 3 transition) problems. There shouldn't have been, but there would have been. Everything about the mode parameter is a sop to convenience. Really you should write open(mode=io.APPEND) or something. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7ACOQP2RG75UMJB2BS3XPXC6PZ3UQGFW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Add a couple of options to open()'s mode parameter to deal with common text encodings
There's a long ongoing thread with the subject "Make UTF-8 mode more accessible for Windows users." There are two obvious problems with UTF-8 mode. First, it applies to entire installations, or at least entire running scripts, including all imported libraries no matter who wrote them, etc., making it a blunt instrument. Second, the problem on Windows isn't that Python happens to use the wrong default encoding, it's that multiple encodings coexist, and you really do have to think each time you en/decode something about which encoding you ought to use. UTF-8 mode doesn't solve that, it just changes the default. It seems as though most of those commenting in the other thread don't actually use Python on Windows. I do, and I can say it's a royal pain to have to write open(path, encoding='utf-8') all the time. If you could write open(path, 'r', 'utf-8'), that would be slightly better, but the third parameter is buffering, not encoding, and open(path, 'r', -1, 'utf-8') is not very readable. UTF-8 mode is somehow worse, because you now have to decide between writing open(path), and having your script be incompatible with non-UTF-8 Windows installations, or writing open(path, encoding='utf-8'), making your script more compatible but making UTF-8 mode pointless. There's a constant temptation to sacrifice portability for convenience - a temptation that Unix users are familiar with, since they omit encoding='utf-8' all the time. My proposal is to add a couple of single-character options to open()'s mode parameter. 'b' and 't' already exist, and the encoding parameter essentially selects subcategories of 't', but it's annoyingly verbose and so people often omit it. If '8' was equivalent to specifying encoding='UTF-8', and 'L' was equivalent to specifying encoding=(the real locale encoding, ignoring UTF-8 mode), that would go a long way toward making open more convenient in the common cases on Windows, and I bet it would encourage at least some of those developing on Unixy platforms to write more portable code also. For other encodings, you can still use 't' (or '') and the encoding parameter. Note that I am not suggesting that 'L' be equivalent to PEP 597's encoding='locale', because that's specified to behave the same as encoding=None, except that it suppresses the warning. I think that's a terrible idea, because it means that open's behavior still depends on the global UTF-8 mode even if you specify the encoding explicitly. This is really a criticism of PEP 597 and not a part of this proposal as such. I think UTF-8 mode was a bad idea (just like a global "binary mode" that interpreted every mode='r' as mode='rb' would have been), and it should be ignored wherever possible. In particular, encoding='locale' should ignore UTF-8 mode. Then 'L' could and should mean encoding='locale'. Obviously the names '8' and 'L' are debatable. 'L' could be argued to be unnecessary if there's a simple way to achieve the same thing with the encoding parameter (which currently there isn't). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/UBGYIGHCJF2MGI6FISBJXQ7SMOQHODQ5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.
On Wed, Jan 27, 2021 at 11:36 PM Inada Naoki wrote: > * UnicodeDecodeError is raised when trying to open a text file written in > UTF-8, such as JSON. > * UnicodeEncodeError is raised when trying to save text data retrieved > from the web, etc. > * User run `pip install` and `setup.py` reads README.md or LICENSE file > written in UTF-8 without `encoding="UTF-8"` > > Users can use UTF-8 mode to solve these problems. > They can use it to solve *those* problems, but probably at the cost of creating different problems. There's a selection bias here, because you aren't seeing cases where a script worked because the default encoding was the correct one. If you switch a lot of ordinary users (not power users who already use it) to UTF-8 mode, I think a lot of scripts that currently work will start failing, or worse, silently producing bogus output that won't be understood by a downstream tool. I'm not convinced this wouldn't be a bigger problem than the problem you're trying to solve. * Put a checkbox in the installer? > I'm pretty Unicode-savvy, but even I would have no idea whether to check that box. Do I wish that everything was UTF-8? Yes. Do I want Python to assume that everything is UTF-8? Probably not. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CKJJKXB6JVOC5SG3OYLXUW2PEWS36Q5E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Thu, Dec 24, 2020 at 1:34 AM Barry Scott wrote:
> if sys.playform == 'win32':
> os.system('cls')
> else:
> sys.stdout.write('\x1b[2J' '\x1b[H')
>
> No concern about CLS being a trojan becuse as a builtin to CMD and
> PowerShell
> and it takes priority over cls.bat etc.
>
Because it's a shell builtin, you have to run the shell to use it. If it
wasn't a shell builtin, you could run it directly. Either way you have to
make sure you're running the right executable. I suppose finding the right
shell is easier since you can use COMSPEC, but if you don't trust the path
then I'm not sure you should trust COMSPEC.
As Eryk Sun said, cls erases the scrollback, while your non-win32 code
doesn't. The popular cmd.exe replacement TCC/LE seems to have different
default behavior for cls, erasing only the visible text by default. You can
add /c to match cmd.exe's behavior, but cls/c is an error in cmd.exe. TCC
also has cls/s which moves the visible text into the scrollback, which
might be the best behavior for Python, but if we want that we'll have to do
it in-process since we can't rely on TCC being present.
Spawning processes that run for 1ms and do a trivial thing is common in the
Unix world but it's not idiomatic in Windows. It feels like a CVE in the
making. (Even on Unix, to be honest.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/2TNKYB5GP2IW6Q4F44QOXIDLBKEKKCUB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Tue, Dec 22, 2020 at 4:46 PM Christopher Barker
wrote:
> Though, yeah shelling out to an arbitrary command on the system is a bit
> scary -- does the Python stdlib currently do that anywhere?
>
Here's what I found by grepping the sources for uses of os.system and
subprocess:
* pydoc shells out to "more" on win32 and "less" or "more" elsewhere if no
pager is configured. Incidentally, on Windows it passes the docs to the
pager via a temp file, with the comment "pipes completely broken in
Windows" - is this left over from the Win9x days? Also the temp-file code
runs the pager with os.system instead of Popen, and doesn't properly quote
it.
* ctypes.util has a ton of hacky code for finding libraries, which includes
calling out to gcc, cc, ld, objdump, /sbin/ldconfig, and /usr/ccs/bin/dump
on various platforms.
* platform.architecture() calls file and ad-hoc parses its output,
except if sys.platform in ('dos', 'win32', 'win16'). On those three
platforms it seems to be totally broken, always returning the pointer size
of the current Python process no matter what executable you pass to it.
* webbrowser looks for a bunch of specific named browsers. On Windows it'll
run any of ("firefox", "firebird", "seamonkey", "mozilla", "netscape",
"opera") from the insecure search path that starts with the current
directory.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/CERIHREMGO3EUMSJPSUIDR6RVUOZQNVI/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: built in to clear terminal
On Sun, Dec 20, 2020 at 8:06 PM Eryk Sun wrote: > The entire buffer can be scrolled out, like > the CMD shell's CLS command, or one can just scroll the buffer enough > to clear the visible window. > The Windows console model is a bit different from POSIX terminals. Instead of having a screen and a scrollback buffer, you have a buffer and a window/view into it. If you want clear() to just erase the screen and preserve "scrollback", the obvious thing to do is to clear the window/view. The problem is that if the user happens to be looking at "scrollback" when clear() is called, whatever they're looking at will be erased, not whatever was written since the last clear(). I wonder if the best approach wouldn't be to set the top-of-screen line on the first call of clear() (perhaps to one more than the current cursor line) and then use that line on all subsequent calls regardless of the current window or cursor position. That way scrollback would be safe both from accidental erasure and from pollution by programs that use clear() as a poor man's curses by erasing and redrawing everything whenever anything changes. I don't think there would need to be any interface to clear the first-line cache, since the whole thing is just a cheap hack anyway. I think it'd be a good idea to include some sort of proper Windows console interface in the standard library. It's only fair since other platforms already have curses. Then os.clear() could be implemented on top of it, and if you didn't like its behavior you could import the real library and do what you want. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CVQLH57SQYYVEONCZ2NEOZF5Z4JJOCS7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Standalone bool?
On Mon, Dec 21, 2020 at 9:25 AM Christopher Barker wrote: > [Mathew Elman wrote:] > >> Surely what you're looking for is some kind of typed hash table? > > > Maybe, maybe not. My impression is that the Typed hash table is a kluge > to get around this one issue. > For what it's worth, functools.lru_cache has a "typed" option that effectively adds the type()s of the arguments to the cache key. It was added by https://bugs.python.org/issue13227 . The only application of it in that patch was to re's compiled pattern cache, which needs to be typed to avoid spurious warnings or errors with -b or -bb, I suppose. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LJZX66JTCVWJHDMX272Z4KPCK7RQEIRO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Making the for statement work better with nested functions
On Wed, Nov 18, 2020 at 9:54 PM Greg Ewing wrote: > Essentially the idea is this: If the loop body contains a def or lambda > that captures the loop variable, then on each iteration, a new binding > is created, instead of changing an existing binding. > > Note that this is *not* the same as introducing a new scope. All the > bindings share the same name, and from the perspective of code outside > the nested function, nothing changes -- the name is still bound to > the most recent value, and can still be accessed after the loop has > finished. > The problem I see is that nested functions referring to the same variable can be defined outside loops as well. If you define a helper before the loop that refers to the loop variable, it won't work inside the loop or ever again after the loop has terminated. Even if the helper has nothing to do with the loop and just uses the same generic variable name like "item", it'll break. It's fairly common to reuse the same identifier for different purposes inside functions. As much as I hate the current behavior, I feel like the proposed behavior is too confusing and weird. Another problem is that the capturing behavior would depend on whether the variable is in a function or module or class scope, since cells only exist in function scopes. Also, "while x := get_next_thing():" loops wouldn't behave consistently with for loops. for new x in some_iterator: >... > I don't think special syntax would help. I've been bitten by this problem fairly often, and 100% of the time I've been able to solve it by adding x=x to the argument list to capture by value. I can't imagine I'd ever want to capture a loop variable by reference, especially if the cell being referenced no longer has a name outside of the nested function when the nested function is called. I get in trouble because I forget to write x=x where it's necessary, but if I forget to write x=x then I'd forget to write "new x" too. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VKHB722PLUMZJPCMHEM2MB4HBEDI463Y/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Use __bytes__ to access buffer-protocol from "user-land"
I don't think __bytes__ is necessarily a bad idea, but I want to point out a couple of things you may be unaware of. First, slicing a memoryview object creates a subview, so you can wrap your mmap object in a memoryview and then create slices for each partition, cluster run, etc. without wasting any memory (but not for fragmented files). Second, your iterator example works in Python as it stands if you substitute __iter__ for __bytes__ and writelines for write. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/BM7KEUPKKU2DM7OP7VDSWJJJKHDJBQT3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Inconsistency in splitting strings
On Tue, Oct 20, 2020 at 8:57 AM Antal Gábor wrote: > My idea is to return a list with an empty string in all cases mentioned > above. > This will never be fixed in 3.x, but if it's fixed in The Version That Must Not Be Named, my preference would be that they all return [] because then it's easy to write "or ['']" if want the other behavior, while it's much more of a pain to fix the other way around. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/SRGPDWIBXPZ4WZLFTW2ADESPAG4B56ZQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: A new suggestion for Python
On Wed, Sep 30, 2020 at 1:43 PM David Mertz wrote:
> Fluent programming is uncommon in Python, and hence few methods return a
> call of the same or similar type.
>
I think that if you include built-in operators as (shorthand for) method
calls, and you count the number of occurrences in typical Python programs
rather than the number of distinct species, it's common.
I don't know if I necessarily support this idea, but it makes sense as a
generalization of the existing augmented-assignment syntax to operations
that are conceptually similar, but are written with method-call syntax
because Python lacks operators for them.
The fact that methods are inconsistent about mutating in place versus
returning a new object, and it's sometimes hard to guess from the name
which will happen, is unfortunate but I don't think this proposal makes the
problem any worse.
Here are a few cases where the syntax would be useful for values other than
strings:
named_tuple .= _replace(...)
pil_image .= crop(...)
numpy_array .= transpose()
node .= get_{parent,left_child,right_child}()
mutable_thing_from_caller .= copy()
What I dislike about this syntax is that it makes no grammatical sense
since . isn't a binary operator and the thing on the right isn't an
expression.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/U52S44ZO7SQCWWUJIHQE7YKQWBLO4FYE/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 637 and keyword only subscripts
On Sun, Sep 27, 2020 at 2:18 PM MRAB wrote: > Consider, for example, the use-case of a function that has an optional > parameter and you want a way to know whether an argument has been > provided but None is a valid value that could be passed in. > > Having a singleton such as Missing would be helpful there. > The trouble is that's what None was supposed to be for. Over time, Missing would presumably fall victim to the same fate that befell None. You'd also probably have to extend PEP 505 to support Missing-aware operators. Maybe a singleton that supported no useful operations, not even __eq__ or __bool__, would be sufficiently inconvenient that it would only be used for defaults and "is" tests for said defaults. On Sun, Sep 27, 2020 at 10:52 PM Chris Angelico wrote: > English treats 1 as special and 0 as the same as other numbers when it > comes to singulars and plurals. [...] What do > other languages do in this way? > You were asking about natural languages but perhaps it's worth mentioning that Haskell has tuples of length 0, 2, 3, ..., but no tuples of length 1. Tuples are meant for putting n values where 1 value is expected, and when n=1 you just put the value there. On Mon, Sep 28, 2020 at 11:47 AM Stefano Borini wrote: > Also, it would give different behavior between d[x] and d[x, **kw], > which in my opinion should be a fully degenerate case. > On the other hand, it would make d[x,] and d[x, **kw] consistent, which they also ought to be. What a mess. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/XQ47DN75KKH5LECSBCGSCH5WVLKMYDIS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 637 and keyword only subscripts
I don't understand the problem here.
d[p=q] --> d.__{get,set,del}item__((), ..., p=q)
d[1, p=q] --> d.__{get,set,del}item__((1), ..., p=q)
d[1, 2, p=q] --> d.__{get,set,del}item__((1, 2), ..., p=q)
d[1, 2, 3, p=q] --> d.__{get,set,del}item__((1, 2, 3), ..., p=q)
d[1, 2, ..., n, p=q] --> d.__{get,set,del}item__((1, 2, ..., n), ...,
p=q)
Now obviously the n=1 case is a wart. But the n=0 case isn't a wart. It's
just like n=2, n=3, etc.
You can't tell the difference between a single tuple argument and n
arguments for any n≠1. As far as I can tell the problem when n=0 is no more
or less serious than the problem when n≥2. I don't see the point of adding
another special case to the spec when it doesn't even solve the general
problem.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/46F3A2IZHWRL2XBQJAQ4Q6PFYEICXHMT/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: PEP 637 - support for indexing with keyword arguments (Was: Re: PEP 9999 (provisional): ...)
On Fri, Sep 25, 2020 at 8:26 PM David Mertz wrote: > E.g. we would have: > > newthing[(), foo=1, bar=4:5] == newthing[foo=1, bar=4:5] > Right, but we also have newthing[(2, 3), foo=1, bar=4:5] == newthing[2, 3, foo=1, bar=4:5] which seems exactly analogous. A disambiguation scheme that worked for every n might be worth it, but one that only works for n=0 and makes that case less consistent with n>=2 doesn't seem worth it to me. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RDI6OOSSNCNIMBSVWLE52NRWHIR52IOT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add the brotli & zstandard compression algorithms as modules
On Wed, Sep 23, 2020 at 3:10 AM David Mertz wrote: > On Tue, Sep 22, 2020 at 11:55 PM Paul Moore wrote: > >> The point of this request is that Python's packaging infrastructure is >> looking at what compression we use for wheels - the current >> compression is suboptimal for huge binaries like tensorflow. > > > There are definitely some intermediate compression levels where both > brotli and zstd are significantly faster [than lzma], but not at the higher > levels where lzma does as well or better. > I'd assume that only decompression speed matters for packages, and on that metric both brotli and zstd beat lzma by a mile regardless of the compression level. But I think that lzma gets exceptionally good ratios on x86/x64 machine code. Even after all these years it seems to be the state of the art for "best ratio that isn't painfully slow to decompress". On Wed, Sep 23, 2020 at 3:10 AM David Mertz wrote: > On Tue, Sep 22, 2020 at 11:55 PM Paul Moore wrote: > >> The point of this request is that Python's packaging infrastructure is >> looking at what compression we use for wheels - the current >> compression is suboptimal for huge binaries like tensorflow. Packaging >> is in a unique situation, because it *cannot* use external libraries > > > It's hard to see where packaging would have any advantage with brotli or > zstd over lzma. XZ is more widely used, and package size seems to dominate > speed. There are definitely some intermediate compression levels where > both brotli and zstd are significantly faster, but not at the higher levels > where lzma does as well or better. > > Is there a concrete need here, or just an abstract point that compression > of packages shouldn't be outside the stdlib? > > Honestly, if you really want compression size over everything else, PPM is > going to beat the LZ based approaches. But being ungodly slow and using > tons of memory. > > -- > The dead increasingly dominate and strangle both the living and the > not-yet born. Vampiric capital and undead corporate persons abuse > the lives and control the thoughts of homo faber. Ideas, once born, > become abortifacients against new conceptions. > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/SC3SQKFK4GHASRNQJXFYDMNRB25P7SJ4/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IIRUPDNBCPIGUM5USU7WSWGW5AMBTRMB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: f-strings as assignment targets
This is a terrible idea. No one should ever be trying to extract data from strings that are obviously meant for human consumption, like "It is 11:45 PM". I'll grant you that it's sometimes necessary. But it needs to be approached very carefully. You need to use a --porcelain flag if available, you need to check that the actual output format matches what you expect, and you definitely need to be able to specify where substring matching stops in more ways than "int or float or string". The last thing Python needs is a built-in language feature that makes hacking together these sorts of parsers seem easy and fun. Especially when it has no way to control backtracking or to specify any but a few trivial restrictions on what can end up in the output variables. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FRBP7D7FHCXG4UAAELMGR36NEI5UM7BO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Accelerate zlib and gzip libraries with Intel's Storage Acceleration Libraries
On Mon, Aug 17, 2020 at 8:02 AM Steven D'Aprano wrote: > Perhaps I have misunderstood, but isn't this a pure implementation > change, with no user visible API changes and backward compatible output? > According to the documentation [1], it only supports compression levels 0-3. They're supposed to be comparable in ratio to zlib's levels 0-3. I found benchmarks [2] of an older version that only had compression level 1, which shows its ratio being quite a bit worse than zlib's level 1, but maybe they've improved it. The library interface seems similar, but it isn't drop-in compatible. It doesn't appear to have equivalents of inflateCopy and deflateCopy, which are exposed by Python's standard binding. There may be other missing features that I didn't notice. The streams it produces are of course standard-compliant, and decompression works with any standard-compliant stream, and is probably always faster than zlib. [1] https://01.org/sites/default/files/documentation/isa-l_api_2.28.0.pdf [2] https://ci.spdk.io/download/events/2018-summit-prc/08_Liu_Xiaodong_&_Hui_Chunyang_ISA-L_Update_and_Usercase_Sharing_SPDK_Summit_2018_China.pdf ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GLI3YMMXVWNADGIEVKRCM5KYJSR4INW4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make start, stop, step properties on islice
On Wed, Aug 12, 2020 at 8:49 AM David Mertz wrote: > The start/stop/step sound like they might be nice. But that wouldn't give > you a length, since you never know when an iterator will be exhausted. I > feel like `len(islice(it, 1, 1_000_000))` telling you the "maximum possible > length" is more a danger than a help. > I think islice should implement __length_hint__, though. As of 3.8.5 it doesn't. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KSGNEU7FMZ7LV6Z6HI7TRWBBDEUC6VXX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add builtin function for min(max())
On Fri, Jul 3, 2020 at 5:07 PM Steven D'Aprano wrote:
> As I recall, there was some positive support but it ran out of steam
> because nobody could agree on how to handle NANs even though the
> IEEE-754 standard tells us how to handle them *wink*
>
> See my responses at the time re NANs here:
>
> https://mail.python.org/pipermail/python-ideas/2016-August/041439.html
>
> https://mail.python.org/pipermail/python-ideas/2016-August/041400.html
>
> https://mail.python.org/pipermail/python-ideas/2016-August/041396.html
>
> Bottom line is that passing a NAN as the lower or upper bound should
> treat it as equivalent to "unbounded", that is, equivalent to ±∞.
That's not what the standard says. It's sorta connected to a personal
opinion of Kahan's expressed in some work-in-progress lecture notes
that you linked in the last message:
https://people.eecs.berkeley.edu/~wkahan/ieee754status/IEEE754.PDF
What he says there (on page 9) is
>Some familiar functions have yet to be defined for NaN . For instance max{x,
>y} should deliver the same result as max{y, x} but almost no implementations
>do that when x is NaN . There are good reasons to define max{NaN, 5} := max{5,
>NaN} := 5 though many would disagree.
It's clear that he's not referring to standard behavior here and I'm
not convinced that even he believes very strongly that min and max
should behave that way.
NaN means "there may be a correct answer but I don't know what it is."
For example, evaluating (x**2+3*x+1)/(x+2) at x = -2 yields NaN. The
correct answer to the problem that yielded this formula is probably
-1, but because of the way floating point hardware works, it has no
way of figuring that out. Likewise, the final result of a computation
involving the square root of a negative real may be well defined, and
may even be real, but the hardware can't compute it, so it "computes"
NaN instead.
It's definitely true that if plugging in any finite or infinite number
whatsoever in place of a NaN will yield the same result, then that
should be the result when you plug in a NaN. For example, clamp(x,
NaN, x) should be x for every x (even NaN), and clamp(y, NaN, x) where
y > x should be a ValueError (or however invalid bounds are treated).
But, e.g., clamp(m, x, M) where m < x could yield any value between m
and x, or a ValueError, depending on the value of M. So, if M is NaN,
there is no way to know what the correct answer should be. Therefore
(in my opinion) it should return NaN.
There's a case for making clamp(m, x, NaN) where m >= x return m
rather than NaN since there's no other *value* it could be (it could
be an exception though).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/PHH7EYAWJBKJCCMPU4KCROKDC3BYACWD/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: For quicker execution, don't refcount objects that can't be deleted
There isn't really any contention for these memory locations in
CPython as it stands because only one interpreter thread can run at a
time. The only time a cache handoff is needed is during a thread
switch when the new thread is scheduled on a different core, which is
pretty rare (at CPU timescales). Adding checks to every incref/decref
would probably cost more time than it would save.
Something that might help performance a bit, and wouldn't hurt it,
would be to drop explicit calls of
Py_{INC,DEC}REF(Py_{None,False,True,...}), such as the ones in
Py_RETURN_{NONE,FALSE,TRUE}, making these objects' refcounts into
freefloating meaningless values. The refcounts would be initially set
to a value far from zero, and on the rare occasions that they hit
zero, the dealloc functions would just set them back to the initial
value. Whether this would save enough time to be worth it, I don't
know.
(To avoid signed wraparound undefined behavior, you'd have to either
change the refcount type from ssize_t to size_t, or else keep the
DECREF calls and set the initial value to something like
PY_SSIZE_T_MAX/2.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/76JRO673TWG7WNPWPD76ZCRLA4PM6EJA/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Adding a built-in data structure with binary search tree semantics
On Mon, Mar 16, 2020 at 12:57 AM Andrew Barnert wrote: > Even if the extra indirection overhead turns out not to be an issue, just > from the added complexity (to every possible implementation) it seems like it > would be a bad idea to make that a requirement. The only change needed to support O(1) copy is making modified copies of nodes instead of mutating them in place. The algorithms are otherwise the same. That said, it's possible that blist does something that's faster but more complicated. I haven't looked. > > pyrsistent has O(1) copying and is maintained but it's terribly slow and > > the interface is nonstandard (Haskellish instead of Pythonic). > > It’s also neither sorted nor mutable, which means it’s pretty far from what > you’re looking for here. The distinction between mutability and immutability is an illusion. You can convert a blist-style data structure into a pyrsistent-style data structure with a wrapper that takes an O(1) copy of its internal blist object, mutates it, wraps it, and returns it. You can convert a pyrsistent-style data structure into a blist-style data structure with a wrapper that maintains an internal pyrsistent object and overwrites it with the modified copy after each operation. The complexity of the operations is unaffected and the underlying implementations can be identical. The key to this equivalence is the O(1) copying of the mutable version. Pyrsistent could switch to a blist backend but not to a sortedcontainers backend unless sortedcontainers added that operation. You're right that blist and pyrsistent don't have the same functionality as they stand. Pyrsistent's PMap and PSet are sorted but don't allow duplicates. The equivalent (modulo interface translation) of blist.sortedlist would be pyrsistent.PMultiSet. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FYK4IJKS5ZGVMKFDFNYVQLDM4N6RLP4D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Adding a built-in data structure with binary search tree semantics
On Sun, Mar 15, 2020 at 9:41 PM Andrew Barnert via Python-ideas wrote: > Do you really want to require “binary”? I don't think so; they never talked about binary trees, only "binary search tree semantics." It could alternately be called autobalanced tree semantics or something. >Sorted Containers, that implemented the consensus API (largely based on >blist’s). One feature blist has that sortedcontainers seems to lack is O(1) copies. It would be nice to have that in a standard library. But maybe I shouldn't press for it as it might impose a significant constant-factor overhead on other operations. blist unfortunately seems to be at least slightly bitrotted at this point: iterating over a sortedset fails because of a method that assumes that StopIteration will escape from a generator. pyrsistent has O(1) copying and is maintained but it's terribly slow and the interface is nonstandard (Haskellish instead of Pythonic). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7KED5MLUIJ6FKXRTZ2ZXJUZBGZEDD5EC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
On Thu, Mar 12, 2020 at 2:32 PM Marco Sulla via Python-ideas wrote: > > On Thu, 12 Mar 2020 at 21:22, Chris Angelico wrote: > > They actually ARE already discarded > > 0O > You're right. So *how* can juliantaylor said he measured a speedup of > 2x for large ndarrays? I think that currently you have roughly the following situation with abcd = a+b+c+d: temp_ab = a+b (with a and b having refcounts of 2 or more) temp_abc = temp_ab+c (with temp_ab having a refcount of 1, and c a refcount of 2 or more) del temp_ab abcd = temp_abc+d (with temp_abc having a refcount of 1, and d a refcount of 2 or more) del temp_abc The temp_ab+c and temp_abc+d computations can reuse the temporary arrays, but only if you walk the stack to verify that there are no hidden references, which is slow, non-portable and "terrifying" to quote one comment on numpy bug #7997. If you can't reuse the arrays, then you have to allocate a third hunk of RAM for the result, which is slower. If there were better information about "temporariness", then the temporaries could be reused without the expensive and complicated stack walk, which would allow this optimization to benefit smaller arrays. If there were del expressions, then (del a)+b+c+d would allow reuse in the a+b expression as well - but only if you walked the stack or had temporariness information. So I think that del expressions are orthogonal to the stack tracing hack in terms of optimization potential. C++11 introduced temporariness information through the type system with rvalue references. I don't know how you'd do it in Python. Of course, you can get the same effect in Python right now with a += b; a += c; a += d; abcd = a; del a ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OVQRIMXSH6D5AWIVPMBD2UH3QECJT5VS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Make `del x` an expression evaluating to `x`
On Thu, Mar 12, 2020 at 10:43 AM Andrew Barnert via Python-ideas wrote: > Also, you need to think through what happens with a del expression inside all > kinds of contexts that currently can’t have del—eval, lambdas and > comprehensions (can I del something from the outer scope?), etc.; just > defining what it does in the specific case of deleting the loop variable > isn’t sufficient. del VAR currently follows the same scoping rules as VAR = EXPR, so I think that a hypothetical (del VAR) expression could and should follow the same scoping rules as (VAR := EXPR). del VAR essentially is an assignment, but of a "null pointer" instead of an object reference. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LX255J27SYFBS5A5K2PT7DHCUI33EN3E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: More appropriate behavior for the NotImplemented object
On Wed, Mar 11, 2020 at 10:09 AM Serhiy Storchaka wrote:
> There is a precedence (although not in the stdlib): NumPy array with
> dtype=bool.
>
> >>> import numpy
> >>> bool(numpy.array([False, False]))
> Traceback (most recent call last):
> File "", line 1, in
> ValueError: The truth value of an array with more than one element is
> ambiguous. Use a.any() or a.all()
There are some cases in the standard library, e.g.
>>> import dbm.dumb
>>> db = dbm.dumb.open('foo')
>>> db.close()
>>> not db
Traceback (most recent call last):
File "", line 1, in
OSError: DBM object has already been closed
mmap.mmap also does this. Maybe it's a bug, or maybe the objects are
considered to be invalid and no longer fit for any use.
-- Ben
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/XOWADGIUQERVNMMUPQPH6YLKXGRMGJSW/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: New syntax for decorators that are compatible with both normal and async functions
On Mon, Feb 10, 2020 at 9:50 AM Andrew Barnert via Python-ideas wrote: > It’s a well-known problem that async is “contagious”: [...] > > But C# and every other language that’s borrowed the idea has the same > problem, and as far as I know, nobody’s thought of a good answer yet. Threads don't have that problem: you can use non-thread-aware code with callbacks in your threaded program if you do your own locking. Haskell (GHC) doesn't have that problem: it has fibers that use a programming interface like C#/Python threads, but they're multiplexed by user-mode code within a single OS thread. 16-bit Windows didn't have that problem. Stackless and greenlet don't have that problem. It's a problem that can be solved by just doing the obvious thing, the thing that Python already did with threads: don't define a novel syntax for coroutines, but instead use the syntax that already existed. Async/await syntax is a static type system with two types, may-yield and will-not-yield. There's no provision for writing generic/polymorphic code over those types, so you have to write everything twice. Like any static type system it has some benefits, but I don't think it's worth the cost, especially in Python, which has always eschewed mandatory static typing. I don't know how to fix Python now that it's gone so thoroughly down this path (starting with the yield keyword 18 years ago). But it's not a problem that ever needed to exist. Coroutines aren't that hard. -- Ben ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/ZDQIZ5ORHLOEQO7OD2FTZNYPUSXMUTUU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: foo.setParseAction(lambda a, b, c: raise FuckPython(":("))
On Sun, Oct 27, 2019 at 4:17 PM Andrew Barnert wrote: > On Oct 27, 2019, at 15:07, Ben Rudiak-Gould wrote: > >from __future__ import raise_function > That’s a pretty big breaking change. I agree, it's a bad idea. It'd have to be Python 4, like print_function was Python 3. > def throw(e): raise e The problem with this is that it adds an extra line to the traceback, so you have to choose between the convenience of using it and the convenience of more readable tracebacks. Writing it in C would fix that, but it's pretty complicated (I think I could copy the logic from do_raise in ceval.c, but I'm not completely sure, and it's almost 100 lines long), and of course not portable. I wouldn't mind having a sys.throw or something that's guaranteed to do it correctly. Actually, it'd be nice to have some general way to write trivial wrappers in Python without uglifying tracebacks, but that's a different discussion. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/7CU6GLIMZIHGESNTBU5OMCYMOJO42B7C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: foo.setParseAction(lambda a, b, c: raise FuckPython(":("))
throw is an expression, not a statement, in C++. I see no reason raise couldn't be an expression in Python. It doesn't even need a special rule in the grammar: from __future__ import raise_function foo.setParseAction(lambda a, b, c: raise(MumbleMumble())) Looking up and calling the raise function would add overhead to every explicitly raised exception, but exceptions are so expensive anyway that I think it wouldn't be noticeable. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2IK43LXP5DFHDL7UGVPXBFP7E6HQPVWG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Python 4000: Have stringlike objects provide sequence views rather than being sequences
Since this is Python 4000, where everything's made up and the points don't matter... I think there shouldn't be a char type, and also strings shouldn't be iterable, or indexable by integers, or anything else that makes them appear to be tuples of code points. Nothing good can come of decomposing strings into Unicode code points. The code point abstraction is practically as low level as the internal byte encoding of the strings. Only lexing libraries should look at strings at that level, and you should use a well written and tested lexing library, not a hacky hand-coded lexer. Someone in this thread mentioned that they'd used ' '.join on a string in production code. Was the intent really to put a space between every pair of code points of an arbitrary string? Or did they know that only certain code points would appear in that string? A convenient way of splitting strings into more meaningful character units would make the actual intent clear in the code, and it would allow for runtime testing of the programmer's assumptions. Explicit access to code points should be ugly – s.__codepoints__, maybe. And that should be a sequence of integers, not strings like "́". >it’s probably worth at least considering making UTF-8 strings first-class >objects. They can’t be randomly accessed, They can be randomly accessed by abstract indices: objects that look similar to ints from C code, but that have no extractable integer value in Python code, so that they're independent of the underlying string representation. They can't be randomly accessed by code point index, but there's no reason you should ever want to randomly access a string by a code point index. It's a completely meaningless operation. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MDX4LXOWJQ2DXPIG27DJ3TVETSUSMSVW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add Subscriptable ABC
On Tue, Oct 1, 2019 at 9:26 AM Andrew Barnert wrote: > On Sep 30, 2019, at 23:46, Ben Rudiak-Gould wrote: > > ABC membership is a subtype relationship in some sense, and ordinary Python > > subclassing is a subtype relationship in some sense, but they aren't quite > > the same sense, > > But in this case, they actually match. Hashable is correctly checking for > structural subtyping, and the problem is that list isn’t actually a proper > subtype of object, not that object isn’t a proper subtype of Hashable. If you take the perspective that the ABC notion of subtyping is the correct one, then list isn't a proper subtype of object. I wasn't taking the perspective that either one is the correct one. I think they're probably both okay in isolation. I can understand the ordinary subclass relation in Python as the partial order generated by the inheritance dag, which is a perfectly good notion of subtyping. (Yes a subclass instance might not *actually* work where a superclass instance is expected, but that's true in any OOP language. A 1234 may not work where an int instance is expected either. Perfect substitutability is not achievable, but at least there's a partial ordering of classes.) I think I can understand the ABC subclass test as a subset relation, which is also a perfectly good notion of subtyping. The problem is that when you combine them, you have neither of those things. I'm not even certain how they're being combined (is it just the union of the graphs of the two relations?) but the result has none of the properties that you'd expect a subtype relation to have, and that the two subtype relations do have in isolation. > >>>> class A(collections.abc.Hashable): > >... __hash__ = None > >... > >>>> issubclass(A, collections.abc.Hashable) > >True > > This one is a direct consequence of the fact that you can lie to ABCs—if you > inherit from an ABC you are treated as a subtype even if you don’t qualify, > and the check isn’t perfect. You are explicitly, and obviously, lying to the > system here. My problem is not that I can't justify the current behavior, it's that if it behaved differently, I could justify that behavior too. I feel like you're using CPython as an oracle of what ABCs should do, and that if issubclass had returned False in this example, you would have been ready with an explanation for that too - namely that I broke the subtyping relation by deleting __hash__, the same explanation you used earlier in the case where it did return False. What *should* it mean to inherit from an ABC? The module encourages you to use them as mixins, but maybe that isn't the intended meaning, just a side hack? Is the primary meaning to do the equivalent of registering the class with the predicate? I was worried that someone would complain about my A not making sense, and thought about using a more complicated example: class A(Hashable): def __hash__(self): return 4 class B(A): __hash__ = None issubclass(B, Hashable) # True So empirically, inheriting from Hashable registers not only that class but all subclasses of it that may later be defined with the predicate. Is that the intended behavior, or is it an accidental side effect of combining two different notions of subclassing in a single test? You could end up with a situation where you'd have to choose between using an ABC as a mixin and living with potentially incorrect ABC predicate tests in subclasses, or implementing the methods yourself (in the same way the mixin would have) to get the correct predicate behavior. Hashable isn't useful as a mixin and I think none of the other ABCs test deletable properties, but that doesn't seem to be a design principle, just a coincidence. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DTELBD2WUJ2V6PKSZZQUY5LYK3TCD2DG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Add Subscriptable ABC
On Mon, Sep 30, 2019 at 10:08 AM Andrew Barnert via Python-ideas wrote: > Also, what we’re checking for really is subtyping. Is it? Subtyping in type theory satisfies some axioms, one of which is transitivity. The addition of the ABCs broke transitivity: >>> issubclass(list, object) True >>> issubclass(object, collections.abc.Hashable) True >>> issubclass(list, collections.abc.Hashable) False ABC membership is a subtype relationship in some sense, and ordinary Python subclassing is a subtype relationship in some sense, but they aren't quite the same sense, and merging them creates an odd hybrid system in which I'm no longer sure which subclass relationships should hold, let alone which do. For example: >>> class A(collections.abc.Hashable): ... __hash__ = None ... >>> issubclass(A, collections.abc.Hashable) True >>> hash(A()) Traceback (most recent call last): File "", line 1, in TypeError: unhashable type: 'A' I didn't know what the issubclass call would return before I keyed in the example, and I can't decide what it should return. In contrast, I have no trouble deciding that the equivalent test implemented as a predicate ought to return False, since instances of A are in fact not hashable. I don't know how predicates would work in type annotations. And the ship has sailed. But I do think there's something wrong with the ABCs. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/G32AS3U67GTCLGPV23G2NFKBFPU6WMLM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: A proposal (and implementation) to add assignment and LOAD overloading (was: (Re: Re: Overloading assignment concrete proposal (Re: Re: Operator as first class citizens -- like in s
On Tue, Jun 25, 2019 at 2:11 PM nate lust wrote: >if an instance is bound to a variable name, any attempts to rebind that name >will result in a call to the __setself__ (name negotiable) of the instance >already bound to that name. I am very, very strongly opposed to this. It would mean that I couldn't trust variables any more. I want to be able to write for x in items: without worrying about whether one of the items might "stick" to x and infect every subsequent iteration of the loop, and any later loop in the same function that also uses x as a loop variable. I want to be able to clarify code by assigning a subexpression to a local variable with a descriptive name, without worrying about whether that's going to change the meaning of the code. I need local variables to be easy to reason about, because Python requires you to use them so much. The thought of the additional cognitive burden that the mere presence of this feature in the language would create, in practically everything I write, scares me. > On first read, that may be surprising, but it extends a behavior pattern that > already exists for things like properties (and generically descriptors) to > object instances themselves. Similar caveats and behaviors will apply here as > well. It seems very different to me. Magic attributes are declared in the class. Assignments to that attribute of an instance then go through the special code in the class. Your proposal doesn't have that split. Your proposal would be as if assigning a special value to an ordinary attribute permanently changed that attribute's behavior for that instance only, in a way controlled by the value, not by the class or even the instance. I would be fine with a proposal to declare special variables whose loading and storing behavior is controlled by Python code. If a declaration like metavariable foo = obj in a scope caused the compiler to generate calls to obj.__get__, obj.__set__ and obj.__delete__ instead of the usual LOAD_*, STORE_*, DELETE_* instructions for all mentions of foo in that scope, I would be fine with that. It would be more similar to descriptor attributes, it would have no runtime overhead if not used, and most importantly, I wouldn't have to learn about it if I didn't want to use it. Also you probably wouldn't need to invent new special method names for it, the existing ones would work. > * Variables which keep track of their assignment history, with ability to > rollback (possibly useful with try except blocks) > * Variables which write out their value to disk when assigned to > * An implementation of context variables using only this new framework (does > not implement tokens, but could be added) > * const variables that can be used to protect module level 'constants' > * Instance properties (reproduced below) that allow dynamically adding > properties > * An implementation of templated expression, to defer the addition of many > arrays to a single for loop, saving possibly expensive python iterations. I don't understand all of these but the ones I do understand seem like they only require some way to magic-ify the variable before using it. The part of your proposal that I strongly oppose is overloading the ordinary assignment syntax for this. Use a special declaration analogous to global/nonlocal and I think it's fine. -- Ben ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MW2CFAQ6BKS7TIRHW6SJCUOR4Y2ECQNN/ Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Simpler thread synchronization using "Sticky Condition"
On Tue, Mar 26, 2019 at 2:40 AM Richard Whitehead wrote: > Using Python’s Condition class is error-prone and difficult. After looking at your example, I think the problem is just that you aren't using condition variables the way they're intended to be used. To write correct condition-variable code, first write code that contains polling loops like with lock: ... while not some_boolean_condition(): lock.release() time.sleep(0) lock.acquire() ... Then (1) for each boolean condition that you polled on, introduce a corresponding condition variable; (2) whenever you do something that causes the truth of that condition to flip from false to true, notify the condition variable; and (3) replace the polling loop with while not some_boolean_condition(): cvar_associated_with_that_condition.wait() You should only call Condition.wait() inside a while loop that tests the associated condition (or use wait_for()). Implementations of condition variables don't necessarily guarantee that the condition is true when wait() returns, even if you do everything else correctly. See https://en.wikipedia.org/wiki/Spurious_wakeup . If the condition variable is notified when you aren't inside wait(), you will miss the notification. That isn't a problem because as long as the boolean condition itself remains true, you will exit the while loop immediately upon testing it. Or, if the condition has become false again, you will wait and you will get the next false-to-true notification. Condition variables are not meant to substitute for polling the actual condition; they just make it more efficient. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Preallocated tuples and dicts for function calls
On Fri, Mar 8, 2019 at 6:23 PM Steven D'Aprano wrote: > A sets the argument tuple to (1, 2) > B sets the argument tuple to (2, 3) > B calls spam() > A calls spam() # Oops! I'm pretty sure the idea was to have constant tuples (1, 2) and (3, 4) in the module instead of LOAD_CONST 1/2/3/4 instructions in the bytecode. There's no setting of a hidden global variable involved. The kwargs dicts are a harder problem. I suppose they would have to be copy-on-write which would add too much complexity, or the language would have to be changed to allow/require kwargs to be a frozendict. > And then the pre- > allocated tuples and dicts would hang around forever, wasting memory. > Even if it turns out that the function never actually gets called: > > for x in sequence: > if condition(x): # always returns False! > function(...) > > the compiler will have pre-allocated the memory to call it. The bytecode for "function(...)" already hangs around forever even if it's never run. There is no need for tuple constants because you can generate LOAD_CONST a; LOAD_CONST b; ...; BUILD_TUPLE n at each usage point instead, but CPython has tuple constants, so they must have some space and/or speed benefit that was considered significant enough to be worth implementing them. It seems like args constants for function calls can be justified on similar grounds. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] The @update operator for dictionaries
On Sat, Mar 9, 2019 at 4:14 PM Greg Ewing wrote: > > A large part of the reason that common operations are written > using infix operators is that the operator symbols used are very > compact. That benefit disappears if your operator is an entire > word. I suppose people bring up Haskell too much, but it does work in Haskell. People write things like (item `notElem` list) all the time and it's readable enough. In Haskell, though, it's sugar for (notElem item list), or notElem(item, list) in Pythonish syntax. In Python, it'd in most cases be sugar for a method call, in which the method name already appears in infix position, so the benefit is less clear. Given that Python's so-called augmented assignments are really mutating operations in disguise anyway (x op= y is not equivalent to x = x op y when x is mutable), I don't see any advantage of a new assignment syntax over the existing mutating methods. I.e., instead of x @update= y, you can just write x.update(y). ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Type hints for functions with side-effects and for functions raising exceptions
> It's well documented how checked exceptions lead to bad code. Please cite a paper. I know "everyone knows" that they're bad, but "everyone knows" a lot of things. Here's a recentish proposal by Herb Sutter to add a kind of checked exception to C++: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0709r0.pdf He talks about some of the same issues I've talked about in this thread: in particular, that the farther exceptions propagate, the less likely that they can be handled in a precise way. In practice, exceptions are either handled early or they are handled generically (e.g. by printing a traceback). > Exceptions may be logically thought as of part of the type of a method, Yes, exceptions designed to be caught and handled by the caller are essentially alternate return values. Examples are KeyError for mapping lookups, FileNotFoundError for open(), etc. They are part of the method's interface whether they are encoded in a type system or not. > but that requires that the type catches every exception that may be raised > from the implementation and either handles it, or translates it to one > belonging to the type. What you're describing is strong static typing. Yes, it is a hassle. It means you have to do a lot of up-front work proving that your program will handle *every* case before the implementation will permit you to run it on *any* case. A lot of programmers don't like that. They want to write code that works for the cases they care about in the moment, and think about the other cases "later" (which in practice often means "after the product has shipped and someone files a bug report"). > It's a lot of work, it's cumbersome, and it is fragile, as the exception > handling may need to change over minor implementation changes. Yes, it's a lot of work and cumbersome. In exchange for this extra effort, you get static guarantees that make it easier to reason about the behavior of the program. > The strategy of catching only exceptions of interest and letting others pass > produces less fragile and easier to test code. It is less of a hassle to write unchecked code. I don't know if it's easier to test, but maybe. It isn't less fragile, at least not in the way I understand "fragile." Fragile code isn't code that is prone to compile-time type errors when it's changed. Fragile code is code that's prone to crashes or other broken behavior at runtime when it's changed, because of hidden constraints that weren't expressible in the type system. At least, that's what "fragile" means in "fragile base class problem." ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Type hints for functions with side-effects and for functions raising exceptions
On Wed, Feb 20, 2019 at 2:43 AM Chris Angelico wrote:
> That's because a generator function conceptually has three ways to
> provide data (yield, return, and raise), but mechanically, one of them
> is implemented over the other ("return" is "raise StopIteration with a
> value"). For other raised exceptions, this isn't a problem.
Other functions also conceptually have three ways of returning:
ordinary return with a value, a documented special return like
KeyError, and pass-through exceptions. If the pass-through exception
is KeyError, it gets conflated with the documented exceptional return,
but correct code should handle them differently. It doesn't matter
whether the syntax for the documented special return is "return x" or
"raise KeyError(x)". I've never been screwed by this as badly with
other exceptions as I was by StopIteration, but it's a general problem
with the design of exceptions.
I don't think exception specifications would solve that problem since
you probably couldn't describe the KeyError's origin in the spec
either. But that doesn't mean it isn't a problem.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Type hints for functions with side-effects and for functions raising exceptions
On Tue, Feb 19, 2019 at 3:19 PM Steven D'Aprano wrote:
> And I note that in Java, where the idea of checked exceptions
> originated, it turned out to be full of problems and a very bad idea.
What should be the default behavior of a language when the programmer
doesn't explicitly handle an error? Options include:
1. Type error (Java, ML/Haskell)
2. Ignore it (C)
3. Pass it as-is to the caller
4. "Genericize" it, e.g. wrap it in a RuntimeError, then pass to the caller
The problem with the Java approach is that people don't want to think
about how to properly handle every error, and just wrap their code in
catch (...) {} instead. I think it works much better in ML/Haskell,
though perhaps only because the average skill level of the programmers
is higher.
The problem with the C approach is that people don't want to think
about how to properly handle every error, and just call every function
in a void context.
The problem with passing exceptions as-is to the caller is that
they're very often implementation details. If you're lucky, they will
propagate to a generic catch-all somewhere which will generate a
traceback that a human may be able to use to fix the problem. If
you're unlucky, the caller wrote `return d[k].frobnicate()` inside a
try block and frobnicate's internal KeyError gets misinterpreted as a
lookup failure in d.
That problem, of an inadvertently leaked implementation detail
masquerading as a proper alternate return value, used to be a huge
issue with StopIteration, causing bugs that were very hard to track
down, until PEP 479 fixed it by translating StopIteration into
RuntimeError when it crossed an abstraction boundary.
I think converting exceptions to RuntimeErrors (keeping all original
information, but bypassing catch blocks intended for specific
exceptions) is the best option. (Well, second best after ML/Haskell.)
But to make it work you probably need to support some sort of
exception specification.
I'm rambling. I suppose my points are:
* Error handing is inherently hard, and all approaches do badly
because it's hard and programmers hate it.
* Stronger typing is only bad if you just want your code to run and
are tired of fixing it to be type-correct. The people who voluntarily
add type annotations to Python programs probably aren't those kinds of
people; they're probably much more likely than the average programmer
to want checked exceptions.
* Declaring that a function only raises Foo doesn't have to mean "it
raises Foo, and also passes exceptions from subfunctions to the caller
unchanged, no matter what they are." It could also mean "it raises
Foo, and converts other exceptions into RuntimeError." This would
actually be useful because it would mean that you could safely put
longer expressions in try blocks, instead of defensively just putting
the one method call whose KeyError you want to handle in the try:
block, and moving everything else to the else: block, as I tend to do
all the time because I'm paranoid and I was bitten several times by
that StopIteration problem.
-- Ben
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Multi-line string indentation
> On 3/31/18 5:43 PM, Steven D'Aprano wrote:
> > But we could avoid that runtime cost if the keyhole optimizer performed
> > the dedent at compile time:
> >
> > triple-quoted string literal
> > .dedent()
> >
> > could be optimized at compile-time, like other constant-folding.
There are a lot of expressions of the form constexpr.method(constexpr)
that could be safely evaluated at compile time and currently aren't.
It also occurred to me that if there were a set.frozen method
returning a frozenset, then (with compiler support) you could write
frozenset literals without new syntax.
And if dict.get and dict.__getitem__ (including the operator syntax
for the latter) made compile-time (frozen)dict literals the way "in
{...}" already makes frozenset literals, it would give you a
functional O(1) switch statement for some simple cases.
-- Ben
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Clearer communication
Note: none of the following is an endorsement of the r/python_ideas idea. I'm just responding point-by-point to what you wrote. On Fri, Feb 1, 2019 at 10:47 PM Steven D'Aprano wrote: > - I can have as many email identities as I like; I can only have one > Reddit identity at a time. Do you mean because your browser doesn't support per-window or per-tab cookie jars? I'm pretty sure there are browsers that do. (I use multiple instances of the same browser with different home directories to solve this general problem, but I think there are other solutions.) Also, many clients (including RES in a browser) support switching accounts by choosing from a drop-down list. > If I want to keep my Reddit persona seperate from my Python persona, I > need to create multiple accounts (possibly violating the terms of > service?) and remember to log out of one and into the other. It isn't a violation of the TOS and it's extremely common and people are open about it. It is a TOS violation to, for instance, up/downvote the same post/comment with two of your accounts. > - Too difficult (impossible?) to keep local human-readable copies of > either the discussion thread, or even your own posts. I agree the Reddit client situation is pretty sad compared to the email client situation, but non-browser clients do exist. You don't have to use Reddit in a browser. RES lets you save things locally, but you are still stuck viewing them in a browser. > - I have to explicitly go to the site to see what is happening, rather > than have the posts automatically arrive in my inbox. Well, you can get an RSS feed of any subreddit or comment thread and stick that in your email client. It's not perfect I agree. >From another of your messages: > Core developer Brett Cannon has taken up editing other people's comments > on github if he doesn't approve of their tone. > [...] > Github (currently) provides the full history of edits to each post. > Reddit just has a flag that shows you whether a post was edited or not. > Isn't technology wonderful? Reddit doesn't allow anyone but the original user to edit posts or comments. Moderators (ordinary users who are selected per subreddit like IRC ops) can only remove the entire text of a comment (or text post) and put "[removed]" in its place. They can also make posts no longer appear on the subreddit, but they continue to be viewable if you have the direct url. There was a scandal in which a Reddit co-founder admitted to editing someone's comment. I think he was able to do that because he had direct database access. If someone has direct database access then of course a full edit history won't help since you can bypass that along with everything else. I am definitely a fan of the distributed nature of email. However, a rogue admin of python.org or their registrar or ISP or some Internet switch could alter emails to this list without leaving any edit history. Proper authentication could solve a lot of that, but as long as we're dreaming, it's only fair to make Reddit distributed and not-for-profit too. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Vectorization [was Re: Add list.join() please]
On Sat, Feb 2, 2019 at 3:31 PM Steven D'Aprano wrote: > The comprehension version isn't awful: > > [(a*2).name.upper() for a in seq] > > but not all vectorized operations can be written as a chain of calls on > a single sequence. > If they are strictly parallel (no dot products) and you know when writing the code which variables hold vectors, then (denoting the vector variables by v1, ..., vn) you can always write [(expr with x1, ..., xn substituted for v1, ..., vn) for x1, ..., xn in zip(v1, ..., vn)] which seems not much worse than the auto-vectorized version (with or without special syntax). Haskell (GHC) has parallel list comprehension syntax ( https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#parallel-list-comprehensions) so you don't have to explicitly call zip. I wouldn't mind having that in Python but I don't know what the syntax would be. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Vectorization [was Re: Add list.join() please]
On Sat, Feb 2, 2019 at 3:23 PM Christopher Barker wrote: > performance asside, I use numpy because: > > c = np.sqrt(a**2 + b**2) > > is a heck of a lot easer to read, write, and get correct than: > > c = list(map(math.sqrt, map(lambda x, y: x + y, map(lambda x: x**2, a), > map(lambda x: x**2, b) > ))) > > or: > > [math.sqrt(x) for x in (a + b for a, b in zip((x**2 for x in a), > (x**2 for x in b) > ))] > You can also write c = [math.sqrt(x**2 + y**2) for x, y in zip(a, b)] or c = list(map(lambda x, y: math.sqrt(x**2 + y**2), a, b)) or, since math.hypot exists, c = list(map(math.hypot, a, b)) In recent Python versions you can write [*map(...)] instead of list(map(...)), which I find more readable. a_list_of_strings.strip().lower().title() > > is a lot nicer than: > > [s.title() for s in (s.lower() for s in [s.strip(s) for s in > a_list_of_strings])] > > or > > list(map(str.title, (map(str.lower, (map(str.strip, a_list_of_strings > # untested > In this case you can write [s.strip().lower().title() for s in a_list_of_strings] -- Ben ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] AMEND PEP-8 TO DISCOURAGE ALL CAPS
On Fri, Feb 1, 2019 at 11:24 AM Abe Dillon wrote: > "why not make everything default to 'final' and put 'var' or something > in-front of the few outliers?". > If it happens, it'll be another example of mainstream languages adopting ideas from functional programming. I love it when that happens, I just wish it didn't take decades. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Single line single expression try/except syntax
Aren't the arguments for accepting PEP 463 basically the same as the arguments for accepting assignment expressions? The current syntax is painfully verbose and people use inefficient and ad hoc constructions to get around it. Better to have a language feature to support the way that people actually want to write code. If Guido wrote that rejection of PEP 463 then I can't help thinking that he changed his perspective between then and PEP 572 and might have accepted PEP 463 if it had been proposed more recently. (I'm aware of the drama surrounding PEP 572, but still.) On Sun, Jan 27, 2019 at 1:04 PM Michael Selik wrote: > Any discussion of except expressions should reference PEP 463 and respond > to the arguments there. > > https://www.python.org/dev/peps/pep-0463/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Allow popping of slices
On Mon, Jun 4, 2018 at 5:11 PM, Steven D'Aprano wrote:
> class MyList(list):
> def pop(self, pos):
> if isinstance(pos, slice):
> temp = self[pos]
> del self[pos]
> return temp
> return super().pop(pos)
>
> Is that what you have in mind?
Yes. (I used almost exactly that to test my examples.)
>> def cut_deck(deck, pos):
>> deck.extend(deck.pop(slice(0, pos)))
>
> I'm not sure that's an advantage over:
>
> deck[:] = deck[pos:] + deck[:pos]
>
> but I suppose one might be faster or slower than the other. But I think
> the version with slices is much more clear.
That's fair. I didn't spend as long creating examples as I probably should've.
> It might help your argument if you show equivalent (but working) code
> that doesn't rely on popping a slice.
When I have a collection of items and I want to consume them, process
them, and produce a new collection of items, I often find myself
writing something along the lines of
items2 = []
for item in items:
...
items2.append(...)
...
items = items2
del items2
The last two statements aren't strictly necessary, but omitting them
is a bug factory in my experience; it's too easy to use the wrong
variable in subsequent code.
When the loop is simple enough I can write
items = [... for item in items]
and when it's complicated enough it probably makes sense to split it
into a separate function. But I've many times wished that I could
write
for item in items.pop_all():
...
items.append(...)
...
This proposal uses pop(slice(None)) instead, because it's a natural
extension of the existing meaning of that method.
My bfs function was a breadth-first search. The outer loop runs once
for each depth, and the inner loop once for each item at the current
depth (the frontier). The code in the Wikipedia article uses a FIFO
queue instead and has just one loop (but doesn't explicitly track the
depth).
> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/splice
>
> Do you have a link for the Perl version?
https://perldoc.perl.org/functions/splice.html
It's slightly more versatile: it essentially does
old = ARRAY[OFFSET:OFFSET+LENGTH]
ARRAY[OFFSET:OFFSET+LENGTH] = LIST
return old
The special case LIST = [] is what I'm suggesting for the pop method.
>> I think it's useful not just because it's more
>> concise, but because it's linear/reversible: it moves data rather than
>> duplicating and then destroying it, which makes it less prone to bugs.
>
> I don't see how that is possible. You still have to move the data out of
> the sequence into a new sequence before deleting it from the original.
Under the hood it copies data because that's how von Neumann machines
work, but from the Python programmer's perspective it moves it.
(Similarly, std::swap(x, y) in C++ probably does { t = x; x = y; y =
t; } internally, but it looks like a swap from the programmer's
perspective.)
>> 3. Promote del statements to expressions that return the same values
>> as the underlying __delitem__, __delattr__, etc., and make those
>> methods of built-in types return the thing that was deleted. (Or
>> introduce __popitem__, __popattr__, etc. which return a value.)
>
> I don't get how this allows us to pass slices to pop.
It doesn't; it basically makes del the new pop. It's almost pop
already, except that it doesn't return the deleted value.
I wasn't seriously proposing this, although I do like the idea. I
don't think it reaches the usefulness threshold for new syntax. Also,
del foo[:] would be slower if it had to move the deleted items into a
new list whether or not the caller had any interest in them. That's
why I suggested that del expressions call __popXXX__ instead, while
del statements continue to call __delXXX__; but now it's getting
complicated. Oh well.
-- Ben
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Add dict.append and dict.extend
On Mon, Jun 4, 2018 at 4:02 PM, Yuval Greenfield wrote:
> The proposed meanings surprised me too. My initial instinct for
> `dict.append` was that it would always succeed, much like `list.append`
> always succeeds.
Many of the methods named `append` in the standard libraries fail if
adding the item would violate a constraint of the data structure.
`list.append` is an exception because it stores uninterpreted object
references, but, e.g., bytearray().append(-1) raises ValueError. Also,
`dict.__setitem__` and `dict.update` fail if a key is unhashable,
which is another dict-specific constraint.
Regardless, I'm not too attached to those names. I want the underlying
functionality, and the names made sense to me. I'd be okay with
`unique_setitem` and `unique_update`.
On Mon, Jun 4, 2018 at 5:25 PM, Steven D'Aprano wrote:
> On Mon, Jun 04, 2018 at 02:22:29PM -0700, Ben Rudiak-Gould wrote:
>> Very often I expect that the key I'm adding to a dict isn't already in
>> it.
>
> Can you give some examples of when you want to do that? I'm having
> difficulty in thinking of any.
One example (or family of examples) is any situation where you would
have a UNIQUE constraint on an indexed column in a database. If the
values in a column should always be distinct, like the usernames in a
table of user accounts, you can declare that column UNIQUE (or PRIMARY
KEY) and any attempt to add a record with a duplicate username will
fail.
People often use Python dicts to look up objects by some property of
the object, which is similar to indexing a database column. When the
values aren't necessarily unique (like a zip code), you have to use
something like defaultdict(list) for the index, because Python doesn't
have a dictionary that supports duplicate keys like C++'s
std::multimap. When the values should be unique (like a username), the
best data type for the index is dict, but there's no method on dicts
that has the desired behavior of refusing to add a record with a
duplicate key. I think this is a frequent enough use case to deserve
standard library support.
Of course you can implement the same functionality in other ways, but
that's as true of databases as it is of Python. If SQL didn't have
UNIQUE, every client of the database would have its own code for
checking and enforcing the constraint. They'd all have different
names, and slightly different implementations. The uniqueness property
that they're supposed to guarantee would probably be documented only
in comments if at all. Some implementations would probably have bugs.
You can't offload all of your programming needs onto the database
developer, but I think UNIQUE is a useful enough feature to merit
inclusion in SQL. And that's my argument for Python as well.
Another example is keyword arguments. f(**{'a': 1}, **{'a': 2}) could
mean f(a=1) or f(a=2), but actually it's an error. I think that was a
good design decision: it's consistent with Python's general philosophy
of raising exceptions when things look dodgy, which makes it much
easier to find bugs. Compare this to JavaScript, where if you pass
four arguments to a function that expected three, the fourth is just
discarded; and if the actual incorrect argument was the first, not the
fourth, then all of the arguments will be bound to the wrong
variables, and if an argument that was supposed to be a number gets a
value of some other type as a consequence, and the function tries to
add 1 to it, it still won't fail but will produce some silly result
like "[object Object]1", which will then propagate through more of the
code, until finally you get a wrong answer or a failure in code that's
unrelated to the actually erroneous code.
I'm thankful that Python doesn't do that, and I wish it didn't do it
even more than it already doesn't. Methods that raise an exception on
duplicated keys, instead of silently discarding the old or new value,
are an example of the sort of fail-safe operations that I'd like to
see more of.
For overridable options with defaults, `__setitem__` and `update` do
the right thing - I certainly don't think they're useless.
> I'm sorry, I can't visualise how it would take you up to five lines to
> check and update a key. It shouldn't take more than two:
>
> if key not in d:
> d[key] = value
>
> Can you give an real-life example of the five line version?
The three lines I was thinking of were something like
if k in d:
raise KeyError(k)
d[k] = ...
The five lines were something like
d = get_mapping()
k = get_key()
if k in d:
raise KeyError(k)
d[k] = ...
as a checked version of
get_mapping()[get_key()] = ...
(or in general, any situation where you can't or don't want to
duplicate the expressions that produce the mapping and the key).
> I don't see any connection between "append" and "fail if the key already
> exists". That's not what it means with
[Python-ideas] Allow popping of slices
The `pop` method of built-in sequences is basically an atomic version of val = self[pos] del self[pos] return val If this behavior was extended to the case where `pos` is a slice, you could write things like: def cut_deck(deck, pos): deck.extend(deck.pop(slice(0, pos))) def bfs(roots): depth, frontier = 0, list(roots) while frontier: depth += 1 for item in frontier.pop(slice(None)): ... frontier.append(...) ... Similar functionality is found in many other languages (e.g. Perl and JavaScript's `splice`). I think it's useful not just because it's more concise, but because it's linear/reversible: it moves data rather than duplicating and then destroying it, which makes it less prone to bugs. The syntax is a bit odd since you have to construct the slice by hand. Here are three solutions for that from least to most extravagant: 1. Don't worry about it. It's still useful, and the syntax, though verbose, makes sense. (The "reference implementation" of pop is literally unchanged.) 2. Give pop methods a __getitem__ that does the same thing as __call__, so you can write xs.pop[-1] or xs.pop[:]. 3. Promote del statements to expressions that return the same values as the underlying __delitem__, __delattr__, etc., and make those methods of built-in types return the thing that was deleted. (Or introduce __popitem__, __popattr__, etc. which return a value.) -- Ben ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
