[Python-ideas] Re: Conditional 1-line expression in python

[Random832]

On Mon, Jul 17, 2023, at 16:41, Dom Grigonis wrote:
> Would be interesting to see if my preference is an outlier or not really.

I think this is a false dichotomy. We should consider VB-like conditional 
expressions if(condition, when_true, when_false) as well.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FZBHN6DVTH2IW4OZH4R36EYZWRRMEYJL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: range() manipulations

[Random832]

On Mon, Aug 1, 2022, at 09:19, Paul Moore wrote:
> There are a lot of complex cases you'd need to consider. What would the 
> value of range(3, 15, 2) + range(8, 12, 2) be? It's not a range in the 
> sense of being describable as (start, end, step). And simply saying 
> "that's not allowed" wouldn't really work, as it would be far too hard 
> to work with if operations could fail unexpectedly like this. In 
> reality, this feels more like you're after set algebra, which Python 
> already has.

Maybe it would make sense for the type of the result to devolve into a sorted 
set (do we have a sorted set type now?) if it's not representable as a range.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PKGCHWYKC5P27FTKZH5LGVQ5RCZMMXUS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Random832]

On Sat, Mar 26, 2022, at 14:30, Brendan Barnwell wrote:
>   To me it doesn't seem reasonable that someone would inherit from two 
> classes and want to call a method from one without even knowing that 
> there's a method name collision.  If you're going to inherit from A and 
> B, you need to know what methods they provide and you need to think 
> about the order you inherit in.

what about __init__?

you can't call A.__init__ and B.__init__ explicitly from C.__init__, because 
then both of them will call object.__init__ [maybe this is fine for object, but 
it isn't fine if you've got another class there that A and B inherit from]

It's hard enough to make signatures that this can be safely done with even 
*with* the mro, but the problem would be downright intractable without it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VK4X5UAZHAW575AQ5M4Q3B6J3PS7ZIIL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: synatx sugar for quickly run shell command and return stdout of shell command as string result

[Random832]

On Thu, Aug 26, 2021, at 08:50, Evan Greenup via Python-ideas wrote:
> Currently, when what to execute external command, either os.system() or 
> subprocess functions should be invoked.
> 
> However, it is not so convenient, it would be nice to use "``" symbol 
> to brace the shell command inside.
> 
> like:
> 
> stdout_result_str = `cat file.txt | sort | grep hello`

I don't think this is an important enough feature to be worth using up one of 
the few free ascii characters available for new syntax. However, it might be 
useful to have a helper function to build pipelines without the shell, so you 
could take the list [['cat', 'file.txt'], ['sort'], ['grep', 'hello']] and get 
this result.

Like, currently, to set up this pipeline (putting aside the useless use of 
cat), you have to do the following:

p1 = subprocess.Popen(['cat', 'file.txt'], stdout=PIPE)
p2 = subprocess.Popen(['sort'], stdout=PIPE, stdin=p1.stdout)
result = subprocess.check_output(['grep', 'hello'], stdin=p2.stdout)

[there are nuances about closing the intermediate pipes in the parent process 
that I've left out of this simple example]

something like subprocess.check_output_pipelined(['cat', 'file.txt'], ['sort'], 
['grep', 'hello']) might be useful.

> Its is enhanced feature which combine both subprocess and os.system

To do this *with* a shell, you can use os.popen and read the resulting file 
stream. But there are good reasons, I think, to want to be able to do this 
without a shell.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7VA3PBQ6KPCRIMTASRC3YOTH5FS7OMHM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Integer concatenation to byte string

[Random832]

On Wed, Mar 3, 2021, at 04:09, Barry Scott wrote:
> I was thinking of the C functions that are executed in ceval.c to run 
> the interpreter for any byte code.

In that case, it's not clear how your proposed syntax would not have the same 
overhead [especially your suggestion of a += operator]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4ZIVPMD4JY73NDAQU46TU2OWB7NV7UTM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: Re: Re: Have virtual environments led to neglect of the actual environment?

[Random832]

On Wed, Feb 24, 2021, at 09:08, Paul Moore wrote:
> I don't use Linux much, and I'm definitely not familiar with Linux
> distribution tools, but from what I can gather Linux distributions
> have made the choices:
> 
> 1. Write key operating system utilities in Python.
> 2. Share the Python interpreter and libraries.
> 3. Expose that Python interpreter as the *user's* default Python.

I think 1 *partially* mischaracterizes the problem, because any "system python" 
would logically be used by *every application written in python [or that embeds 
python] distributed by the OS's package management*, not just by "key operating 
system utilities". To suggest otherwise implies that they should not distribute 
any python applications at all.

That also makes asking all of their package maintainers to change their #! line 
to point at a different interpreter [or to pass an option, as I had suggested 
in another post] a more significant ask than the "just change a few core 
utilities" that some people seem to be assuming it would be. It also means that 
making a "system python" does not remove the need to distribute the largish 
subset of python *libraries* that they distribute, because even when these 
libraries aren't used by key OS utilities, they are still used by packaged 
applications.

[this, in turn, means we don't want the user's default python environment to 
stand entirely separate from the system python, or we'll start getting 
complaints along the lines of "I apt-get installed numpy, why can't I import it 
in my python interpreter?" - particularly from users who don't really care if 
it runs a couple versions behind]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F7YNRI7LWU2VLS6AU6PQZUHHA3M5WFQJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

[Random832]

I accidentally removed Python-Ideas from the recipient list while posting this 
reply.

On Wed, Feb 24, 2021, at 09:16, Random832 wrote:
> On Wed, Feb 24, 2021, at 06:27, Christian Heimes wrote:
> > Separate directories don't prevent clashes and system breakage. But they
> > provide an easy way to *recover* from a broken system.
> 
> I think it could be turned into a way to prevent them by A) having 
> site-packages always take precedence over dist-packages [i believe this 
> is already the case] in normal usage and B) providing an option to the 
> interpreter, used by system scripts, to exclude site-packages entirely 
> from the path.
> 
> Basically, site-packages would effectively be layered on top of "Lib + 
> dist-packages" in a similar way to how a venv is layered on top of the 
> main python installation - the inverse of the suggestion someone else 
> in the thread made for the system python to be a venv. This wouldn't 
> *exactly* be a venv because it wouldn't imply the other things that 
> entering a venv does such as "python" [and script names such as pip] 
> being an alias for the correct version of python, but it would provide 
> the same kind of one-way isolation, whereby the "system environment" 
> can influence the "normal environment" and not vice-versa, in the same 
> way that packages installed in the main installation affect a venv 
> [unless system-site-packages is disabled] but the venv obviously has no 
> effect on the main installation.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IQA6RHY24WGVVGRJT67CFABQVA5MMO3H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Arrow functions polyfill

[Random832]

On Tue, Feb 23, 2021, at 17:01, Rob Cliffe via Python-ideas wrote:
> As far as I know, there is no case of valid syntax using 'lambda' where 
> replacing 'lambda' by 'def' results in valid syntax.
> Can anyone provide a counter-example?
> If not, I would support allowing 'def' as an alternative to 'lambda' 
> (allowing 'def' to ultimately become the recommended usage).

I have an objection to this: "def" is short for define, and a lambda does not 
produce a definition. This isn't just about saving keystrokes, and even if it 
were, saving only three would not be worth it for a syntax that is just as 
confusing as the existing one.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DQA6G2AWX3TCAKCOTC3JPGX66S772JPK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Have virtual environments led to neglect of the actual environment?

[Random832]

I was reading a discussion thread 
 about various 
issues with the Debian packaged version of Python, and the following statement 
stood out for me as shocking:

Christian Heimes wrote:
> Core dev and PyPA has spent a lot of effort in promoting venv because we 
> don't want users to break their operating system with sudo pip install.

I don't think sudo pip install should break the operating system. And I think 
if it does, that problem should be solved rather than merely advising users 
against using it. And why is it, anyway, that distributions whose package 
managers can't coexist with pip-installed packages don't ever seem to get the 
same amount of flak for "damaging python's brand" as Debian is getting from 
some of the people in the discussion thread? Why is it that this community is 
resigned to recommending a workaround when distributions decide the 
site-packages directory belongs to their package manager rather than pip, 
instead of bringing the same amount of fiery condemnation of that practice as 
we apparently have for *checks notes* splitting parts of the stdlib into 
optional packages? Why demand that pip be present if we're not going to demand 
that it works properly?

I think that installing packages into the actual python installation, both via 
distribution packaging tools and pip [and using both simultaneously - the 
Debian model of separated dist-packages and site-packages folders seems like a 
reasonable solution to this problem] can and should be a supported paradigm, 
and that virtual environments [or more extreme measures such as shipping an 
entire python installation as part of an application's deployment] should 
ideally be reserved for the rare corner cases where that doesn't work for some 
reason.

How is it that virtual environments have become so indispensable, that no-one 
considers installing libraries centrally to be a viable model anymore? Are 
library maintainers making breaking changes too frequently, reasoning that if 
someone needs the old version they can just venv it? Is there some other cause?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KMRNKSVEPHAXA7BHFT4PWX4EKWYUF4G7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Alternate lambda syntax

[Random832]

On Tue, Feb 16, 2021, at 23:24, Stephen J. Turnbull wrote:
> except a couple of characters.  So what currently looks like
> 
> some_list.sort(key=lambda e: e[3].priority)
> 
> would then be
> 
> some_list.sort(key=(e)->e[3].priority)

Let's not pretend the key argument being keyword-only isn't a wart. Surely this 
would be better if it could be some_list.sort(e->e[3].priority).

> which is shorter but not particularly more readable (and already has a
> familiar meaning in C-like languages).

this side point is an argument in favor of using => instead.

[and if => can be confused with >=, surely so can -> be confused with >-]

> 1.  In a one-line def of the form "def foo([arglist]): return EXPR",
> "return" may be omitted, and the function returns the value of
> EXPR (rather than None as currently).  (As a multiline def, EXPR
> would be presumed to be evaluated for side effects, and 

I would like something like this - it's worth noting that C# [and as someone 
helpfully pointed out, Dart] uses => for this case as well, and the argument 
about the syntax looking bad when a return type annotation is present is, I 
think, overblown - return type annotations are almost never needed for a 
function with a single return statement.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BHH7EIXT7KSFMBGTSGG6VKWKD55Y2TE4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Arrow functions polyfill

[Random832]

On Sat, Feb 13, 2021, at 21:57, Steven D'Aprano wrote:
> # Erlang
> multiply(X,Y) -> X * Y.

For the record, Erlang's lambda syntax is the relatively unpleasant "fun(X,Y) 
-> X * Y end". Elixir is the same, except that the keyword is fn.

> Dart uses "=>"
> 
> multiply(x, y) => x * y;

And Dart has a very C#-like (x, y) => x * y for lambdas, though a lot of 
material describing Dart's function syntax options seems to have some confusion 
about what a lambda is, with some describing the above ordinary function 
definition as a lambda, and some forgetting to explain it, only bothering to 
mention => for ordinary definitions and (x, y) { return x * y; } for lambdas.

> More examples here:
> 
> http://www.rosettacode.org/wiki/Function_definition

That page doesn't really do well at explaining lambda expressions specifically, 
omitting it for some languages that definitely have them [such as Dart]. 
https://rosettacode.org/wiki/Higher-order_functions and 
https://rosettacode.org/wiki/First-class_functions both show examples of lambda 
syntax for some languages that it's missing from in the "Function definition" 
page... It's hard to sort through those from the ones that *don't* have a 
lambda syntax [compact or otherwise], since that's not the point of *any* of 
these pages though. https://en.wikipedia.org/wiki/Anonymous_function may be 
better.

C# uses =>, same as Javascript, as does Scala, as does D - and like I said 
earlier, Java (and Kotlin, and as you mentioned Julia) uses essentially the 
same syntax but with ->.

Anyway, Javascript started using this syntax in *2015*, C# did in *2007* [you 
may find some references that seem to say 2015 if not read closely, but these 
are discussing the introduction of => syntax to ordinary non-lambda function 
definitions]. In case that helps with some of the attitudes I'm seeing that 
this is in some way a Javascript thing that would somehow contaminate Python.

I don't really buy that anyone wouldn't understand what passing "x=>x+1" as an 
argument means if they understand the concept of first-class functions 
generally. And if they don't, that difficulty isn't really syntactic.

To add a few more examples to the language survey you seem to have started
Go uses func(x, y) x * y
Swift uses {x, y in return x * y}
Rust uses |x, y| x * y, as does Ruby

I still think arrows are the clear winner. I think they're intuitive, and I 
think that's *why* C# chose them and why other languages like Javascript copy 
them. It wouldn't be the first syntactic feature we'd have copied from C#.

> - how many people coming to Python will be familiar with Javascript as 
>   opposed to R, Julia, Maple, etc, or no language at all?

I don't really understand why you're bringing up R/Julia/Maple specifically. 
Are you advocating for ->? I have no particular objection to that, but I don't 
think the difference between two slightly different looking arrows [one used in 
C#, Javascript, Scala, and D; the other used in Java, Kotlin, Julia, R, and 
Maple] is particularly difficult. I think either one is just as good as the 
other in terms of the meaning being intuitive when reading code.

> - given how many other things they will have to learn that is different, 
>   is one comparatively rarely used feature more that much of a burden?

It's not about being the same to make it easier for people coming from a 
particular language. It's being the same because C# (and Javascript) gets it 
right, and we should get it right. *semi-wink*

[and it being a "comparatively rarely used feature" is a bit of a circular 
argument - there's a case to be made that it's less used than it could be 
because the current syntax is cumbersome]

> > b) Python already uses "->" for function return *type*. And there's
> > idea to generalize it to *function type* in general. E.g. a function
> > "(a, b) => a + b" can have type "(int, int) -> int".
> 
> This supports the idea of using "->" as the "return operator".

Guido thinks it opposes it.  I don't know if it makes much of a difference 
either way - people who wish to use annotations can use a full function 
definition. I don't think these should support annotations anyway, and I 
suspect they would mostly be used in contexts where annotations are 
unnecessary, such as as an argument to a function where the whole type of the 
function is already annotated.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UQXNFG37QLNGPKXSBVBTSC7SEF5H7ORT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Alternate lambda syntax

[Random832]

On Fri, Feb 12, 2021, at 02:23, Paul Sokolovsky wrote:
> > Great to hear there's no desire to stray away from JavaScript just for
> > the purpose of being different.
> 
> ... And on the 2nd thought, that won't work. The reason it works in JS
> is that it doesn't have tuples. In Python, "(a, b) => (1, 2)" means
> "compare a tuple for greater-or-equal".

Nope. greater-or-equal is >=, not =>. And Javascript may not have tuples, but 
it does have the comma operator.

> But fear not, we can steal "lambda operator" from Haskell:
> 
> \(a, b): (1, 2)

That's another option [and might be easier to parse], but I think it's less 
readable.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/H7PKBX2AAAXD4NG6EN7MIOUCP47V6UDH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Alternate lambda syntax

[Random832]

On Thu, Feb 11, 2021, at 06:48, Paul Sokolovsky wrote:
> Hello,
> 
> On Thu, 11 Feb 2021 12:24:55 +0100
> "J. Pic"  wrote:
> 
> > Hi all,
> > 
> > Lambdas can be defined as such:
> > 
> > w = lambda: [12]
> > x = lambda y: len(y)
> > 
> > I'd like to propose the following:
> > 
> > w = (): [12]
> 
> What will be the meaning of {(): [12]} ? Hint: it will be a dictionary
> of empty tuple mapping to a list, where do you see lambda here?

This could be solved with parentheses, but I think it'd probably be better to 
use similar syntax to C#, Java, and Javascript instead, and use () -> [12] or 
() => 12...

It's worth noting that all three of these are later additions to their 
respective languages, and they all have earlier, more difficult, ways of 
writing nested functions within expressions. Their designers saw the benefit of 
an easy lambda syntax, why don't we?

It also may be worth looking at what it would take to allow asynchronous 
lambdas. Syntactically, while "any lambda containing await" is tempting, the 
lack of static typing means that we need a way to specify async lambdas that do 
not contain await. Javascript prefixes the argument list with async [i.e. 
"async () => ..." or "async onearg => ..."], as does C# even though it could in 
principle get away without doing so because of static typing.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/44OSHX36GMKSUKWIHA4Q4STCSXHZTMFD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: What's up with assignment expression and tuples?

[Random832]

> On Sat, Feb 6, 2021 at 5:21 PM Random832  wrote:
> >
> > While we're on the subject of assignment expression limitations, I've 
> > occasionally wanted to write something like
> >
> > try:
> > return a_dict[key]
> > except KeyError:
> > return (a_dict[key] := expression to construct value)

On Sat, Feb 6, 2021, at 01:26, Chris Angelico wrote:
> 
> That's what the __missing__ method is for.

Requires a dict subclass

Requires that the value be determinable from the key alone [rather than any 
other variables available in the function containing the above code]

On Sat, Feb 6, 2021, at 01:50, Brendan Barnwell wrote:
>   You can already do that with `return a_dict.setdefault(key, 
> your_expression_here)`.  If the expression is expensive to evaluate you 
> can use a short-circuiting conditional expression to guard it.

How exactly would you use a short-circuiting conditional expression to do this?

If you're suggesting `... if key not in a_dict else ...` this creates a race 
condition, and also involves checking the key in the dictionary twice.

Perhaps a constructdefault method could be added to dict, broadly equivalent to

def constructdefault(self, func):
try:
return self[key]
except KeyError:
return (self[key] := func())

you could use self[key] = value = func(); return value; and the same in the 
original [not part of dict class] snippet I posted above, but the point is this 
seems so much like exactly the sort of use case that := is intended for, that 
it comes across as weird that it's not usable.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OYTML3CG5G5NUX4GSGS533NPAV4CRW46/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: What's up with assignment expression and tuples?

[Random832]

While we're on the subject of assignment expression limitations, I've 
occasionally wanted to write something like

try:
return a_dict[key]
except KeyError:
return (a_dict[key] := expression to construct value)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GNQCME3OFES5WY65LJJ3ZUPQQH3UPXPE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a couple of options to open()'s mode parameter to deal with common text encodings

[Random832]

On Thu, Feb 4, 2021, at 18:46, Ben Rudiak-Gould wrote:
> 
> On Thu, Feb 4, 2021 at 3:29 PM Chris Angelico  wrote:
> > With "t", it takes/gives Unicode objects, but with "b" it uses bytes.
> 
> Sure, in Python 3, but not in Python 2, or C.
> 
> Anyway, moral correctness is beside the point. People in point of fact 
> don't write encoding='utf-8' when they should, because it's so much to 
> type.

I'll say again, what if it were accepted as a positional argument? the current 
third positional argument is "buffering", which is an integer, but I doubt 
there's even much code that uses it intentionally.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QASTB6O7DJXXDGNV3EQZAGTVBQZBYGZP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Provide UTF-8 version of Python for Windows.

[Random832]

On Mon, Jan 25, 2021, at 22:49, William Pickard wrote:
> Looks like that's only available for Microsoft Store apps only, so it 
> might not be viable for Python.

I think the "Fusion manifest for an unpackaged Win32 app" part applies to 
non-store apps.

[English version of the page: 
https://docs.microsoft.com/en-us/windows/uwp/design/globalizing/use-utf8-code-page
 ]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FIIXK5DISHHDMUMA4UEZYOV2UBMZ2MEF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Random832]

On Sun, Jan 24, 2021, at 13:18, MRAB wrote:
> Well, if you see patterns like b'\x00H\x00e\x00l\x00l\x00o' then it's 
> probably UTF16-BE and if you see patterns like 
> b'H\x00e\x00l\x00l\x00o\x00' then it's probably UTF16-LE.
> 
> You could also look for, say, sequences of Latin characters and 
> sequences of Han characters.

This is dangerous, as Microsoft discovered: a sequence of ASCII latin 
characters can look a lot like a sequence of UTF-16 Han characters.

On Windows, Notepad always writes UTF-16 with BOM, even though it now writes 
UTF-8 without it by default.

Probably the winning combination is "if there is a UTF-16 BOM, it's UTF-16, 
else if first few non-ASCII bytes encountered are valid UTF-8, it's UTF-8", 
otherwise it's the system default 'ANSI' locale.

The one problem with that is what to do if something like a pipe or a socket 
gets a sequence of bytes that are a valid *partial* UTF-8 character, then 
doesn't get any more data for a while. It's unacceptable to have to wait for 
more data before interpreting data that has been read.

Notepad has the luxury of only working on ordinary files, and being able to 
scan the whole file before making a decision about the character set [I believe 
it mmaps the file rather than using ordinary open/read calls].
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DR4GEIPOWNQFWHETWM6L5Y2GGRZL2YRH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Random832]

On Sat, Jan 23, 2021, at 22:43, Matt Wozniski wrote:
> 1. Deprecate calling `open` for text mode (the default) unless an 
> `encoding=` is specified,

I have a suggestion, if this is going to be done:

If the third positional argument to open is a string, accept it as encoding 
instead of buffering. Maybe even allow the fourth to be errors.

It might be worthwhile to consider making the other arguments keyword-only - 
are they ever used positionally in real-world code?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HX37WBOP5PSMTVVNK7FVHLMEEGW4B2VX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Random832]

On Sat, Jan 23, 2021, at 08:00, Stephen J. Turnbull wrote:
> I see very little use in detecting the BOMs.  I haven't seen a UTF-16
> BOM in the wild in a decade (as usual for me, that's Japan-specific,
> and may be limited to the academic community as well), and the UTF-8
> BOM is a no-op if the default is UTF-8 anyway.

It's not *entirely* a no-op, you'd want the decoder to consume the leading BOM 
rather than returning '\ufeff' on the first read. And AIUI they're much more 
common on Windows (being able to detect UTF-16 *without* BOMs might be useful 
as well, but has historically been a source of problems on Windows) - until 
recently all UTF-8 or UTF-16 files saved with notepad would have them.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GNV2JJVRUI5QGXRAA6VTZYNPCD7OGVNA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Random832]

On Sat, Jan 23, 2021, at 05:06, Inada Naoki wrote:
> On Sat, Jan 23, 2021 at 2:43 PM Random832  wrote:
> >
> > On Fri, Jan 22, 2021, at 20:34, Inada Naoki wrote:
> > > * Default encoding is "utf-8".
> >
> > it might be worthwhile to be a little more sophisticated than this.
> >
> > Notepad itself uses character set detection [it might not be reasonable to 
> > do this on the whole file as notepad does, but maybe the first 512 bytes, 
> > or the result of read1(512)?] when opening a file of unknown encoding, and 
> > msvcrt's "ccs=UTF-8" option to fopen will at least detect at the presence 
> > of UTF-8 and UTF-16 BOMs [and treat the file as UTF-16 in the latter case].
> 
> I meant Notepad (and VS code) use UTF-8 without BOM when creating new text 
> file.
> Students learning Python can not read it with `open()`.

Right, I was simply suggesting it might be worthwhile to target "be able to 
open all files that notepad can open" as the goal rather than simply defaulting 
to UTF8-no-BOM only, which requires a little more sophistication than just a 
default encoding.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VJ67ZCY7HG6JTWM4K2JDZDQAJIXEMF4T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Random832]

On Fri, Jan 22, 2021, at 20:34, Inada Naoki wrote:
> * Default encoding is "utf-8".

it might be worthwhile to be a little more sophisticated than this.

Notepad itself uses character set detection [it might not be reasonable to do 
this on the whole file as notepad does, but maybe the first 512 bytes, or the 
result of read1(512)?] when opening a file of unknown encoding, and msvcrt's 
"ccs=UTF-8" option to fopen will at least detect at the presence of UTF-8 and 
UTF-16 BOMs [and treat the file as UTF-16 in the latter case].
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7TUNPIXTWSWKTFD2LE4UBV5SOOEUBGMY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] reversed enumerate

[Random832]

On Thu, Jan 21, 2021, at 19:19, Chris Angelico wrote:
> Yeah, I don't think that'll work if you slice more than once,
> especially with some iteration in between.

I think part of the point of this implementation [or the other person's 
suggestion of having a "collection view", which amounts to the same thing] is 
that iterating wouldn't consume items anymore. Which does make it incompatible 
with enumerate currently returning an iterator, though, in situations where the 
iteration is dropped and picked up again.

> The precise semantics for mixing iteration and slicing are complex
> enough and variable enough that you may as well just go for a concrete
> list at that point.

I do think it's possible to have well-defined semantics [next() could simply 
return self[0] and increment a value to be added to input indices], but it's 
probably not worth it. I'd say this and e.g. similar map/zip functions belong 
in a "sequencetools" module rather than replacing the builtin, so that 
compatibility in these situations doesn't need to be maintained.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D67HZZ2QX22SZ427Y6UKAB44UHIKHTL3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] reversed enumerate

[Random832]

On Thu, Jan 21, 2021, at 18:48, Chris Angelico wrote:
> Note that slicing is NOT easy. The proposed semantics for a reversed
> enumeration would make slicing extremely odd.

What proposed semantics? You were the one who posted a pure-python 
implementation that didn't bother to implement slicing.

It's easy enough to add slicing to your Enumerated class concept, though.

class Enumerated:
def __init__(self, basis, indices=None):
self.basis = basis
self.indices = indices if indices is not None else range(len(basis))
def __getitem__(self, idx):
if isinstance(idx, slice):
return Enumerated(self.basis, indices=self.indices[idx])
else:
return (self.indices[idx], self.basis[self.indices[idx]])
def __len__(self):
return len(self.indices)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZV3DG5HEM34WY2C5ETO744XBP4SAQENU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: pathlib enhancements

[Random832]

On Fri, Jan 8, 2021, at 15:47, Joseph Martinot-Lagarde wrote:
> One remark about this : .tar.gz files are the exception rather than the 
> rule, and AFAIK maybe the only one ? It's pretty common to have dots in 
> filenames instead of blanks for example, and stem does the right thing 
> here : '/data/my.little.file.txt'. There is also the case of hidden 
> files on Linux, what do you expect for /home/toto/.program.cfg ?

Hidden files are already treated specially - a file called simply ".whatever" 
is considered to be the stem, not the suffix.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R263HIE7466FYARL52HIQFAXQ4ZQI2ID/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python with braces formal proposal?

[Random832]

On Tue, Jan 5, 2021, at 17:17, lasizoillo wrote:
> Sorry, but if I'm understanting the point is to make one-liners. For 
> example, if I want to do something like:
> 
> $ env | grep "^XDG"
> 
> In one python one liner like
> 
> $ python -c 'import os;print("\n".join([f"{key}:{value}" for key, value 
> in os.environ.items() if key.startswith("XDG
> ")]))'
> 
> I can, but If I have things that require indentation I must change it 
> to something like:
> 
> $ python << EOF 

what if we had special support for python -c (and maybe in some other places 
like exec(), but definitely not for source files) for the purpose of 
one-liners? Then the syntax wouldn't need to be suitable for general purpose 
use, and you could do something like "have ~{ ~} ~; as alternate spellings of 
INDENT, DEDENT, and NEWLINE respectively"

python -c 'import os~;for key, value in os.environ.items():~{if 
key.startswith("XDG"):~{print(f"{key}:{value}")~}~}'
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QBUKHJBOXYTYO4TMWUJQZ5RNYDCVXLY5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: pathlib.Path.makedirs

[Random832]

On Wed, Jan 13, 2021, at 08:31, Antonio Cavallo wrote:
> Hi,
> I've found myself typing too many time this:
> pathlib.Path("some-dir-name").mkdir(parent=True, exist_ok=True)
> 
> Wouldn't be better to have a pathlib.Path.makedirs with parent/exist_ok 
> set to True by default?

Worth mentioning, that os.makedirs does not set exist_ok to True by default. I 
have been baffled by this decision, but not following it in pathlib would 
arguably be an inconsistency.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BMN2A2FMDIMAZ3F62BYFP73CR3T3WK5F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: TextIOWrapper support for null-terminated lines

[Random832]

On Sun, Oct 25, 2020, at 18:45, Chris Angelico wrote:
> If you actually DO need to read null-terminated records from a file
> that's too big for memory, it's probably worth just rolling your own
> buffering, reading a chunk at a time and splitting off the interesting
> parts. It's not hugely difficult, and it's a good exercise to do now
> and then. And yes, I can see the temptation to get Python to do it,
> but unfortunately, newline support is such a weird mess of
> cross-platform support that I don't think it needs to be made more
> complicated :)

Maybe a getdelim method that ignores all the newline support complexity and 
just reads until it reaches the specified character? It would make sense on 
binary files too.

The problem with rolling your own buffering is that there's not really a good 
way to put back the unused data after the delimiter if you're mixing this 
processing with something else. You'd have to do it a character at a time, 
which would be very inefficient in pure python.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CXZWUKIIJNGP7EDXG7P3CHZKF3XW2P6P/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: f-string: empty expression should be allowed

[Random832]

On Thu, Oct 22, 2020, at 21:00, Steven D'Aprano wrote:
> "Who"? Syntax is a person now?
> 
> For what it's worth, I don't think escaping special symbols is ugly. 
> More like homely. It does the job, plainly and simply, but I don't think 
> it's especially good looking. Every other character stands for itself, 
> with this handful of exceptional cases.

I suspect that calling this particular syntax ugly is picking at a bit of an 
open wound in the history f-string implementation... consider precisely why the 
escaping syntax is {{}} instead of \{\}, and all the implications of that.

(with the PEG parser, incidentally, I wonder if it may be time to revisit 
certain limitations in f-string syntax, though that particular one technically 
presents a backward compatibility problem that the rest don't)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OXIGDJWQNNONKKLJ47JLY3W57YEJZAQM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: f-strings as assignment targets

[Random832]

On Tue, Oct 20, 2020, at 22:03, David Mertz wrote:
> My initial impression of your intent was:
> 
> foo, bar = 42, 99
> # ... a million lines ...
> line = "123/"
> # ... more lines ...
> f"{foo}/{bar}" = line
> # raises if bar wasn't previously set
> # binds to prior value if it was set
> 
> But I think what you want is for the binding line never to raise, but 
> also not to have any local means to know whether 'bar' is a name after 
> that line. Or whether 'foo' is, for that matter.

"whether 'bar' is a name"? It is definitely a name, what you have no means to 
know is whether it has been assigned a value. I suspect you're trying to do the 
thing some people do where they insist on 'name' to avoid using the term 
'variable', and I don't want to start a debate about that, but you've used it 
incorrectly here, and your language seems to suggest it would be reasonable to 
treat bar as *global* if unassigned here. In fact, all names assigned in the 
function are in scope for the entire function regardless of whether they have 
been assigned values, whether that assignment is in a conditional that wasn't 
taken or even simply later in the function than when it is used.

Do you have similar objections to assigning inside an if statement? Because the 
suggested semantics are essentially the same as
x = sscanf(line, "%d/%d")
if len(x) >= 1: foo = x[0]
if len(x) >= 2: bar = x[1]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JKEJYTG2TS3AS263GKNEXZYK4C7D3YRT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Bringing the print statement back

[Random832]

On Mon, Oct 19, 2020, at 22:41, James T Moon wrote:
> tl;dr *boo* *hiss*
> 
> I can only imagine the new Python programmer's look of confusion, 
> turning to disgust, turning to giving up.

Ten years from now, hopefully, Python 2 will be a distant memory, and Python 
3.8 will be, at the very least, long out of support and unlikely to be the 
system new users are learning Python on, and none of the dialogue you've just 
written will have to take place at all.

Does anyone else remember when xkcd first mentioned python? The main selling 
point it had for it [and the one that was actually literally true, vs 'import 
antigravity' which was a semi-satirical bit about the batteries-included 
philosophy] was 'Hello world is just print "Hello, world!"'

Wouldn't it be nice if that were true again, even if the details of how it 
works [and other things like how you print to other files] aren't quite the 
same as they used to be?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AEWAFNPPYDGQC2VWKX3OHIFN6EVWLH7C/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exceptions as function calls to kill boilerplate

[Random832]

On Mon, Oct 12, 2020, at 17:16, jmwar...@gmail.com wrote:
> Instead of needing a whole new class definition, wouldn't it be nice to 
> just have something like:
> 
> 
> #notice there isn't a boilerplate custom class created!
> try:
>   if some_test_that_fails(variables):
> #I still have a base exception to fall back on for handlers that 
> don't know my special exception
> raise Exception.my_special_exception(a, b, c, d, e, f)
> except Exception.my_special_excpetion(a:int, b:str, d, e, f):
>   logger.warning(f"BAD THING {a} HAPPENED!")
>   if not handle_it(a, b, c, f):
> raise

It seems like this could be a good use case for pattern matching...

try:
...
raise Exception(a, b, c, d, e, f)
except Exception as e match e.args:
case (a: int, b: str, c, _, _ f):
   logger.warning(f"BAD THING {a} HAPPENED!")
   if not handle_it(a, b, c, f):
 raise

[Incidentally PEP 634 is very light on actual examples, and I'm having trouble 
visualizing what the syntax actually looks like, so please forgive me if I 
misunderstood what the pattern should look like]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I5SA3Q67DMKUVSOCLGG7K23XUQBQDMF5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exact decimal multiplication and division operations

[Random832]

On Fri, Oct 9, 2020, at 17:38, Tim Peters wrote:
> [Random832 ]
> > My suggestion was for a way to make it so that if an exact result is
> > exactly representable at any precision you get that result, with
> > rounding only applied for results that cannot be represented exactly
> > regardless of precision.
> 
> That may have been the suggestion in your head ;-) , but - trust me on
> this - it's taken a long time to guess that from what you wrote.
> Again, you could have saved us all a world of troubles by giving
> concrete examples.

The problem is that the issue wasn't so much a concrete use case [other than 
the one thing with fraction, which doesn't exercise nearly all of the things 
that I had been suggesting changes for], but a general sense of unease with the 
philosophy of the decimal module - the fact that people are willing to say 
things like "if an exact result is exactly representable, then that's the 
result you get" when that's not actually true, because they've twisted the 
meaning of "exactly representable" up in knots.

And the documentation isn't very up front about this - for example where it 
talks about how significant figures work for multiplication, it doesn't talk 
about the context rounding at the end. It says "For multiplication, the 
“schoolbook” approach uses all the figures in the multiplicands. For instance, 
1.3 * 1.2 gives 1.56 while 1.30 * 1.20 gives 1.5600.", - this would be a good 
time to mention that it will be rounded if the context precision is less than 5.

It was omissions like these that led me to believe that addition and 
subtraction *already* behaved as I was suggesting when I first posted this 
thread.

> What you wrote just now adds another twist:  apparently you DO want
> rounding in some cases ("results that cannot be represented exactly
> regardless of precision"). The frac2dec function I suggested
> explicitly raised a ValueError in that case instead, and you said at
> the time that function would do what you wanted.
> 
> > It seems like this is incompatible with the design of the decimal module, 
> > but
> > it's ***absolutely*** untrue that "if an exact result is exactly 
> > representable,
> > then that's the result you get", because *the precision is not part of the
> > representation format*.
> 
> Precision certainly is part of the model in the standards the decimal
> module implements.

But the context precision is not part of the actual data format of a decimal 
number. The number Decimal('1.23456789') is an identical object no matter what 
the context precision is, even if that precision is less than 9.

Even though it's part of the *model* used for calculations, it's not relevant 
to the *representation*, so it has no effect on the set of values that are 
exactly representable. So, "if the exact result is exactly representable that's 
what you get" is plainly false.

> No, it does:  for virtually every operation, apart from the
> constructor, "exactly representable" refers to the context's precision
> setting.

That is nonsense. "exactly representable" is a plain english phrase and has a 
clear meaning that only involves the actual data format, not the context.

>  If you don't believe that, see how and when the "inexact"
> flag gets set. It's the very meaning of the "inexact" flag that "the
> infinitely precise result was not exactly representable (i.e.,
> rounding lost some information)". 

Nonsense. The meaning of the inexact flag means that the module *chose* to 
discard some information. There is no actual limit [well, except for much 
larger limits related to memory allocation and array indexing] to the number of 
digits that are *representable*, the context precision is an external limit 
that is not part of the *representation* itself.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E47C6V3VIEXWERKXHCXR56OHIGXVP72M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exact decimal multiplication and division operations

[Random832]

On Thu, Oct 8, 2020, at 16:07, Tim Peters wrote:
> See above.  It's very much a feature of the IEEE 754/854 standards (of
> which family Python's decimal model is a part)  that if an exact
> result is exactly representable, then that's the result you get.

My suggestion was for a way to make it so that if an exact result is exactly 
representable at any precision you get that result, with rounding only applied 
for results that cannot be represented exactly regardless of precision. It 
seems like this is incompatible with the design of the decimal module, but it's 
***absolutely*** untrue that "if an exact result is exactly representable, then 
that's the result you get", because *the precision is not part of the 
representation format*.

What threw me off is the fact that there is a single type that represents an 
unlimited number of digits, rather than separate types for each precision. I 
don't think that feature is shared with IEEE, and it creates a philosophical 
question of interpretation [the answer of which we clearly disagree on] of what 
it means for a result to be "exactly representable", which doesn't exist with 
the IEEE fixed-length formats.

At this point, doing the math in Fractions and converting back and forth to 
Decimal as necessary is probably good enough, though.

> But that's apparently not what you're asking for.
> 
> > ...
> > Incidentally, I also noticed the procedure suggested by the documentation 
> > for
> > doing fixed point arithmetic can result in incorrect double rounding in 
> > some situations:
> > >>> (D('1.01')*D('1.46')).quantize(TWOPLACES) # true result is 1.4746
> > Decimal('1.48')
> 
> Are you using defaults, or did you change something?  Here under
> 3.9.0, showing everything:

This was with the precision set to 4, I forgot to include that. With default 
precision the same principle applies but needs much longer operands to 
demonstrate. The issue is when there is a true result that the last digit 
within the context precision is 4, the one after it is 6, 7, 8, or 9, and the 
one before it is odd. The 4 is rounded up to 5, and that 5 is used to round up 
the previous digit.

Here's an example with more digits - it's easy enough to generalize.

>>> ctx.prec=28
>>> (D('1.01')*D('1.46')).quantize(D('.01'))
Decimal('1.48')
>>> ctx.prec=29
>>> (D('1.01')*D('1.46')).quantize(D('.01'))
Decimal('1.47')

The true result of the multiplication here is 1.4746, 
which requires 29 digits of precision

[and, no, it's not just values that look like 99 and 01, but a brute 
force search takes much longer for 15-digit operands than 3-digit ones] 

> > I didn't know that Decimal supported E notation in the constructor, so I 
> > thought
> > I would have to multiply or divide by a power of ten directly [which would
> > therefore have to be rounded]... going through the string constructor seems
> > extremely inefficient and inelegant, though,
> 
> Not at all!  Conversion between decimal strings and decimal.Decimal
> objects is close to trivial.  It's straightforward and linear-time (in
> the number of significant digits).

I think for some reason I'd assumed the mantissa was represented as a binary 
number, since the .NET decimal format [which isn't arbitrary-precision] does 
that I should probably have looked over the implementation more before jumping 
in.

> > I'd like a way to losslessly multiply or divide a decimal by a power of ten
> > at least... a sort of decimal equivalent to ldexp.
> 
> Again, without concrete examples, that's clear as mud to me.

Er, in this case the conversion of fraction to decimal *is* the concrete 
example, it's a one-for-one substitution for the use of the string constructor: 
ldexp(n, -max(e2, e5)) in place of D(f"{n}E-{max(e2, e5}").
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AA5P4PPBAPA66WLWQP4ZN4CM4ZMARNIN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exact decimal multiplication and division operations

[Random832]

On Thu, Oct 8, 2020, at 13:12, Steven D'Aprano wrote:
> I trimmed it because I didn't understand the relevance. If the 
> denominator is an exact multiple of only 2 and 5, then I think the 
> conversion will be exact if the precision is sufficient.

I wanted the conversion to be exact irrespective of the precision, not least 
because I'm not sure how to determine what the needed precision would be.

After I looked into the issue more it looks like changing addition and 
subtraction [which I was wrong about already supporting this] would be too 
invasive for this concern to justify adding full "exact if possible" support to 
decimal, but I do think it might be worthwhile to add a lossless decimal 
equivalent to frexp/ldexp.

On Thu, Oct 8, 2020, at 12:40, Steven D'Aprano wrote:
> Related: why doesn't Decimal support direct conversion from Fraction?

That's what I wanted to know - I noticed that it didn't when I was trying to 
figure out what Guido meant when he alluded to some issue with Decimal in the 
SupportsString thread, and figured it should be easy enough to add support, 
then ran into a wall [because I wasn't aware the constructor supported E 
notation].
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TEXOYM5RYCTCMUGAM4D5R4Q5GXCMWHNG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Exact decimal multiplication and division operations

[Random832]

On Thu, Oct 8, 2020, at 13:17, Tim Peters wrote:
> [Random832 ]
> > I was making a "convert Fraction to Decimal, exactly if possible" function 
> > and
> > ran into a wall: it's not possible to do some of the necessary operations 
> > with
> > exact precision in decimal:
> >
> > - multiplication
> > - division where the result can be represented exactly [the divisor is an
> > integer whose prime factors are only two and five, or a rational number
> > whose numerator qualifies]
> 
> Without you supplying careful concrete examples, I really have no idea
> what you're looking for.

I don't understand what's unclear - I was suggesting there should be an easy 
way to have the exact result for all operations on Decimal operands that have 
exact results representable as Decimal.

I did misunderstand part of the documentation, though... the documentation 
claims "Some operations like addition, subtraction, and multiplication by an 
integer will automatically preserve fixed point. ", and I assumed without 
checking that this applied even if the size of the operands was greater than 
the context precision, and therefore that addition and subtraction were always 
exact. This is why my post was so focused on multiplication and division.

Sine correcting that misunderstanding, I've looked over the implementation and 
I no longer think this is worth adding without a concrete use case, since 
supporting adding very large and very small numbers together would introduce 
more inefficiencies than it's worth.

Incidentally, I also noticed the procedure suggested by the documentation for 
doing fixed point arithmetic can result in incorrect double rounding in some 
situations:
>>> (D('1.01')*D('1.46')).quantize(TWOPLACES) # true result is 1.4746
Decimal('1.48')

> Integers have unbounded precision, so stick to those when part of a
> computation needs to be unbounded.  If by "Decimal" you mean Python's
> "decimal.Decimal" class, the constructor ignores the context
> precision, and retains all the info passed to it,  So there's no need
> at all to change Decimal precision.

This only applies to the constructor though, not the arithmetic operators.

> Here's a guess at what you want:

That works for the specific use case of converting Fraction to Decimal- I 
didn't know that Decimal supported E notation in the constructor, so I thought 
I would have to multiply or divide by a power of ten directly [which would 
therefore have to be rounded]... going through the string constructor seems 
extremely inefficient and inelegant, though, I'd like a way to losslessly 
multiply or divide a decimal by a power of ten at least... a sort of decimal 
equivalent to ldexp.

And I guess for the other operations it's possible to work around by doing the 
operations in Fraction and converting to Decimal in the end - that also seems 
inelegant, but oh well.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XF25A7K55FYRGZ2DWYD5MTTISD62XY56/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Exact decimal multiplication and division operations

[Random832]

I was making a "convert Fraction to Decimal, exactly if possible" function and 
ran into a wall: it's not possible to do some of the necessary operations with 
exact precision in decimal:

- multiplication
- division where the result can be represented exactly [the divisor is an 
integer whose prime factors are only two and five, or a rational number whose 
numerator qualifies]

I assume there's some way around it that I haven't spent enough time to figure 
out [create a temporary context with sufficint digits for multiplication, and 
work out the reciprocal power of 10 by hand to use this multiplication to 
implement division], but I feel like these exact operations should be supported 
in the standard library.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AEOK6HJJAEG65YO7T4C3KLQ4BPLX735T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: CPP Namespaces For Python

[Random832]

On Tue, Oct 6, 2020, at 23:13, Guido van Rossum wrote:
> A. New syntax is way too high a bar for a questionable feature. Classes 
> full of static or class methods were a pattern at my last employer and 
> it was unpleasant to work with. (Others at the company agreed but it 
> was too late to change.)
> 
> B. At some point we realized that metaclasses have too much power over 
> subclasses (action at a distance) and we switched to recommending class 
> decorators. Another problem with metaclasses is that it's a pain to 
> combine two different metaclasses.

I think a metaclass [well, "pseudo-metaclass", to use a term I made up for a 
metaclass that is not a subclass of type and/or does not return an instance of 
type] would be better in this case because the resulting object should not be a 
type.

The reason I wanted different syntax was because I wanted to change how the 
functions inside are compiled [to allow nonlocal accesses to refer to the 
namespace's scope], though I think making them globals and a custom dict 
subclass [possibly ChainMap, haven't checked if it does everything I want] for 
globals makes this workable as a metaclass or decorator - either one can take 
all of the functions and replace them with a new function with different 
globals and the original function's code object.

> > namespace Foo:
> > x=1
> > def bar():
> > pass
> > def baz()
> > return bar() + x
> 
> That could be done with a function and a function decorator. "Proof 
> left as an exercise for the reader." :-)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2HWZUS6G33TRBN7NINH4FJB4ROSGDLLY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: CPP Namespaces For Python

[Random832]

On Tue, Oct 6, 2020, at 02:50, Alperen Keleş wrote:
> Hi,
> 
> Please pardon me if my idea is not making sense or already exists, I'm 
> kind of new to developing in Python but I had this idea today and I 
> wanted to share it with you.
> 
> I think a class type such as "@functionclass" may be helpful for 
> creating functions intended to keep a list of methods in a scope. 
> 
> At the moment, I achieved this via writing "@classmethod" to all my 
> functions but I think such a decorator might help clarify intent for 
> the reader and ease the programmers' job.

I think new syntax would be better than a decorator (or a metaclass, which for 
some reason never seems to get suggested for these things), because I think the 
feature should allow for the functions to directly access each other from the 
namespace's scope without requiring an attribute lookup.

namespace Foo:
x=1
def bar():
pass
def baz()
return bar() + x
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4QP77RRSTAGHBRHFNSEIKY4HL3B7CJXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Curious : Why staticmethod if classmethods can do everything a static method can?

[Random832]

On Sat, Sep 12, 2020, at 23:14, Steven D'Aprano wrote:
> We already have an instancemethod, it's just spelled differently:
> 
> py> from types import MethodType
> 
> 
> And while it is not useful as a decorator, it is *really* useful for 
> adding methods to an individual instance rather than the class:

This isn't what I was suggesting - I meant something like this:

class instancemethod:
def __init__(self, wrapped):
self.wrapped = wrapped
def __get__(self, obj, objtype):
if obj is None: return self.wrapped
else: return MethodType(self.wrapped, obj)

this wouldn't be useful for functions, but would give other callables the same 
functionality as functions, automatically creating the bound method object, 
e.g.:

class D:
def __init__(self, obj, *args): ...

class C:
foo = instancemethod(D)

obj = C()
obj.foo(...) # D(obj, ...)

Same for other types of callables such as functools.partial objects etc.

> So an instancemethod decorator would be a waste of time, but the 
> instancemethod type, spelled types.MethodType, is very useful.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CYBMRSMOQOKSYELNW7AZ36WL3W23RD5Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Curious : Why staticmethod if classmethods can do everything a static method can?

[Random832]

On Fri, Sep 11, 2020, at 19:57, Cameron Simpson wrote:
> The default (an instance method) requires "self" to perform.

Of course, this is only the default if the method is a function object. If it 
is a different callable class, the default is effectively staticmethod.

Perhaps there should be an @instancemethod?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5ZMYQ33BPI2VNWADQIBSGD4NVLXI4SQF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: 'Infinity' constant in Python

[Random832]

On Fri, Sep 4, 2020, at 12:45, Cade Brown wrote:
> I am positing that Python should contain a constant (similar to True, 
> False, None), called Infinity.

What if we created a new syntax [and used it for the repr] that is not 
currently a valid identifier?

something like "1.INF"
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BN5ZRN5IWERVBZFUSPBD4RJKQ7FGO5HB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Changing item dunder method signatures to utilize positional arguments (open thread)

[Random832]

On Tue, Sep 1, 2020, at 21:06, Ricky Teachey wrote:
> Here's my attempt, it is probably lacking. I'm five years into a 
> self-taught venture in python... If I can't get this right the first 
> time, it worries me a little. 
> 
> 
> MISSING=object()
> 
> def __getitem__(self, key=MISSING, time=MISSING, money=MISSING):
> if time is MISSING or money is MISSING:
> if time is not MISSING and key is not MISSING:
> money = key
> elif money is not MISSING and key is not MISSING:
> time = key
> else:
> time, money = key
> 
> Is this right? Wrong? The hard way? The slow way? 
> 
> What about when there are three arguments? What then?

I'm using () rather than a MISSING as the sentinel here because it makes the 
code simpler. For the same reason, if we need to choose a key to pass in by 
default for a[**kwargs only] without defining a () in __getitem__, I have 
consistently advocated using () for this. Likewise, *not* having a default [or 
requiring the method to define its own default] key presents a slight problem 
for __setitem__, illustrated below.

def __getitem__(self, key=(), /, **kwargs):
return self._getitem(*(key if isinstance(key, tuple) else (key,)), **kwargs)
def __setitem__(self, key=(), value=???, /, **kwargs):
return self._setitem(value, *(key if isinstance(key, tuple) else (key,)), 
**kwargs)
def _getitem(self, time, money):
...
def _setitem(self, value, time, money):
...
[delitem the same as getitem]

Basically, you can easily write code that leverages the existing function 
argument parsing, simply by performing a function call.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EWJAKIBXQQJNMWT7EXZHAKOUP35YOREV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

[Random832]

On Mon, Aug 31, 2020, at 02:45, Greg Ewing wrote:
> On 31/08/20 3:35 pm, Random832 wrote:
> 
> > x[(1,)] old arg=(1,); new args=((1,),)?
>  > x[(1,2)] old arg=(1,2); new args=((1,2),)?
> 
> No, I proposed *not* to do those -- putting parens around the
> arguments would continue to make no difference, regardless of
> which dunder was being called.

What about passing in a tuple object that's in a variable?

a=1,2
x[a]

should args be ((1,2),) or (1, 2)?

Having x[a] be different from x[(1,2)] would be *bizarre*, but having it result 
in args=(1,2) would be keeping almost as much baggage from the current paradigm 
as not having a new dunder at all. I think that's why I assumed as a matter of 
course that a new dunder meant a tuple argument would unambiguously become a 
single argument.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7ZQRK6Y6Y6HVNB6PI7PQQBAF35NGT6PP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

[Random832]

On Mon, Aug 31, 2020, at 00:28, Guido van Rossum wrote:
> On Sun, Aug 30, 2020 at 8:36 PM Random832  wrote:
> > The thing that bothers me with new dunders is - can it be done in such a 
> > way that *all* possible ways of calling it with only positional arguments 
> > behave the same as presently when the old one is defined, and an intuitive 
> > way with the new one, without requiring the calling bytecode to know which 
> > signature is going to be used?
> 
> Probably not. (And knowing the signature is impossible -- there are too 
> many layers of C code between the bytecode and the function object 
> being called.) This is why I ended up with the simplest proposal 
> possible -- keyword args get added to the end of `__getitem__` and 
> `__setitem__`, ensuring that `d[1, k=3]` is like `d[1]` + keyword, not 
> like `d[1,]` + keyword.

perhaps, but I did have a thought after making that post.

A new bytecode operation (we'll need one anyway, right?) which, in addition to 
passing in the positionals and the keywords, also passes along the information 
of whether or not the subscript contents consisted of precisely a single 
expression without a trailing comma (or with one, if that'd work better... i 
believe flagging either one of these cases provides enough information to 
determine what form of argument should be passed to the single-argument 
__getitem__).
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6KOU2FWQZX7YGINZYESMVDA3MJWSWWS3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

[Random832]

On Mon, Aug 31, 2020, at 00:26, Chris Angelico wrote:
> Maybe I'm misreading this, but my understanding is that the definition
> of the dunder is actually how it's called, not what its signature is.
> The simplest and most obvious way to do it would be to have the
> interpreter pass the value positionally, and then keyword arguments
> separately. You can use whatever name you like for the value, and if
> you want, you can mandate that it be positional-only:
> 
> def __setitem__(self, key, value, /, **kw):
> 
> and then you can accept any arguments you like by keyword, even
> "self", "key", or "value".

Keep in mind that I am responding to a post that seems to call for new dunder 
methods that are passed multiple positional "key" arguments instead of a single 
one.

Passing the value last [i.e. in between the passed-as-positional and 
passed-as-keyword arguments] seems like a non-starter:

a[1, 2] = 3: f(self, 1, 2, 3)
a[1, y=2] = 3: f(self, 1, 3, y=2)
a[2, x=1] = 3: f(self, 2, 3, x=1)

There's no possible function signature that can reasonably deal with those.

essentially, by being a named argument that is not an intended keyword 
argument, value acts like / in the argument list, preventing any arguments 
before it from being passed in as keywords without causing errors. But since it 
is passed in positionally, it also prevents any arguments after it from being 
passed in as positional at all.

passing the value *first* might be reasonable (it may be the only viable way), 
but it would be a change from how it's currently done, and I do think this 
needs to be discussed for any proposal around passing in multiple positional 
keys.

passing the value in as a keyword called __value__ might be another possible 
way.

> This doesn't change the issue of tuplification, but it does mean that
> the issue is no worse for setitem than for getitem.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YOQAOJDES4AQPAFCNYFM4KYIGOCK5LYV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

[Random832]

On Sat, Aug 29, 2020, at 20:14, Greg Ewing wrote:
> I think this could be done more simply as
> 
>  def __getitem__(self, index, **kwds):
>  self.real_getitem(*index, **kwds)
> 
>  def real_getitem(self, x, y):
>  ...
> 
> The point about obscuring the signature still remains, though.
> 
> Also, this is a hack that we would never be able to get rid of,
> whereas new dunders would provide a path to cleaning things up
> in the future.

The thing that bothers me with new dunders is - can it be done in such a way 
that *all* possible ways of calling it with only positional arguments behave 
the same as presently when the old one is defined, and an intuitive way with 
the new one, without requiring the calling bytecode to know which signature is 
going to be used?

__getitem__(self, arg) vs __getitem_ex__(self, *args, **kwargs)

x[1] - old arg=1; new args=(1,)
x[1,] - old arg=(1,); new args=(1,)?
x[(1,)] old arg=(1,); new args=((1,),)?
x[1,2] old arg=(1,2); new args=(1,2)
x[(1,2)] old arg=(1,2); new args=((1,2),)?
x[*a] old arg=a new args=a?

Also, do we want to walk the MRO looking for both in turn, or look for the new 
one on the whole chain before then looking for the old one? is there a 
precedent for two different ways to define a method on a single class? [getattr 
vs getattribute is not a case of this, and i say "single class" to exclude the 
complexity of binary operators]

Another concern is where a new setitem should put the value argument? i may 
have missed something mentioning it, but I don't think I've seen a proposal 
that goes into detail on that? Having the user define a __setitem__ that calls 
the real_setitem if needed gets around that by leaving the signature up to the 
user [where they can e.g. put it in a keyword arg with a name they know they 
won't need]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IOMTOY43LUFBXJS2PYHR2FMCHFCI2NNK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - new dunder attribute, to influence item access

[Random832]

On Sat, Aug 29, 2020, at 10:12, Ricky Teachey wrote:
> If we add kwargs to the subscript operator, we'll be able to add new 
> required or optional names to item dunder signatures and supply named 
> arguments :
> 
> def __getitem__(self, key, x, y): ...
> 
> q[x=1, y=2]
> 
> But if we want to have the same behavior without supporting function 
> style syntax, we will have to write code like this:
> 
> MISSING = object()
> 
> def __getitem__(self, key, x=MISSING, y=MISSING):
> if x is MISSING and y is MISSING::
> x, y = key
> if x is missing:
> x, = key
> if y is MISSING:
> y, = key

[side note: I don't believe the behavior suggested by this last "if y is 
MISSING:" clause is supported by your proposal either. It's certainly not 
supported by function calls. Are you suggesting q[x=1, 2], or q[2, x=1] to be 
equivalent to y=2?]

def _getitem(self, x=..., y=...):
...

def __getitem__(self, arg, /, **kwargs):
if isinstance(arg, tuple): return self._getitem(*arg, **kwargs)
else: return self._getitem(arg, **kwargs)

or, as a more compact if slightly less efficient version

def __getitem__(self, arg, /, **kwargs):
return self._getitem(*(arg if isinstance(arg, tuple) else (arg,)), **kwargs)

This is only a few lines [which would have to be duplicated for set/del, but 
still], you're free to implement it on your own if necessary for your use case.

My code assumes that no positional arguments results in an empty tuple being 
passed - it would be slightly more complex but still manageable if something 
else is used instead.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XEMYGQDSEUOIN3RZMORJLWTNS3JUH6C4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding additionnal common request handlers to http.server

[Random832]

On Mon, Aug 24, 2020, at 12:39, Chris Angelico wrote:
> On Tue, Aug 25, 2020 at 2:36 AM Simon  wrote:
> > In my opinion, REST is the best protocol to make two pieces of software 
> > communicate with each other, because of its simplicity, yet, the standard 
> > library lacks in that regard in my opinion.
> >
> 
> Fair enough. I guess the real question is, how much advantage is
> RESTRequestHandler over directly subclassing BaseHTTPRequestHandler?
> Maybe it'd be worth it just to simplify that case.
> 
> ChrisA

What about making a stdlib class for building RESTful WSGI applications, which 
could be used with wsgiref for the lightweight-no-external-dependencies use 
case?

basically a "flask light".
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZO7EDEFZM5IVWI7J2X4DC364HIGC6ITE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - slices in keyword indices, d[x=1:3]

[Random832]

On Mon, Aug 24, 2020, at 00:43, Christopher Barker wrote:
> But thus brings up a broader question:
> 
> Why not allow slice syntax as an expression everywhere? Everywhere I’ve 
> tried, it’s a syntax error now, but is there any technical reason that 
> it couldn’t be used pretty much anywhere?

is {a:b} a set containing a slice, or a dict? obviously it's a dict, but are 
there any other places that might be affected? what about other forms of 
slices, or if it's not the only element? should we support {a, b:c} as a set 
containing a slice? what about {a:b, c}?

there may be other places it might be desirable to add new syntax that uses the 
colon character, allowing slices anywhere would foreclose that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CEF2MFP6WA4RKRV4TNWCDTNLAPZ243VD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve error message for trying to import itself

[Random832]

On Sat, Aug 22, 2020, at 20:17, Chris Angelico wrote:
> A speed drawback on every import would almost certainly be too high a
> price to pay.

My proposal would only add an extra check 1) on module load [not on import, so 
you only pay it once per module, and not at all for built-in modules] 2) only 
for modules loaded from the first entry in sys.path [script directory / current 
directory], so it shouldn't add much overhead at all to imports of stdlib or 
installed packages.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/APNOGQUDRMNJWIKMSR3VZD7BJYL5YISD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve error message for trying to import itself

[Random832]

On Sat, Aug 22, 2020, at 03:09, Random832 wrote:
> > AttributeError: partially initialized module 'spam' has no attribute 
> > 'egsg' (most likely due to a circular import)
> > 
> > If the error wasn't due to shadowing or a circular import, then knowing 
> > that I had misspelled 'eggs' would be very useful.
> 
> hmm
> 
> what about
> 
> AttributeError: module 'spam' from './spam.py' has no attribute 

I prematurely hit send, I don't mean to suggest remove the attribute name, just 
to add the module path [maybe the full path, whatever's in __file__ would be 
easiest]. This way wouldn't even require detecting circular imports or 
shadowing specifically, though it would require a little bit more alertness on 
the part of the user to notice that the path is not the system module they 
expect.

this could be done in combination with a warning or additional text detecting 
shadowing as suggested in my previous post.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UJKDGAJIIMNJYS2NYUNJ2XCAEFFLUTUC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve error message for trying to import itself

[Random832]

On Fri, Aug 21, 2020, at 21:52, Steven D'Aprano wrote:
> Having said all that, I cannot help but feel that shadowing of modules 
> is such an issue that maybe we need to make a special case of this. Not 
> specifically tkinter, of course, but if a module attempts to directly 
> import a module with the same name, and that import fails for any 
> reason, we should include a message about shadowing in addition to the 
> normal exception traceback.

I don't think a majority of these cases result in the error happening during 
the import rather than at a subsequent attribute access.

I think there are two possible paths here: either always print a warning [not 
an error] when importing a module from the current directory [the first entry 
in sys.path] that is also available in any other entry in sys.path, or check 
for that situation whenever a module attribute access fails, to add more 
information to the error message at that point.

I do think circular imports are a bit of a red herring - it's often the case, 
but not always. I've seen this happen as a result of a chaotic project 
directory whereby one script they'd written - which didn't itself depend on the 
named system module - shadowed a module that was imported from a different 
program.

> > I think the following would be an option:
> > "ImportError: partially initialized module 'tkinter' can't import itself"
> 
> That's a regression. In this specific case it doesn't matter because you 
> know that the error actually is shadowing (the user named their own file 
> "tkinter.py") but in the general case we lose the information of what 
> name was being looked up:
> 
> AttributeError: partially initialized module 'spam' has no attribute 
> 'egsg' (most likely due to a circular import)
> 
> If the error wasn't due to shadowing or a circular import, then knowing 
> that I had misspelled 'eggs' would be very useful.

hmm

what about

AttributeError: module 'spam' from './spam.py' has no attribute 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JD3DR4DRHKQ3OQAONRVXAP7SH3M4ZT3T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improve error message for trying to import itself

[Random832]

On Fri, Aug 21, 2020, at 15:13, Gustav O wrote:
> In the beginning of the programming journey, getting a message about 
> circular imports when they were testing out tkinter in a file names 
> "tkinter.py" would probably not be helpful. Getting a message that 
> hints that you may have accidentally imported itself, however, will 
> probably help significantly.
> 
> Maybe an error like this would be better, even though the wording could 
> be worked on:
> "AttributeError: partially initialized module 'tkinter' has no 
> attribute 'Tk' (most likely
> due to trying to import the file that is being executed)"

Detecting being partially loaded won't be enough for the shadowed file case, 
consider this minimal example of a file named tkinter.py:

import tkinter

if name == '__main__':
root = tkinter.Tk()

This file is loaded twice, once as __main__ and once as tkinter. The one that 
is loaded as tkinter and imported is *not* partially initialized, it finishes 
initializing with no errors.

It may be necessary to do more introspection (does the import machinery have a 
way to see if a module name exists twice on the path?) to determine when an 
AttributeError is caused by shadowing. It can also happen deep inside stdlib 
code, if one module whose name is not shadowed imports a shadowed module.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YJYJBRSUGBZ4MJCFP56OE34TCGHXEGYX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Thu, Aug 20, 2020, at 20:09, Steven D'Aprano wrote:
> This is likely to be a rare and unusual case, but we don't want to break 
> anyone who already has a dunder something like this:
> 
> def __getitem__(self, index, extra=None)

People do similar things with regular arguments to regular functions now, 
trusting callers not to pass stuff in for no reason that will break the 
function has been adequate there.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XYYOGRZ673D4S7YLGGZBMB6QJ4CXNC3U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Thu, Aug 20, 2020, at 09:46, Steven D'Aprano wrote:
> And what about all the objects that don't have a .value attribute? 
> What's so special about that attribute that subscripting with a missing 
> subscript should return that attribute rather than some other?

er, I meant "it should, on ctypes scalar objects specifically, have the same 
semantics as those classes' .value attribute", not "it should proxy to the 
attribute called 'value' on all objects"

sorry if that was unclear
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OBE2HTKNBI5QZUCB3TZU4MXDEN4FHK4A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Thu, Aug 20, 2020, at 08:56, Random832 wrote:
> I have an implementation proposal that I believe is distinct from any 
> of the ones mentioned in the PEP currently.

on further reflection, this seems mostly equivalent to the "kwargs argument" 
strategy [which I had wrongly read as __getitem__(self, arg, kwargs) instead of 
__getitem__(self, arg, **kwargs)], except I think the specification that it 
"should be keyword only" is unnecessary and harmful.

I think it would be perfectly reasonable to have a __getitem__(self, idx, 
option1=None, option2=None) which can be called as d[i, option1=foo] or d[i, 
option2=bar] - for a class which has a clear distinction between positional and 
keyword arguments there's no reason to force them to use the wrapper pattern I 
mentioned in my previous post.

Incidentally, "It doesn't preserve order, unless an OrderedDict is used" is out 
of place since PEP 468 was accepted.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MNEAYBTOSXKY6RTOEM3GK7RUQADGPEOK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Mon, Aug 17, 2020, at 14:00, Christopher Barker wrote:
> From an implementation perspective, the [] operator is another way to 
> call __getitem__ and __setitem__. And from that perspective, why not 
> have it act like a function call: no arguments, positional arguments, 
> keyword arguments, the whole shebang.
> 
> But from a language design perspective, the [] operator is a way to 
> "index" a container -- get part of the container's contents. And from 
> this perspective, no index makes no sense.

I think it makes perfect sense. Remember that numpy *currently* has a concept 
of "no index" [an empty tuple is used for this], it results in a view of the 
whole array, or the content as a scalar for a 0-dimensional array.

[I've occasionally been tempted to try the same thing on ctypes objects, 
incidentally, I think it might be useful to make obj[] equivalent to obj.value]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G4WY4R5N5BAEKCR4ECTBQLXPT33TMD4B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Fri, Aug 14, 2020, at 06:03, Jonathan Fine wrote:
> I'd like to sound out consensus regarding mapping access, where none of 
> the keys are positional. In particular, I suggest that PEP 472 allow 
> syntax and semantics such as
> >>> d[x=1, y=2] = 42
> >>> d[x=1, y=2]
> 42 
> and ask whether the class
> >>> X = type(d)
> should be part of standard Python.


I have an implementation proposal that I believe is distinct from any of the 
ones mentioned in the PEP currently.

Pass keyword arguments as ordinary keyword arguments [which any particular 
__getitem__ implementation is free to handle as **kwargs, specific keywords, or 
simple named arguments]. When a single positional argument is passed, it's used 
directly; when zero or two or more are passed, they are bundled into a tuple 
and passed as a single positional argument. Having zero arguments result in an 
empty tuple allows for easy conceptual compatibility with numpy.

d[]: d.__getitem__(())
d[0] : d.__getitem__(0)
d[0,1] : d.__getitem__((0, 1))
d[x=0]: d.__getitem__((), x=0)
d[0, y=1]: d.__getitem__(0, y=1)
d[0, 1, z=2]: d.__getitem__((0, 1), z=2)

if an implementation wishes to support a more conventional style of argument 
handling, the tuple can be deparsed easily in python code:

def __getitem__(self, arg, **kwargs):
return self._getitem(*(arg if isinstance(arg, tuple) else (arg,)))

def _getitem(self, x=slice(), y=slice(), z=slice()): ...

but an implementation could also define __getitem__ with any other signature 
that accepts the arguments outlined above, for example def __getitem__(self, 
arg, option) would effectively treat option as a keyword-only argument, and 
__getitem__(self, arg) would, as currently, not accept any keyword arguments.

and if __getitem__ itself defines named keyword args instead of **kwargs, or 
does not define anything at all, it will result in a TypeError in exactly the 
same way as if the calls outlined above were made directly, with only 
positional arguments being accepted and handled in the same way as they are 
currently.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JCIEMERDATRVSPQZGTNBFYHZGDVWY4OC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 472 - regarding d[x=1, y=2] and similar

[Random832]

On Sat, Aug 15, 2020, at 01:55, Steven D'Aprano wrote:
> Of course we would not. So why are we even considering a language change 
> to force every single subscriptable object to accept arbitrary 
> keyword-only arguments unless the maintainer changes their class to 
> explicitly reject them?

what?

why would they have to "explicitly" reject them?

it seems like the most obvious way to implement this would be to just... pass 
the keyword arguments. if you pass an argument to a function that doesn't have 
them in its signature it's a TypeError. Thus *automatically* "rejecting" them, 
with no effort required by the author.

right? if that's not the case, what exactly is being proposed?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DF545C5EZPUTTHLQQPE3FU54XFG36VUH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: encode/decode for all objects

[Random832]

On Thu, Jul 16, 2020, at 02:13, Stephen J. Turnbull wrote:
> Michael A. Smith writes:
> 
>  > It seems to me that obj.encode("json") and str.decode("json"), for example,
>  > would be a powerful feature,
> 
> This idea comes up a lot in various forms.  The most popular lately is
> an optional __json__ dunder, which really would avoid the complication
> of working with custom JSONEncoders.  That hasn't got a lot of takeup,
> though.  Perhaps we could broaden the appeal by generalizing it to
> obj.__serialize__(protocol='json'), but that looks like overengineering
> to me.

This kind of thing [especially having objects directly call their referenced 
objects' serialize methods rather than calling back to some other method to 
serialize them] seems like it would limit the ability of the protocol to handle 
with serialization formats that do anything to handle recursive or shared 
references. Particularly if we're serious about Pickle not being a viable 
foundation for secure deserialization, I think a new serialization protocol 
needs to be flexible enough to handle these cases.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CPOPVGEWL2RH6YUUFXO5U6IB5647DYEH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Sun, Jul 19, 2020, at 12:42, Stephen J. Turnbull wrote:
>  > Sure - it'd have to be a new opcode at this point,
> 
> Why?  The REDUCE opcode invokes load_reduce which ... oh heck, just post it:
>
>  > I just think the wrong decision was made in the first place,
> 
> Which "first place", the earlier pickle that had a restricted mode, or
> the "modern" pickle which based on that experience removed restricted
> mode?

er, I think you've lost some context - these particular statements were 
regarding an aside about a hypothetical opcode that would be defined to call 
__newobj__, which could be defined other than by calling __new__. The function 
could be written to inspect its own arguments more carefully than a general 
constructor, and having that name could constitute advertisement by the class 
that the function is safe to be called by unpickle.

It was the decision to call __new__ rather than __newobj__ in the NEWOBJ opcode 
that I was questioning.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3AAXFSZBDGYUAAM4HRADBHJOJPY7UXR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Sat, Jul 18, 2020, at 12:54, Stephen J. Turnbull wrote:
>  > I think I got all of them, but if you think there may be others
>  > feel free to be an extra pair of eyes. But these overrides are not
>  > available for the C version,
> 
> That's going to be a sticking point, as many pickle use cases want to
> be as fast as possible.  Additional overhead is likely to be
> unwelcome, although I guess the default would be minimal (I guess
> checking for the default of None and only calling if non-None would be
> fastest and do the job).

The *default* would be to just pass the call through as-is, e.g. def 
do_call(self, f, *a, **k): return f(*a, **k); or whatever is the equivalent C - 
my proposal is all just about having an internally called method that *can* be 
overridden, not defining anything special with it by default.

I guess part of where I'm not sure I'm on solid ground is... is the pure-python 
version guaranteed to always exist and always be available under the name 
_Unpickler, or is that an implementation detail? I've been assuming that there 
was no such guarantee and any change would have to be clearly defined and 
ultimately available in both versions.

> It certainly is *not* sufficient to be safe, if the threat model
> includes, say, a zero-day in the code being called, or an extended
> attack in which one allowed call sets the stage for another allowed
> call to blow up.  And it may be sufficient to be useless, depending on
> the use case.  You *are* going to die on *that* hill, you know.

Well, sure, anything can have bugs. I meant it's sufficient for it not to have 
any special vulnerabilities vs anything else you might ever do with python. I'm 
basically just trying to push back against "this is, like eval, the keys to the 
kingdom and thus not worth hardening in any way at all".

>  > On a mostly unrelated note I also have to admit I am baffled why
>  > the NEWOBJ opcodes are defined to call __new__ instead of
>  > __newobj__, when the latter is expected to exist and be a valid
>  > reduce function.
> 
> A lot of these decisions have implications for backward compatibility
> of pickles.  If you add code to check versions and decide whether to
> call __new__ or __newobj__, that has performance and complexity
> implications that may have been judged not worth the marginal[1]
> improvement in security.  Again, a request to change this seems likely
> to get pushback.

Sure - it'd have to be a new opcode at this point, and almost certainly isn't 
worth it... I just think the wrong decision was made in the first place, and 
we'd have more solid ground to design a version that doesn't require every 
application to provide its own specific filters if the decision had gone the 
other way. It doesn't matter at this point, I was just mentioning it as an 
aside.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OHSE4CJHMGIR5NEA7GBRHWPZAPTJADWE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Thu, Jul 16, 2020, at 02:36, Stephen J. Turnbull wrote:
> Random832 writes:
> 
>  > I was asking for the current Unpickler class, which currently has a
>  > whitelist hook for loading globals,
> 
> Callables are globals in this sense. 

not all callables are globals, as has been pointed out attributes of objects 
(methods) can also be callable.

this does require calling getattr which is itself global, but you can't 
exercise any fine-grained control over this without substituting your own 
getattr function, which creates problems if you are unpickling an object which 
contains a reference to the getattr function. Ultimately, *this* is the problem 
that made me realize that find_class isn't an adequate hook - that you cannot 
block or substitute a callable [whether that's a class, a global function, or a 
method] without also impacting the ability to unpickle objects that contain 
references to the same callable *as data*.

> So overriding
> Unpickler.find_class will allow you to restrict to specified
> callables.  It's not clear to me why you would want more fine-grained
> control: why not put the argument checking in the object constructor?
> In most cases that's probably where it's most useful, anyway, by DRY.
> 
>  > to be modified to also have a whitelist hook so that an application
>  > can provide a function that looks at a callable and its arguments
>  > that the pickle proposes to call, and can choose to either evaluate
>  > it, raise an error, or return a substitute value.
> 
> I would guess you can already have this by overriding
> Unpickler.load_reduce and patching Unpickler.dispatch[REDUCE[0]] to
> the new load_reduce.  Is there any other way for a pickle to specify
> the code to invoke on data supplied by that pickle?

I think I got all of them, but if you think there may be others feel free to be 
an extra pair of eyes. But these overrides are not available for the C version, 
and may well not be available on other python implementations.

>  > The idea that the pickle format is "inherently unsafe" and cannot
>  > be made safe is magical thinking.
> 
> I think you're quite wrong.  Pickle format itself is inherently unsafe
> because it allows a pickle to specify code to be executed on data that
> pickle specifies. 

Which is why an unpickler that *does not directly execute the specified code* 
is necessary and sufficient to be safe.

> Of course they already exist, and can be overriden.  I guess what
> you're asking for is a promise that the interface won't change in
> future versions of pickle.py.  That's the only difference between
> overriding Unpickler.find_class and overriding Unpickler.load_reduce
> (or any other method).

The other difference is that overriding find_class is supported by _pickle.c

[The dispatch mechanism is also unreasonably annoying to override, and having 
to override four separate methods, one of which is underscore-prefixed, to do 
the same thing, is not ideal]

On a mostly unrelated note I also have to admit I am baffled why the NEWOBJ 
opcodes are defined to call __new__ instead of __newobj__, when the latter is 
expected to exist and be a valid reduce function. Having a dunder name for a 
method that expects to be called from pickle seems like it would have been 
useful for security, since then you could add classes to a whitelist without 
allowing unrestricted calls to their constructor. Is this just a performance 
micro-optimization, or was there some other reason to prefer calling __new__ 
directly?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OVOGXXB2EQ7CJWRLWCR4HE2A337XTSNJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Wed, Jul 15, 2020, at 21:16, Chris Angelico wrote:
> Are you sure of that? I don't have any examples to hand, but are you
> able to pickle something identified as pkg.module.cls(x)?

This produces find_class('pkg.module', 'cls').

Doing pkg.module.cls.method produces find_class('builtins', 
'getattr')(find_class('pkg.module', 'cls'), 'method')

> > Second of all, with no way to exfiltrate, why is reading arbitrary 
> > attributes from objects problematic?
> 
> Because the moment you can read arbitrary attributes from arbitrary
> objects, Python becomes impossible to sandbox.

Not if you can't call them.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UFD7XKCG4JG2KGKGIQYCFY6RJ5RSHLIG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

[Random832]

On Tue, Jul 14, 2020, at 17:55, Rob Cliffe wrote:
> > What if "instead of a special kind of if clause that can only be placed 
> > after a loop", we simply defined these three special expressions [usable in 
> > any if/elif statement] to reference special boolean flags that are set 
> > after exiting any loop?
> The problem is: how long would these "special boolean flags" be retained?
> Could they still be tested
>      100 lines of code later (assuming no other loops were executed)?
>      After returning from a function?
>      Inside a new function call?
>          etc.
> This would violate the principle of ... I can't remember the computer 
> science name for it, but let's call it ... local transparency.
> Rob Cliffe

Sorry if this was unclear, but my idea is that they would be per-frame like 
local variables. Perhaps they could even be implemented *as* local variables, 
with assign statements emitted during loops by the compiler if they are used 
anywhere in the function. If that violates 'local transparency', so does using 
the same scope for ordinary local variables [such as, say, the iteration 
variable of a for loop, which can also be used 100 lines of code later].

And of course PEP 8 would forbid using them anywhere other than directly after 
a loop, but allowing them to be used in any if clause prevents you from having 
two different kinds of compound statement, which can't be mixed together, that 
both begin with the word "if".
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/D52HTWGHHPBVLQQBYK7O6BDWS6EFSKTF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Wed, Jul 15, 2020, at 07:40, Steven D'Aprano wrote:
> On Tue, Jul 14, 2020 at 09:47:15PM -0400, Random832 wrote:
> 
> > I was asking for the current Unpickler class, which currently has a 
> > whitelist hook for loading globals, to be modified to also have a 
> > whitelist hook so that an application can provide a function that 
> > looks at a callable and its arguments that the pickle proposes to 
> > call, and can choose to either evaluate it, raise an error, or return 
> > a substitute value.
> 
> Could you provide a proof of concept subclass?

I was thinking of something like this... this is largely a trivial modification 
of the pure-python unpickler, but there's no methods that can be overridden for 
this effect in the C one.


class MyUnpickler(pickle._Unpickler):
# this method is intended to be overriden by subclasses
def do_call(self, func, *a, **k):
#print(f"blocked call {func}(*{a}, **{k})")
#return None
raise NotImplementedError("This unpickler can't handle this pickle")

# these methods are defined the same as in _Unpickler except for the use of 
do_call
def _instantiate(self, klass, args):
if (args or not isinstance(klass, type) or
hasattr(klass, "__getinitargs__")):
try:
value = do_call(klass, *args)
except TypeError as err:
raise TypeError("in constructor for %s: %s" %
(klass.__name__, str(err)), sys.exc_info()[2])
else:
value = do_call(klass.__new__, klass)
self.append(value)

def load_newobj(self):
args = self.stack.pop()
cls = self.stack.pop()
obj = self.do_call(cls.__new__, cls, *args)
self.append(obj)

def load_newobj_ex(self):
kwargs = self.stack.pop()
args = self.stack.pop()
cls = self.stack.pop()
obj = self.do_call(cls.__new__, cls, *args, **kwargs)
self.append(obj)

def load_reduce(self):
stack = self.stack
args = stack.pop()
func = stack[-1]
stack[-1] = self.do_call(func, *args)

dispatch = pickle._Unpickler.dispatch.copy()
# load_inst and load_obj use _instantiate and don't need to be overridden 
directly
dispatch[pickle.NEWOBJ[0]] = load_newobj
dispatch[pickle.NEWOBJ_EX[0]] = load_newobj_ex
dispatch[pickle.REDUCE[0]] = load_reduce


def loads(s, /, *, fix_imports=True, encoding="ASCII", errors="strict", 
buffers=None, unpickler=pickle.Unpickler):
if isinstance(s, str):
raise TypeError("Can't load pickle from unicode string")
file = io.BytesIO(s)
return unpickler(file, fix_imports=fix_imports, buffers=buffers,
  encoding=encoding, errors=errors).load()
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PGYB6ARHJMUAC4TY4X2HXU4ANZ33KIUN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Wed, Jul 15, 2020, at 08:14, Chris Angelico wrote:
> That's fair, but are you actually guaranteeing that it will never read
> arbitrary attributes from objects? 

First of all, reading an attribute of an object in a pickle requires the 
getattr function. Even currently, you can substitute your own function for 
getattr in find_class, and with my proposal you wouldn't have to because you 
could control attempts to evaluate even the real getattr function.

Second of all, with no way to exfiltrate, why is reading arbitrary attributes 
from objects problematic?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4ZDXYU7JGEFWYNCO35FCB6OXBOGSXZAQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Wed, Jul 15, 2020, at 07:54, Edwin Zimmerman wrote:
> The idea that the pickle module can be made "safe" is magical thinking. 
>  Pickle's attack surface is just too large and too powerful.

I don't think that makes something *inherently* unsafe, it just makes it 
difficult to make it safe. The problem I have is with the idea that it is 
*conceptually inevitable* [in the same way as, say, eval] for it to be unsafe, 
and therefore that it's not worth fixing bugs or adding whitelist features or 
doing anything other than saying "oh well it's their fault for using pickle" 
if/when an exploit is found.

[that said, it might also be a worthwhile project to make an alternate 
"advanced de/serializer" that primarily works by creating empty objects [i.e. 
with object.__new__(cls)] and populating their slots/dictionaries by assignment 
rather than by executing any constructor code, though it would need special 
support for extension types with C structures]

> As I said in a previous message, a stupid pickle fuzzer I wrote several 
> years ago took about 60 seconds to start finding bugs (on an
> old slow-as-molasses single-core Intel Atom processor).  A more 
> intelligent fuzzer, on a much more powerful machine would probably
> do just as well today.  It would help slightly to throw out the _pickle 
> module and default to the pure Python version, but even then
> I wouldn't consider it anywhere close to secure.
> 
> That said, I agree with the idea of giving users an easier way to 
> control what pickle does.  I think that any such modifications
> should continue to make clear that pickle has not magically become 
> "safe".
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YR3WET2G5S42MUBXPN7BAFGAE7XYSJEH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Tue, Jul 14, 2020, at 21:24, Chris Angelico wrote:
> I actively oppose it because it isn't possible. Anything that is safe
> will not have all of pickle's functionality. A nerfed version of
> pickle that can only unpickle a tiny handful of core data types is no
> better than other options that already exist. The entire point of
> pickling arbitrary objects is that you can unpickle arbitrary objects.

I don't understand why no-one's engaging with what I actually suggested. I was 
not asking for a magically safe or arbitrarily restricted pickle function.

I was asking for the current Unpickler class, which currently has a whitelist 
hook for loading globals, to be modified to also have a whitelist hook so that 
an application can provide a function that looks at a callable and its 
arguments that the pickle proposes to call, and can choose to either evaluate 
it, raise an error, or return a substitute value.

> That's inherently unsafe if there is any possibility that the pickle
> file came from an untrusted user, and I do indeed oppose plans to try
> to make pickle what it isn't.

We already have one whitelist hook, why not another?

The idea that the pickle format is "inherently unsafe" and cannot be made safe 
is magical thinking. What I am asking for is the ability for application code 
subclassing Unpickler to control how certain opcodes are evaluated by 
overriding methods... something that *already exists*, just not for the right 
ones needed to be adequately expressive.

To the person who ran a fuzzer against it and found inputs that can cause 
segfaults or MemoryError, those are no more inherent to pickle than an 
equivalent bug in the JSON parser would be to JSON. They are bugs which can and 
should be fixed.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2RL6EKMOFZYU6SERHYDBOOS6G7OTFUJO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

[Random832]

> On 11/07/2020 06:22, Олег Комлев wrote:
> > ELSE-clause in FOR and WHILE has unclear syntax. I suggest new clause 
> > instead:
> 
> if COND:
>   ...
> [elif COND:
>   ...]
> [else:
>   ...]
> 
> This IF-clause like must be immediately after FOR- or WHILE-cycle (only 
> comment allowed between). It looks like a regular IF, but COND is 
> special.

On Sun, Jul 12, 2020, at 00:16, Rob Cliffe via Python-ideas wrote:
>  Something to consider: Would it be legal to mix these CONDs with other 
> conditions in the 'elif' chain? E.g.
>  if break:
>  ...
>  elif x > 0:
>  ...
>  elif finally:


What if "instead of a special kind of if clause that can only be placed after a 
loop", we simply defined these three special expressions [usable in any if/elif 
statement] to reference special boolean flags that are set after exiting any 
loop?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Z76AKHQUB2DZWBB3CQGHPIHQ7WUW5BYV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pickle security improvements

[Random832]

On Sat, Jul 11, 2020, at 20:15, Greg Ewing wrote:
> The set of callables that can be considered "safe" depends
> on the application, so there can't really be a generic
> "safe" option. If that were possible, it would no doubt
> already exist and be the default.

My main concern is wanting to make the, yes, application specific decision on 
whether calling a callable is safe *at call time* [and with access to the 
arguments e.g to determine if getattr is safe], rather than simply at the time 
a global is loaded.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W5OEXMDQBRVW26LGBLG4SYTW3D2QC5YZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Access (ordered) dict by index; insert slice

[Random832]

On Thu, Jul 9, 2020, at 13:26, Stestagg wrote:
> Obviously, python is a general-purpose, turing complete language, so 
> each of these options can be written in other ways. But it would be 
> nice if the simple, readable versions also worked :D
> 
> The idea that there are future, unspecified changes to dicts() that may 
> or may not be hampered by allowing indexing sounds like FUD to me, 
> unless there are concrete references?

Does the current implementation support indexing? It is certainly possible in 
principle to preserve ordering without indexing, for example if a linked list 
is used to support the ordering, or if items are stored in an array where 
deleted items leave holes permanently until the dict is resized.

I don't know how dict works, but I am not sure how you would support indexing 
while also allowing deletion to be O(1). A specific random key function would 
be narrower in scope than this, and for anyone who wants full sequence support, 
perhaps that could be added to OrderedDict.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/66KEODX2EPDIN5K4Y72DRRUW432V3O3S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[Random832]

On Thu, Jul 9, 2020, at 15:32, Dominik Vilsmeier wrote:
> On 09.07.20 21:04, Ethan Furman wrote:
> > I'm having a hard time understanding this line:
> >
> >    if lower == upper is not None:
> >
> > As near as I can tell, `upper is not None` will be either True or 
> > False, meaning the condition will only ever be True if `lower` is also 
> > either True or False, and since I would not expect `lower` to ever be 
> > True or False, I expect this condition to always fail.  Am I missing 
> > something?
> >
> It's operator chaining and shorthand notation for 
> (https://docs.python.org/3/reference/expressions.html#comparisons)
> 
>      if (lower == upper) and upper is not None:

If PEP-8 does not currently forbid using the shorthand notation in cases other 
than relational/equality operators in the same general direction [e.g. A > B == 
C >= D] or like equivalence operators [E is F is G; H == I == J], I think it 
should.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/JDNR63FOM2O6AKCKEW6SMSDI5VQNHMBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Pickle security improvements

[Random832]

The current practice, by overriding find_class, is limited to overriding what 
globals get loaded. This makes it impossible to distinguish globals that will 
be used as data from globals that will be called as constructors, along with 
similar concerns with object attributes [especially methods] obtained by 
loading builtins.getattr as global.

I would suggest also exposing for overrides the points where a callable loaded 
from the pickle is called - on the pure-python _Unpickler these are 
_instantiate, load_newobj, load_newobj_ex, and load_reduce, though it might be 
worthwhile to make a single method that can be overridden and use it at the 
points where each of these call a loaded object.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BB2TLAF6YSE5PEDQCPLWATDTUYPNXA4D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add __eq__ to colletions.abc.Sequence ?

[Random832]

On Mon, Jul 6, 2020, at 01:47, Neil Girdhar wrote:
> Are all objects in Python equality-comparable? I know that you can 
> delete __hash__ to make an object unhashable (e.g., dicts). If so, this 
> is a great addition.

Anyone can in principle override __eq__ to throw an exception, but they're not 
"supposed to" - the default behavior is that an object is only equal to itself, 
and floating point NaNs aren't equal to anything including itself which isn't 
very useful, but in all cases the operation itself is valid and simply returns 
false e.g. when the other operand is a different type rather than treating it 
as any kind of error.

Which of course means that, right now, a sequence that does not define its own 
__eq__ method is equal only to itself, rather than it being an error to try to 
compare it.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PZXXP2IBQ7IFOS4JV3OGDZNNKTNAWC76/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add builtin function for min(max())

[Random832]

On Sat, Jul 4, 2020, at 15:57, 2qdxy4rzwzuui...@potatochowder.com wrote:
> > Simplifying the signature, in Python we have:
> > 
> > def min(*iterable):
> > iterator = iter(iterable)
> > minimum = next(iterable)
> > for item in iterator:
> > if item < minimum:
> > minimum = item
> > return minimum
> > 
> > Due to this, min(0, float('nan')) == 0 and same for max. I would hence
> > expect clamp to behave similarly.
> 
> Yuck:  We also have min(float('nan'), 0) == float('nan').
> 
> I'm not sure what I'd expect a hypothetical clamp function to do.
> Someone with actual use cases will have more insight.

IEEE 754-2019 defines minimum and maximum functions that return NaN in all 
cases, and apply a strict ordering to signed zero... however, there is also a 
minimumNumber and maximumNumber which returns the number if the other operand 
is NaN [the asymmetric behavior depending on the order of the operands isn't 
allowed by either]

it might be worthwhile to define a "min2" etc that applies the rules for one of 
these functions when both arguments are floats [and possibly when one argument 
is a float and the other is any numeric type], and then define min(*iterable) 
as:

def min(*iterable):
iterator = iter(iterable)
minimum = next(iterable)
for item in iterator:
minimum = min2(minimum, item)
return minimum

I can't find anything about a clamp-like function in IEEE.

It may be worth surveying what other implementations do... to start with:
- Rust has a clamp function that returns NaN if the given number is NaN, and 
"panics" if either boundary is NaN.
- Numpy's clip function effectively accepts NaN or None boundaries as "don't 
care"
- - This appears to be implemented as min(max(x, a), b), with min and max 
themselves having the asymmetric behavior.
- C++'s clamp function seems to be undefined if any of the operands are NaN
- - T must meet the requirements of LessThanComparable in order to use 
overloads (1).
- - However, if NaNs are avoided, T can a be floating-point type. 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HUJFXYVWWZRNQVM7SAJY2MS3TFFC5P3Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Sat, Jul 4, 2020, at 14:27, William Pickard wrote:
> There is only 2 ways an extension is distributed to people in the 
> Python universe:
> As a SOURCE CODE distribution OR a COMPILED BINARY distribution.
> 
> Wheels are generally the "compiled" distribution, these are also 
> generally built for a specific runtime + python version.
> For Python 3.10 wheels, these would already be compiled against the 
> precompiled header.
> 
> For source distributions, they require "building" before being 
> deployed, for Python 3.10, this will include the precompiled header.

My point was that I still don't understand what the benefit of the precompiled 
header is, if extensions still have to distribute source to get the benefit of 
it. This is why I assumed you *didn't* think extensions would have to 
distribute source. So, how is it better than a normal header? If the point of 
an opaque PyObject isn't to allow the same compiled distribution of an 
extension to be used with different versions of python that have different 
implementations of PyObject then... well, what *is* the point? Why not just 
recompile normally, with the textual header?

> I'm aiming at least to not increase the API overhead for when invoking 
> stuff like "Py_TYPE", "Py_INCREF", "Py_DECREF", etc.
> Py_INCREF and Py_DECREF are the worst offenders as they are generally 
> HIGHLY INVOKED functions and are also the functions that are ALWAYS 
> INLINE OPTIMIZED away by modern compilers.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2D7UMEJBQ6IQTT7CWG6FHKX73GJBCYTS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Sat, Jul 4, 2020, at 13:22, William Pickard wrote:
> Oh and the other question:
> 
> Yes, a conversion function will be required to utilize the value of the 
> offset member, the conversion is sadly a one way affair BUT CPython's C 
> API is handy enough that a sacrificial 'PyObject *' stack variable can 
> exist, most compilers may end up just optimizing the variable away 
> anyhow.

hmm...

If only single inheritance is still allowed, there's no reason you couldn't 
support the other direction by subtracting the offset. 

Supporting multiple inheritance [of two different base classes that both define 
structures, anyway - something that's not currently allowed in cpython, but 
might be interesting to add support for] would require knowing the runtime type 
of the object, but I don't think your code is currently doing anything to 
support that case.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RNLTR3A6GVZIOM7JJM7FLXJ3TK4OVLJW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Sat, Jul 4, 2020, at 13:19, William Pickard wrote:
> A precompiled header is a combination of a plain header with a 
> companion special file (.pch/.gch).
> The companion file is generated from a source file that is designated 
> as the Precompiled Header creator ('/Yc' on MSVC).
> 
> Every other source file is told to use the special file ('/Yu' on 
> MSVC), the source file compilation will fail if the special file is 
> missing.
> 
> CPython/third party runtimes will only need to ship this special file 
> with the compiled code, the only downside is a burden of checking the 
> checksum of the file before it's used in a compile process.

But this file can only be used *with the source code* of extension libraries, 
so shipping this file isn't any better than shipping an ordinary text-based 
header file (which is, obviously, already done). 

You seem to believe that the pch/gch file can be used somehow to adapt 
already-compiled extension libraries to each implementation. I do not think 
this is true at all, certainly it doesn't seem to be implied by anything in the 
documentation at https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html .
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZSEWKVKR2AIMWQM3TL33NF4NRUJRXLFP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add __eq__ to colletions.abc.Sequence ?

[Random832]

On Sat, Jul 4, 2020, at 12:30, Joao S. O. Bueno wrote:
> As far as types are concerned, the `__eq__` should worry about it - 
> just Sequences that are
> a subtype of other, or the other being a subtype of one, should be able 
> to compare equal 
> (As happens with lists and tuples as well: subclasses of both will 
> compare equal to base lists and tuples
> with the same content.). 

As will two different subclasses of list. Should we try to replicate this 
behavior as well (i.e. determine the the "base sequence type" somehow, so that 
two subclasses of the same one can be compared and others cannot)?

This is something that we have to get right the first time; equality 
comparisons can't be made more permissive later since they simply return False 
rather than being an error... otherwise I might be advocating to "fix" list and 
tuple.

> The code for that is on my first reply to Guido, above:
>  if not issubclass(type(other), type(self)) and not 
> issubclass(type(self), type(other)):
>  return False
> 
> I am actually using that on the file that motivated me sending the 
> first e-mail here -as it
> makes sense in that project. 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IICSIZM4ASSEGNMZW6D56HN4YKIPVSPM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Sat, Jul 4, 2020, at 12:48, William Pickard wrote:
> For backwards compatibility, PyTypeObject will eventually have the 
> flag: Py_TPFLAG_OMIT_PYOBJECT_SIZE, but from what I can tell, 
> eventually all extensions should be using PyType_FromSpec/it's variants.

Er... I don't mean how do they create the PyType. I mean how do they create the 
actual data type they use?

https://docs.python.org/3/extending/newtypes_tutorial.html#adding-data-and-methods-to-the-basic-example

What does the definition of CustomObject look like? How do I access the 
"number" member, given a PyObject *? Can I still cast the PyObject * to 
CustomObject *, or do I have to go through some conversion function?

> I'm now leaning towards replacing the Static Library with a precompiled 
> header (Those will not users from building DLLs as they're just a plain 
> headers compiled to machine code used differently from obj files, I 
> BELIEVE at least, haven't tested inlinability yet).

My understanding is that you can't combine the precompiled header with a dll or 
even obj file, you have to combine it with source code, which means requiring a 
precompiled header prevents extension libraries from being shipped as binaries.

Requiring modules to be rebuilt from source code to support any changed 
definition of PyObject, as far as I can tell, defeats the entire purpose of 
making it opaque.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LDJEKJ2F4W6GLT623R27NIPGI5AZYQNL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Sat, Jul 4, 2020, at 11:59, William Pickard wrote:
> CPython PR #21262, GitHub username is Wildcard65.

ok looking at that code, I'm not sure I understand how it addresses the 
PyObject_HEAD thing, unless there's something I'm not seeing

I'll ask directly - how would an extension library define a type, and how would 
it access its own data at the end of objects of that type?



I'm also skeptical that a static library would provide any savings vs treating 
the functions like every other function in the api... a precompiled header 
might, but while I don't fully understand how precompiled headers are used, I 
don't *think* it would allow extension libraries to ship as a dll/so file as 
they currently can.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NJNKZ4473AHEUD52YZ7Y5DAFJS7TOXVL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add __eq__ to colletions.abc.Sequence ?

[Random832]

On Fri, Jul 3, 2020, at 03:57, Wes Turner wrote:
> Can a Sequence be infinite? If so, an equality test of two 
> nonterminating sequences would be a nonterminating operation.

I think not - an infinite sequence would make len, contains, and reversed 
ill-defined (it also wouldn't allow certain kinds of slices)

> Do Sized and *Reversible* imply that a sequence terminates?
> Could __len__ return inf?

__len__ must return an integer.

> Perhaps `Ordered` is a requisite condition for defining a comparator 
> for Sequences.
> `OrderedSequence`?
> 
> Are there unordered Sequences for which a default `__eq__` / `__cmp__` 
> (et. al) would be wrong or inappropriate?

I don't think so [index as a mixin implies being ordered, i think]... the 
bigger problem is the one I mentioned earlier, that allowing comparison between 
sequences of different types is inconsistent with tuple and list.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LMR7PSLJHSYQFMMWEAW337G3YBXC437U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: giving set.add a return value

[Random832]

On Mon, Jun 29, 2020, at 04:51, Stephen J. Turnbull wrote:
> Of course, we may prefer a boolean return, despite the general rule
> about returning elements.  I'm single-threaded, and so agnostic on
> that. :-)  But if it turns out that somebody *wants* to check "2 is 
> 2.0",
> this .add_unique can serve both purposes.

I don't think this is a good idea - for one thing, an element return means you 
can't handle, or have to handle specially, None being in a set.

If we want a way to check the type or identity of an item in a set, I think it 
would be best to add another method to access the item. Or an operator - i.e. 
check if type(set[2]) is int or float, or whatever it is you want. I did once 
write a way to do that in O(1) time in unmodified python [abusing the __eq__ 
method of a custom class to  smuggle the value out] as an exercise, 
incidentally, and it seemed to work fine.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NBMII2CZRA7SSZSRMH3LTDUECUQV3BGB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add __eq__ to colletions.abc.Sequence ?

[Random832]

On Wed, Jul 1, 2020, at 08:23, Joao S. O. Bueno wrote:
> collections.mixins.SlicedSequence that would override `__delitem__`, 
> `__setitem__` and `__getitem__` and 
> handle slices could pair up with the "ComparableSequence" - people 
> could use these "a la carte", and 
> no backwards compatibility would be hurt.

This one raises the question of where you put the single-item accessors. And 
sliced __delitem__ may be difficult to implement efficiently without knowing 
the internals of the sequence type.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7HLNBVJSVA5S5EDOK7LFCQNWUMOTJXLS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Thu, Jul 2, 2020, at 20:39, William Pickard wrote:
> I do have a plan to solve that problem, it involves adding about 24 
> more bytes to PyTypeObject, but it would allow Python to track offset 
> from base object, total object size and best alignment value.

I'm not sure I'm following how that would help, but it sounds like you're 
saying that pointers to extension-defined data structure would not be 
PyTypeObject pointers, and vice versa? At least not without interconversion 
using this offset, which adds more overhead.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KLN5GG26WXWUTJ7R3RT2XYNY545QE4PB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Best approach for opaque PyObject

[Random832]

On Thu, Jul 2, 2020, at 16:04, William Pickard wrote:
> The goal as it stands is to make PyObject an opaque type (interpreted 
> to mean, "incomplete type"), but there is a blocking problem to 
> accomplishing that.
> 
> Functions/Macros like PY_TYPE require a complete PyObject.

Is it acceptable if the performance hit only applies to external [not part of 
the python implementation] callers? The macro could be defined as to call an 
exported function from external code but as its current definition within the 
python implementation.

I do think there's a much bigger problem, though... if PyObject is opaque, then 
extension modules cannot define their own types, since the definition of any 
extension type has to begin with PyObject_HEAD. It could be replaced by a 
"semi-opaque" definition of the same size and alignment, but would that solve 
whatever problem making PyObject opaque is intended to solve?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SCKWJFK4BFU7Q25O4OYSULYKRQQMZO4F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add __eq__ to colletions.abc.Sequence ?

[Random832]

On Tue, Jun 30, 2020, at 10:57, Guido van Rossum wrote:
> I don't recall why this was done. It seems somewhat odd, since Set and 
> Mapping in the same module do have __eq__. I don't care much for the 
> default implementation though.
> > 
> > Anyway, maybe there is a reason it is not a given. Any thoughts?

One thing that may be worth considering is that tuples and lists with the same 
respective contents are not equal to each other [whereas sets and frozensets 
are]

I do think it might be worthwhile to have a "compare two sequences" [and 
possibly also "hash a sequence", to match the tuple hash without making a 
tuple] building block as a function somewhere, so people could relatively 
easily make their own [perhaps even something like "__eq__ = 
collections.sequence_eq"]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L2L7FVQFHTMSOU6BE6VGFWKTIORIBPNO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: prefix/suffix for bytes (was: New explicit methods to trim strings)

[Random832]

On Sat, Mar 7, 2020, at 19:31, Cameron Simpson wrote:
> >I *think* I understand the issues. And I can see that some software would
> >need to work with filenames as arbitrary bytes. But that doesn't mean that
> >you can do much with them that way.
> 
> Given that the entire UNIX filename API is bytes, I think this isn't 
> very true.

Most real-world UNIX systems only support ASCII-compatible encodings. There's 
no reason not to solve the problem on such systems by using os.fsdecode().

On those few that do not (of which I don't know if any support *both* ASCII and 
non-ASCII-compatible encodings in locales - from what I can find, those that 
don't use ASCII-compatible encodings tend to exclusively use EBCDIC ones) I 
don't know how they handle these cases, or if python even supports any of them 
at all, but it seems likely that b'/' will not be the same byte as the path 
separator.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MKOPNOW6S4PLTDYHQDD4RDDWO5GQXZ5K/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: statement-scoped context managers

[Random832]

On Sat, Feb 8, 2020, at 18:06, Random832 wrote:
> My own expectation, for what it's worth, would be something like
[snip]

After thinking about it some more, I realized that my idea can basically be 
translated to "all with-expressions in the statement get added to an implicit 
ExitStack, which is then cleaned up after the statement."

> It's also tempting to try to define a way to, e.g. only include it in 
> scope for the evaluation of the condition in if statements and while 
> loops.

On further reflection, I don't think this is worth it in the general case - 
though the idea of a context manager being opened in each iteration of a while 
loop's condition not being closed until the end of the loop is concerning.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MYSSES3BOYG4XKW2UXLF5KRQM36LSVLF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: statement-scoped context managers

[Random832]

On Sat, Feb 8, 2020, at 17:14, Soni L. wrote:
> On 2020-02-08 6:53 p.m., Bruce Leban wrote:
> > On Sat, Feb 8, 2020 at 1:22 PM Chris Angelico  wrote:
> >>  Exactly how much code would be wrapped in the 'with' block?
> > 
> >  This is an intriguing idea, and in the example it's fairly easy to wrap 
> > the entire statement in the with block. It gets a bit more complicated with 
> > short-circuit logic. This is a contrived example to make it easier to read:
> > 
> >> result = (with alpha()) and ((with beta()) if (with gamma()) else (with 
> >> delta()))
> > 
> > needs to be interpreted something like:
(snip)
> > I don't think there's anything surprising there although precisely defining 
> > the semantics will be a little tricky.
> 
>  I'd expect it to go more like
(snip)

My own expectation, for what it's worth, would be something like

try:
  _alpha_set = _beta_set = _gamma_set = _delta_set = False
  result = (_alpha_cm := alpha(), _alpha_set:=True)[0].__enter__() and 
((same transform for beta) if (...gamma) else (...delta))
finally:
try:
if _delta_set: _delta_cm.__exit__()
finally:
try:
if _gamma_set: _gamma_cm.__exit__()
finally: ...

but this is more of a mess than I originally thought to define in scenarios 
with multiple and/or conditionally-used context managers. It's also tempting to 
try to define a way to, e.g. only include it in scope for the evaluation of the 
condition in if statements and while loops.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YJ5Q7BGERLC6L2D3IDZ6PGUCW2O6WIPZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] statement-scoped context managers (was: Re: Pickle to/from filename or path)

[Random832]

On Fri, Feb 7, 2020, at 13:03, Todd wrote:
> approaches like opening within the dump or load, which the wiki 
> still recommends [1].
> 
> So something like:
> 
> with open('myfile.p', 'wb') as f:
>  pickle.dump(myobj, f)
> 
> Would be:
> 
> pickle.dump(myobj, 'myfile.p')


What if you could write pickle.dump(myobj, with open('myfile.p', 'wb'))?

Or other similar examples such as (with open('myfile')).read() - have the 
compiler automatically transform the code into equivalent to wrapping the 
statement in a with block.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/S46XU2SZYCH5PA3AOLODI2W7Y2J6P3QI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: !$? operators for gratuitous overloading

[Random832]

On Sun, Feb 2, 2020, at 11:41, MRAB wrote:
> On 2020-02-02 15:00, Karl Ramm wrote:
> > We have all shared in the observation that all but the most carefully
> > considered operator overloading tends to reduce code readability.  There
> > also many programmers that feel that they need it to make there code terse
> > (or perhaps shiny) enough.
> > 
> > I propose adding ! and $ as (normally unimplemented) binary operators and ?
> > as a unary operator so as to explicitly to give people the rope they want
> > but clearly tagged "here there be shenanigans".
> > 
> In the past there have been suggestions to add None-coalescing 
> operators 
> such as "?." and "??", comparable to the null-coalescing operators of 
> C#, so I'm -1 on just adding "!", "$" and "?" without a solid use-case, 
> in case we find a better use for them later in the future.

Before the nameof discussion got shut down, I was going to propose $nameof as a 
way to avoid collision with a function called "nameof", and I still think it's 
viable as a general way to make an open-ended set of keywords that don't 
collide with identifiers [frozenset literals?]... so I'm also -1 on cutting 
down the number of reserved ASCII characters without a very good concrete use 
case.

If we are going to have a general binary operator mechanism, maybe it should be 
something more haskell-like, with an open-ended set of binary operator names 
[`identifier` as haskell, perhaps? and/or maybe the unicode math ops section, 
with stuff like circled plus] and a way to define precedence before the point 
of use for the parser.

Perhaps it would also be useful to provide a utility function for performing 
the __op__/__rop__/NotImplemented/most-derived-class binary operator execution 
rules on an arbitrary pair of method names, without *any* syntactical support 
[in order to, say, allow evaluation of custom ASTs for non-python languages on 
python objects]
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GISGMZLBZ3FRT77QWJ3EXUGJTGJC2MEG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: addition of "nameof" operator

[Random832]

On Tue, Jan 21, 2020, at 16:32, Andrew Barnert via Python-ideas wrote:
> What would the semantics of nameof be in Python? Would it just be 
> lambda obj: obj.__name__? Or some fancy inspect-module style chain of 
> “try this, then that, then the other”? Or does it need to look at the 
> compiled source code context or something?

The C# version becomes a string constant and doesn't have to look at anything 
other than the expression itself. It is the literal name of the variable [or 
attribute] in the expression, irrespective of its value.

The intent is for it to be a string literal that survives refactors, so you 
could do nameof(a_person.age) and that would compile down to "age" [without 
actually performing an attribute access. C# allows you to do this with instance 
members without making an instance, but it is unclear what the best way to do 
this in python would be], but if you used a refactoring tool to rename the age 
attribute, it would change this expression along with all of the other places 
it is used, to give it the new attribute name.

There's no fancy run-time lvalue reflection stuff going on as you suggested in 
your other post, it is just the name from the expression, as a string constant. 
It is purely a compile time operation (which is why it has to be an operator, 
not a function)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7IZRVFY2QB5LFDYL4MWL7L6I7GRV5SFP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Compound statement colon (Re: Re: Improve SyntaxError for obvious issue:)

[Random832]

On Fri, Jan 17, 2020, at 21:32, Josh Rosenberg wrote:
> The colon remains syntactically necessary in some cases, particularly 
> to disambiguate cases involving one-lining (no block involved). Stupid 
> example: If the colon is optional, what does:

I was only proposing making it optional in the multi-line case.

The following in the grammar:

[anywhere it appears]: ... ':' suite
suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT

would be replaced with

[whatever]: ... compound_body
compound_body: ':' simple_stmt | [':'] NEWLINE INDENT stmt+ DEDENT

(I don't understand how TYPE_COMMENT interacts with NEWLINE/INDENT/DEDENT well 
enough to do the same for func_body_suite in terms of the actual grammar)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UGS2WPCEUWVOZFRZ6W2ZSDVN44OMSDG4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow relative seek on StringIO

[Random832]

On Thu, Jan 16, 2020, at 23:07, Guido van Rossum wrote:
> Sounds like you should just submit a bug report (and a PR with a fix if 
> you feel up to it). Since a relative seek to position p is typically 
> just implemented as an absolute seek to position f.tell()+p, this looks 
> like an odd omission, and I can't remember a reason for it. Looking at 
> the code, the fix looks pretty easy -- update _io_StringIO_seek_impl() 
> in Modules/_io/stringio.c.

I guess my instinct was to post here first in case there was (or in case 
someone thought there was) a reason, since it's certainly something that's 
being specifically disallowed in the code rather than a consequence of 
something that's not checked properly.

Submitted as issue 39365.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KRXGIC4HACJQRRZ4VLT6CRASQDVYJCUF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Compound statement colon (Re: Re: Improve SyntaxError for obvious issue:)

[Random832]

On Tue, Jan 14, 2020, at 18:15, David Mertz wrote:
> For what it's worth, after 20+ years of using Python, forgetting the 
> colon for blocks remains the most common error I make by a fairly wide 
> margin. Of course, once I see the error message—even being not all that 
> descriptive of the real issue—I immediately know what to fix too.

What if the colon were made optional, with an eye to perhaps eventually no 
longer using it as the preferred style for new code?

We had a post a while ago about the possibility of using the lack of a colon as 
an implicit line continuation (like with parentheses, e.g. "if a\nand b:", and 
this was (reasonably) rejected. But if a line beginning as a compound statement 
and ending without a colon is *never* going to have a valid meaning as 
something else... what's the point of the colon, otherwise? Seems like just 
grit on the screen.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/J47TEY2KFGATFMQ7RSZJO7B4RV7KEYWJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: python -m quality of life improvements

[Random832]

On Sat, Jan 11, 2020, at 02:06, Andrew Barnert via Python-ideas wrote:
> Well, there are definitely subtle differences. If it’s a single file 
> rather than a package, running it as a script means you have to include 
> the extension; with -m you can’t. If you -m a package, argv[0] is the 
> path to its __main__.py; if you run a package as a script it’s the path 
> to the package. If you -m the parent directory of the package obviously 
> has to be on sys.path, but if you run it as a script, it generally 
> won’t be. There might also be differences with which paths (argv[0], 
> __file__, __cached__, etc.) get abspathed on each platform (I can never 
> remember the rules for that anyway). Do they both work with namespace 
> packages? Fire the same auditing events? Interact the same way with the 
> Windows py.exe launcher and shbang lines? Does -m ignore case on *nix 
> on case-insensitive filesystems? What if you’ve got weird stuff in your 
> metapath from a site-installed import hook? I don’t know the answers 
> for most of these.

speaking of executing packages from the command line... I noticed a different 
set of anomalies (I was going to suggest this as a solution or workaround, but 
it turned out to be non-viable).

If you specify a directory name as the primary script argument to python 
*without* -m:
- __main__.py will be executed
- argv[0] will be the dir name as-is, not a path to __main__.py
- sys.path[0] is the given directory
- __package__ is *the empty string*. [under normal circumstances for a module 
not in a package, __package__ is None. However, the empty string here does not 
allow from . import to work]

I expected:
- __main__.py will be executed
- I had no particular expectations regarding argv[0], but I believe this turns 
out to be *only* circumstance under which it can name something other than a 
python file
- sys.path[0] to be the parent directory of the given directory
- __package__ to be the name of the directory

Also note that executing a zip file on the command line behaves similarly to 
the actual behavior described above, and there is no way to achieve behavior 
analogous to what I expected because there is no clear way to treat a zip file 
as a package. I have not considered the implications regarding zip files 
further.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SP6BK5P7QXIEU7LWUK4HQFBPQCR4I2MM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: timezone-aware datetime.now

[Random832]

On Sat, Jan 4, 2020, at 15:11, Random832 wrote:
> ...

I just discovered, in the course of trying to make a more fleshed-out 
implementation, that the correct way to do this is 
datetime.now().astimezone(None) - this isn't explained very well in the 
documentation.

I did notice, though that a datetime created in this way does not return 
anything for dst(). And the part where I think naive datetimes should be able 
to answer utcoffset(), tzname(), and dst() using the system local time still 
stands. I think that calling one of these functions on a naive datetime is 
likely to be a mistake in any code that does so now, and that calling one of 
them check if it is naive seems unlikely vs just checking obj.tzinfo is None.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G7SZNL6E5KIWUJUHHVHSUGV2SHRRDQIU/
Code of Conduct: http://python.org/psf/codeofconduct/