from:"James Lu"

It’s been almost a week since this “discussion” first started. Can we please 
stop this in the name of productive work on python-ideas? Frankly, you don’t 
need to reply just because you can point out something wrong with someone 
else’s argument. Post because it’s worthwhile to hear, not because you have 
something to say.

This mindless and combative attitude is a big reason why Guido was motivated to 
suspend himself.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Moving to another forum system where moderation is possible

I agree completely. 

I propose Python register a trial of Stack Overflow Teams. Stack Overflow Teams 
is essentially your own private Stack Overflow. (I will address the private 
part later.) Proposals would be questions and additions or criticism would be 
answers. You can express your support or dissent of a proposal using the 
voting. Flags and reviews can be used to moderate.

Stack Overflow Chat can be used for quick and casual discussion, and also to 
move irregular or irrelevant discussions away from the main site. 

Although the Stack Overflow platform is typically used for technical Q&A, there 
is precedent for using it as a way to discuss proposals: this is precisely what 
Meta Stack Overflow dies and it’s seen decent success. 

Anyone can register a @python.org email. Stack Overflow Teams can be configured 
to allow anyone with a @python.org email join the python-ideas team. 

I’m sure Stack Overflow Inc. is willing to grant Stack Overflow Teams to the 
PSF pro bono after the trial period expires. 

You can configure stack overflow to get email notifications as well. 

> On Sep 17, 2018, at 1:16 PM, Anders Hovmöller  wrote:
> 
> 
>> It’s been almost a week since this “discussion” first started. Can we please 
>> stop this in the name of productive work on python-ideas? 
> 
> A better use of time might be to discuss moving to a better forum system 
> where moderation is easier/possible. Email somehow has a shape that makes 
> those things 100% probable and you can’t easily silence discussions that are 
> uninteresting. 
> 
> / Anders
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where moderation is possible

> It was decided to try https://www.discourse.org at the core dev
> sprints.  We'll likely try it for the upcoming governance model/vote
> discussions.  If it works well we'll consider using it for other
> discussions in the future.
> 
> Let's table this topic for now as we're unlikely to

So... we’re going to be using discourse instead of Python-ideas mailing list? 
Or will we only try that until Discourse works for “core sprints”?

> On Sep 17, 2018, at 3:34 PM, Yury Selivanov  wrote:
> 
> It was decided to try https://www.discourse.org at the core dev
> sprints.  We'll likely try it for the upcoming governance model/vote
> discussions.  If it works well we'll consider using it for other
> discussions in the future.
> 
> Let's table this topic for now as we're unlikely to
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where moderation is possible

How can the Zulip chat be joined? Im interested in consolidating all the 
discussion into one centralized forum. 

Sent from my iPhone

> On Sep 17, 2018, at 3:35 PM, Philippe Godbout  wrote:
> 
> Also, by restricting to python.org email address, do we not run the risk of 
> cutting off a lot of would be contributor?
> 
>> Le lun. 17 sept. 2018 à 15:23, Abdur-Rahmaan Janhangeer 
>>  a écrit :
>> py already has a Zulip chat
>> 
>> Abdur-Rahmaan Janhangeer
>> Mauritius
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

2018-09-18 Thread James Lu

> Is that really an issue here? I personally haven't seen threads where
> Brett tried to stop an active discussion, but people ignored him and
> kept fighting.
Not personally with Brett, but I have seen multiple people try to stop the 
“reword or remove beautiful is better than ugly in Zen of Python.” The 
discussion was going in circles and evolved into attacking each other’s use of 
logical fallacies. 

Other than that, my biggest issues with the current mailing system are:

* There’s no way to keep a updated proposal of your own- if you decide to 
change your proposal, you have to communicate the change. Then, if you want to 
find the authoritative current copy, since you might’ve forgotten or you want 
to join he current discussion, then you have to dig through  the emails and 
recursively apply the proposed change. It’s just easier if people can have one 
proposal they can edit themselves.
  * I’ve seen experienced people get confused about what was the current 
proposal because they were replying to older emails or they didn’t see the 
email with the clear examples.
* The mailing list is frankly obscure. Python community leaders and package 
maintainers often are not aware or do not participate in Python-ideas. Not many 
people know how to use or navigate a mailing list.
  * No one really promotes the mailing list, you have to go out of your way to 
find where new features are proposed. 
  * Higher discoverability means more people can participate, providing their 
own use cases or voting (I mean using like or dislike measures, consensus 
should still be how things are approved) go out of their way to find so they 
can propose something. Instead, I envision a forum where people can read and 
give their 2 cents about what features they might like to see or might not want 
to see. 
   * More people means instead of having to make decisions from sometimes 
subjective personal experience, we can make decisions with confidence in what 
other Python devs want. 

Since potential proposers will find it easier to navigate a GUI forum, they can 
read previous discussions to understand the reasoning, precedent behind 
rejected and successful features. People proposing things that have already 
been rejected before can be directed to open a subtopic on the older 
discussion. 

> On Sep 18, 2018, at 3:19 PM, python-ideas-requ...@python.org wrote:
> 
> Is that really an issue here? I personally haven't seen threads where
> Brett tried to stop an active discussion, but people ignored him and
> kept fighting.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

2018-09-18 Thread James Lu

It would be nice if there was a guide on using Python-ideas and writing PEPs. 
It would make it less obscure. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

2018-09-19 Thread James Lu

Oh wow, Google Groups is actually a much better interface.

Any better forum software needs a system where people can
voluntarily leave comments or feedback that is lower-priority.
I'm not sure if Discourse has this, actually. Reddit comments
are extremely compact as are Stack Overflow comments.

I was going to propose that the PSF twitter account post a
link to https://groups.google.com/forum/#!topic/python-ideas/,
but I was worried that getting more subjective personal
experiences might undesirably decrease the signal-to-noise
ratio.

On Wed, Sep 19, 2018 at 12:48 AM Franklin? Lee <
leewangzhong+pyt...@gmail.com> wrote:

> On Tue, Sep 18, 2018 at 8:21 PM James Lu  wrote:
> >
> > > Is that really an issue here? I personally haven't seen threads where
> > > Brett tried to stop an active discussion, but people ignored him and
> > > kept fighting.
> > Not personally with Brett, but I have seen multiple people try to stop
> the “reword or remove beautiful is better than ugly in Zen of Python.” The
> discussion was going in circles and evolved into attacking each other’s use
> of logical fallacies.
>
> I disagree with your description, of course, but that's not important
> right now.
>
> Multiple people *without any authority in that forum* tried to stop a
> discussion, and failed. Why would it be any different if it happened
> in a forum? Those same people still wouldn't have the power to lock
> the discussion. They could only try to convince others to stop.
>
> If the ones with authority wanted to completely shut down the
> discussion, they can do so now. The only thing that a forum adds is,
> when they say stop, no one can decide to ignore them. If no one is
> ignoring them now, then locking powers don't add anything.
>
> > Other than that, my biggest issues with the current mailing system are:
> >
> > * There’s no way to keep a updated proposal of your own- if you decide
> to change your proposal, you have to communicate the change. Then, if you
> want to find the authoritative current copy, since you might’ve forgotten
> or you want to join he current discussion, then you have to dig through
> the emails and recursively apply the proposed change. It’s just easier if
> people can have one proposal they can edit themselves.
> >   * I’ve seen experienced people get confused about what was the current
> proposal because they were replying to older emails or they didn’t see the
> email with the clear examples.
>
> I agree that editing is a very useful feature. In a large discussion,
> newcomers can comment after reading only the first few posts, and if
> the first post has an easily-misunderstood line, you'll get people
> talking about it.
>
> For proposals, I'm concerned that many forums don't have version
> history in their editing tools (Reddit being one such discussion
> site). Version history can be useful in understanding old comments.
> Instead, you'd have to put it up on a repo and link to it. Editing
> will help when you realize you should move your proposal to a public
> repo.
>
> > * The mailing list is frankly obscure. Python community leaders and
> package maintainers often are not aware or do not participate in
> Python-ideas. Not many people know how to use or navigate a mailing list.
> >   * No one really promotes the mailing list, you have to go out of your
> way to find where new features are proposed.
> >   * Higher discoverability means more people can participate, providing
> their own use cases or voting (I mean using like or dislike measures,
> consensus should still be how things are approved) go out of their way to
> find so they can propose something. Instead, I envision a forum where
> people can read and give their 2 cents about what features they might like
> to see or might not want to see.
>
> Some of these problems are not about mailing lists.
>
> Whether a forum is more accessible can go either way. A mailing list
> is more accessible because everyone has access to email, and it
> doesn't require making another account. It is less accessible because
> people might get intimidated by such old interfaces or culture (like
> proper quoting etiquette, or when to switch to private replies).
> Setting up an email interface to a forum can be a compromise.
>
> >* More people means instead of having to make decisions from
> sometimes subjective personal experience, we can make decisions with
> confidence in what other Python devs want.
>
> I don't agree. You don't get more objective by getting a larger
> self-selected sample, not without carefully designing who will
> self-select.
>
> But getting more people means getting MORE subjective per

Re: [Python-ideas] Moving to another forum system where

2018-09-19 Thread James Lu


>
> Most of the real decisions are actually taken
> outside of it, with more direct channels in the small groups of
> contributors.
>
It would be very nice if there was more transparency in
this process. The language is better if more subjective
personal experience heard- but to make that happen, 
the forum experience must be better for both

On Tuesday, September 18, 2018 at 8:21:46 PM UTC-4, James Lu wrote:
>
> > Is that really an issue here? I personally haven't seen threads where
> > Brett tried to stop an active discussion, but people ignored him and
> > kept fighting.
> Not personally with Brett, but I have seen multiple people try to stop the 
> “reword or remove beautiful is better than ugly in Zen of Python.” The 
> discussion was going in circles and evolved into attacking each other’s use 
> of logical fallacies. 
>
> Other than that, my biggest issues with the current mailing system are:
>
> * There’s no way to keep a updated proposal of your own- if you decide to 
> change your proposal, you have to communicate the change. Then, if you want 
> to find the authoritative current copy, since you might’ve forgotten or you 
> want to join he current discussion, then you have to dig through  the 
> emails and recursively apply the proposed change. It’s just easier if 
> people can have one proposal they can edit themselves.
>   * I’ve seen experienced people get confused about what was the current 
> proposal because they were replying to older emails or they didn’t see the 
> email with the clear examples.
> * The mailing list is frankly obscure. Python community leaders and 
> package maintainers often are not aware or do not participate in 
> Python-ideas. Not many people know how to use or navigate a mailing list.
>   * No one really promotes the mailing list, you have to go out of your 
> way to find where new features are proposed. 
>   * Higher discoverability means more people can participate, providing 
> their own use cases or voting (I mean using like or dislike measures, 
> consensus should still be how things are approved) go out of their way to 
> find so they can propose something. Instead, I envision a forum where 
> people can read and give their 2 cents about what features they might like 
> to see or might not want to see. 
>* More people means instead of having to make decisions from sometimes 
> subjective personal experience, we can make decisions with confidence in 
> what other Python devs want. 
>
> Since potential proposers will find it easier to navigate a GUI forum, 
> they can read previous discussions to understand the reasoning, precedent 
> behind rejected and successful features. People proposing things that have 
> already been rejected before can be directed to open a subtopic on the 
> older discussion. 
>
> > On Sep 18, 2018, at 3:19 PM, python-ideas-requ...@python.org wrote:
> > 
> > Is that really an issue here? I personally haven't seen threads where
> > Brett tried to stop an active discussion, but people ignored him and
> > kept fighting.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python-ideas Digest, Vol 142, Issue 110

> Frankly, I think the bigger issue is all too human -- we get sucked in and
> participate when we really know we shouldn't (or maybe that's just me).
> 
That may be why some people misbehave, but we have no way of discouraging that 
misbehavior.

> And I'm having a hard time figuring out how moderation would actually
> result in the "good" discussion we really want in an example like the
> "beautiful is better than ugly" issue, without someone trusted individual
> approving every single post -- I don't imagine anyone wants to do that.

In a forum, the beautiful is better than ugly issue would be locked. No more 
posts can be added. If someone wants to discuss another proposal branching off 
of the original discussion, they can start a new thread. If they just want to 
lampoon, we can courteously ask them to 1) take it elsewhere or 2) move the 
post to a “malarkey” section of the forum where people don’t get notified.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where moderation is possible

> Frankly, I think the bigger issue is all too human -- we get sucked in and
> participate when we really know we shouldn't (or maybe that's just me).
> 
That may be why some people misbehave, but we have no way of discouraging that 
misbehavior.

> And I'm having a hard time figuring out how moderation would actually
> result in the "good" discussion we really want in an example like the
> "beautiful is better than ugly" issue, without someone trusted individual
> approving every single post -- I don't imagine anyone wants to do that.

In a forum, the beautiful is better than ugly issue would be locked. No more 
posts can be added. If someone wants to discuss another proposal branching off 
of the original discussion, they can start a new thread. If they just want to 
lampoon, we can courteously ask them to 1) take it elsewhere or 2) move the 
post to a “malarkey” section of the forum where people don’t get notified.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where moderation is possible

Were there any productive parts to that conversation?

Sent from my iPhone

> On Sep 20, 2018, at 9:47 AM, Chris Barker  wrote:
> 
>> On Thu, Sep 20, 2018 at 1:39 PM, James Lu  wrote:
>> In a forum, the beautiful is better than ugly issue would be locked. No more 
>> posts can be added.
> 
> Exactly -- but that means we are stopping the discussion -- but we don't want 
> to stop the discussion altogether, we want to have the productive parts of 
> the discussion, without the non-productive parts -- not sure there is any 
> technical solution to that problem.
> 
> - CHB
> 
> -- 
> 
> Christopher Barker, Ph.D.
> Oceanographer
> 
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
> 
> chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where moderation is possible

> It's absence is a big advantage. We're not a social network with
> "likes". We don't need a bunch of argumentless "voting".

Up/ down voting indicates how much consensus we have among the entire 
community- an expert might agree with another expert’s arguments but not have 
anything else to add, and an outsider might agree with the scenario an expert 
presents without having much more to add. Granular up/down votes are useful.

> Believe it or not, I like the fact that you can't just edit posts.  I've 
> lost count of the number of forum threads I've been on where comments to 
> the initial post make *no sense* because that initial post is nothing 
> like it was to start with.

There is version history. Not all of us have the time to read through every 
single post beforehand to get the current state of discussion. 

Hmm, what if we used GitHub as a discussion forum? You’d make a pull request 
with an informal proposal to a repository. Then people can comment on lines in 
the diff and reply to each other there. The OP can update their branch to 
change their proposal- expired/stale comments on old diffs are automatically 
hidden.

You can also create a competing proposal by forming from the OP’s branch and 
sending a new PR.

> Just editing your post and expecting people to notice 
> is not going to cut it. 

You would ping someone after editing the post.

> Approximately none of this has anything to do with the medium.  If the 
> mailing list is obscure (and personally I don't think it is), it just 
> needs better advertising.  A poorly advertised forum is equally 
> undiscoverable.

It does have to do with the medium. First, people aren’t used to mailing lists- 
but that’s not what’s important here. If the PSF advertised for people to sign 
up over say twitter, then we’d get even more email. More +1 and more -1. Most 
of us don’t want more mailing list volume. 

The fact that you can’t easily find an overview people will post arguments that 
have already been made if they don’t have the extreme patience to read all that 
has been said before.

For the rest of your comments, I advise you to read the earlier discussion that 
other people had in response to my email.

> That message was rather bad in my not so humble opinion -- it was
> just "I want my +1 button" without any argument. Your message is much
> better as it have arguments. See, the absence of the button work!
> 
>   We're proposing and *discussing* things here not "likes" each other.
> Write your arguments or be silent.

Please respond to the actual arguments in both of the two emails that have 
arguments in support of +1/-1.

+1/-1 reflects which usage scenarios people find valuable, since Python 
features sometimes do benefit one group at the detriment to another. Or use 
syntax/behavior for one thing that could be used for another thing, and some 
programming styles of python use cases would prefer one kind of that 
syntax/behavior.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Moving to another forum system where

2018-09-21 Thread James Lu

One of the reasons Guido left was the insane volume of emails he had to read on 
Python-ideas. 

> A tiny bit of discussion is still better than none at all.
> And even if there's no discussion, there's a name attached
> to the message, which makes it more personal and meaningful
> than a "+1" counter getting incremented somewhere.
> 
> Counting only makes sense if the counts are going to be
> treated as votes, and we don't do that.

I agree. I think this is good evidence in favor of using GitHub pull requests 
or GitHub issues- you can see exactly who +1’d a topic.

GitHub also has moderation tools and the ability to delete comments that are 
irrelevant, and edit comments that are disrespectful. 

> I hope that, if any such change is made, a forum system is
> chosen that allows full participation via either email or news.
> Otherwise it will probably mean the end of my participation,
> because I don't have time to chase down and wrestle with
> multiple web forums every day.

+1, everyone should be accommodated. I believe GitHub has direct email 
capability. If you watch the repository and have email notifications on, you 
can reply directly to an email and it will be sent as a reply.

—
To solve the problem of tons of email for controversial decisions like :=, I 
don’t think GitHub issues would actually be the solution. The best solution 
would to have admins receive all the email, and broadcast a subset of the email 
sent, only broadcasting new arguments and new opinions. 

Admins can do this “summary duty” every 12 hours on a rotating basis, where 
each admin takes turns doing summary duty. 

This solution would mean a slower iteration time for the conversation, but it 
would significantly lessen the deluge of email, and I think that would make it 
more bearable for people participating in the conversation. After all, once a 
proposal has been fleshed out, what kind of conversation needs more than say 30 
rounds of constructive discussion- in that case, if people reply every 25 
hours, the discussion would be done in a month.

For transparency purposes, all of the email can be made received for approval 
can be published online. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] JS’ governance model is worth inspecting

2018-09-21 Thread James Lu

JS’ decisions are made by a body known as TC39, a fairly/very small group of JS 
implementers.

First, JS has an easy and widely supported way to modify the language for 
yourself: Babel. Babel transpires your JS to older JS, which is then run.

You can publish your language modification on the JS package manager, npm. 

When a feature is being considered for inclusion in mainline JS, the proposal 
must first gain a champion (represented by 🚀)that is a member of TC-39. The 
guidelines say that the proposal’s features should already have found use in 
the community. Then it moves through three stages, and the champion must think 
the proposal is ready for the next stage before it can move on. I’m hazy on 
what the criterion for each of the three stages is. The fourth stage is 
approved.

I believe the global TC39 committee meets regularly in person, and at those 
meetings, proposals can advance stages- these meetings are frequent enough for 
the process to be fast and slow enough that people can have the time to try out 
a feature before it becomes main line JS. Meeting notes are made public.

The language and its future features are discussed on ESDiscuss.org, which is 
surprisingly filled with quality and respectful discussion, largely from 
experts in the JavaScript language. 

I’m fairly hazy on the details, this is just the summary off the top of my head.

—
I’m not saying this should be Python’s governance model, just to keep JS’ in 
mind. 


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JS’ governance model is worth inspecting

2018-09-21 Thread James Lu

> Babel's primary purpose is transpiling to run on older browsers, which isn't 
> that much of an issue with Python. It's also complicated a bit by the large 
> number of implementations that *must* be developed in sync, again due to 
> running in user's browsers. 
It’s true that one of Babel’s purposes is to transpile to older browsers. 
However, if you look at the React Native project template (a fairly typical 
JavaScript project template), the Babel transpiler preset looks like this:
- Remove Flow and transpile JSX: these are language extensions for type hinting 
and inline XML, not intended to be merged with mainline JavaScript. 
- Transpile Standardized JavaScript to older JavaScript 
- Stage-3 Proposal: adds dedicated syntax for setting static and instance 
variables outside of the constructor but within the class.
- Stage-1 Proposal: syntax to support trailing comma in functions
function foo(
a,
b,
c,
) { }
Inspired from Python’s syntax:
def foo(a,
  b, 
  c,
 ):
...

As you can see, two non-standard features under consideration for inclusion in 
the standard are included in the preset. This inclusion of non-standard 
features is typical for JS starter projects. One of the requirements for 
advancing stages is seeing practical use in the industry.

Since almost everyone uses Babel anyways, this four stage process acts as a way 
to gain consensus on the base set of JS features. 

Almost all of the newest standard JS took this route of unofficial use before 
official inclusion. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JS’ governance model is worth inspecting

2018-09-22 Thread James Lu

> To my mind, there is one very big reason we should be cautious about 
> adopting JS language-design policies, namely, that they have led to a 
> very, very poorly designed language.  No doubt a good deal of that is 
> baggage from early stages in which JS had a poor to nonexistent language 
> design governance model.  Nonetheless, the failure of JS to fix its 
> numerous fundamental flaws, and especially the rapid feature churn in 
> recent years, suggests to me that their model should be viewed with 
> skepticism.
I disagree. The language is often very flexible and effective in its domains. I 
don’t know what you mean by “rapid feature churn”, churn usually means existing 
features are superseded by newer ones- this isn’t the case.

JS is much more nuanced than it appears on the surface. It’s understandable 
that those with only a glossing of JS look down on it, because JS really was a 
primitive language a few years ago.

You can learn about JS in depth with the poorly-named “You don’t know JS” free 
online book.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

2018-09-24 Thread James Lu

You could disassemble (import dis) the lambda to biew the names of the lambdas.

@before(lambda self, key, _, length, get: self.length(), self.get(key))

Perhaps you could disassemble the function code and look at all operations or 
accesses that are done to “old.” and evaluate those expressions before the 
function runs. Then you could “replace” the expression.
@post(lambda self, key, old: old.get is None and old.length + 1 ==
self.length())

Either the system would grab old.get and old.length or be greedy and grab 
old.get is None and old.length + 1. It would then replace the old.get and 
old.length with injects that only respond to is None and +1.

Or, a syntax like this 
@post(lambda self, key, old: [old.get(old.key)] is None and [old.self.length() 
+ 1] ==
self.length())

Where the stuff inside the brackets is evaluated before the decorated function 
runs. It would be useful for networking functions or functions that do 
something ephemeral, where data related to the value being accessed needed for 
the expression no longer exists after the function. 

This does conflict with list syntax forever, so maybe either force people to do 
list((expr,)) or use an alternate syntax like one item set syntax { } or double 
set syntax {{ }} or double list syntax [[ ]]. Ditto with having to avoid the 
literals for the normal meaning.

You could modify Python to accept any expression for the lambda function and 
propose that as a PEP. (Right now it’s hardcoded as a dotted name and 
optionally a single argument list surrounded by parentheses.)

I suggest that instead of “@before” it’s “@snapshot” and instead of “old” it’s 
“snapshot”.

Python does have unary plus/minus syntax as well as stream operators (<<, >>) 
and list slicing syntax and the @ operator and operators & and | if you want to 
play with syntax. There’s also the line continuation character for crazy 
lambdas.

Personally I prefer
@post(lambda self, key, old: {{old.self.get(old.key)}} and {{old.self.length() 
+ 1}} ==
self.length())

because it’s explicit about what it does (evaluate the expressions within {{ }} 
before the function runs. I also find it elegant.

Alternatively, inside the {{ }} could be a special scope where locals() is all 
the arguments @pre could’ve received as a dictionary. For either option you can 
remove the old parameter from the lambda. Example:
@post(lambda self, key: {{self.get(key)}} and {{self.length() + 1}} ==
self.length())

Perhaps the convention should be to write {{ expr }} (with the spaces in 
between).

You’d probably have to use the ast module to inspect it instead of the dis 
modul. Then find some way to reconstruct the expressions inside the double 
brackets- perhaps by reconstructing the AST and compiling it to a code object, 
or perhaps by finding the part of the string the expression is located. dis can 
give you the code as a string and you can run a carefully crafted regex on it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why is design-by-contracts not widely adopted?

2018-09-24 Thread James Lu

Perhaps it’s because fewer Python functions involve transitioning between 
states. Web development and statistics don’t involve many state transition. 
State transitions are where I think I would find it useful to write contracts 
out explicitly.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] JS? governance model is worth inspecting

2018-09-24 Thread James Lu

> Which features of the TC39 committee's ECMAscript (ES) language governance
> model would be helpful to incorporate into the Python language governance
> model?

Having “beta” or “alpha” editions of features, special versions of the 
interpreter people can test out to see if they prefer the version with the new 
feature. 

To prevent splintering, the main releases would only support the main feature 
set. In a worst case scenario, people can compile incompatible code to .pyc 
before running it.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

2018-09-25 Thread James Lu

Have you looked at the built-in AST module, ast? 
https://docs.python.org/3/library/ast.html

I don’t see anything preventing you from walking the AST Python itself can give 
you- you’d look for two Set AST nodes if we were to do {{ }}.

There’s also the parser built-in module. You can use it if you first use 
dis.code_info to get the source then re-parse it. It helps with parse trees. 
Parse trees are generated before the AST I think. You’d use the parser module’s 
ST objects with the token module’s constants, for example token.LBRACE or 
token.RBRACE.

Have you looked at the built-in dis module? You can use dis.code_info(obj) to 
get the string of the function. Then you could look for your specified syntax 
with regex and recompile that with the ast module.

Sent from my iPhone

> On Sep 25, 2018, at 1:49 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> Thanks for the feedback!
> 
> I also thought about decompiling the condition to find its AST and figure out 
> what old values are needed. However, this is not so easily done at the moment 
> as all the decompilation libraries I looked at (meta, ucompyle6) are simply 
> struggling to keep up with the development of the python bytecode. In other 
> words, the decompiler needs to be updated with every new version of python 
> which is kind of a loosing race (unless the python devs themselves don't 
> provide this functionality which is not the case as far as I know).
> 
> There is macropy (https://github.com/lihaoyi/macropy) which was suggested on 
> the other thread 
> (https://groups.google.com/forum/#!topic/python-ideas/dmXz_7LH4GI) that I'm 
> currently looking at.
> 
> Cheers,
> Marko
> 
> 
>> On Tue, 25 Sep 2018 at 00:35, James Lu  wrote:
>> You could disassemble (import dis) the lambda to biew the names of the 
>> lambdas.
>> 
>> @before(lambda self, key, _, length, get: self.length(), self.get(key))
>> 
>> Perhaps you could disassemble the function code and look at all operations 
>> or accesses that are done to “old.” and evaluate those expressions before 
>> the function runs. Then you could “replace” the expression.
>> @post(lambda self, key, old: old.get is None and old.length + 1 ==
>> self.length())
>> 
>> Either the system would grab old.get and old.length or be greedy and grab 
>> old.get is None and old.length + 1. It would then replace the old.get and 
>> old.length with injects that only respond to is None and +1.
>> 
>> Or, a syntax like this 
>> @post(lambda self, key, old: [old.get(old.key)] is None and 
>> [old.self.length() + 1] ==
>> self.length())
>> 
>> Where the stuff inside the brackets is evaluated before the decorated 
>> function runs. It would be useful for networking functions or functions that 
>> do something ephemeral, where data related to the value being accessed 
>> needed for the expression no longer exists after the function. 
>> 
>> This does conflict with list syntax forever, so maybe either force people to 
>> do list((expr,)) or use an alternate syntax like one item set syntax { } or 
>> double set syntax {{ }} or double list syntax [[ ]]. Ditto with having to 
>> avoid the literals for the normal meaning.
>> 
>> You could modify Python to accept any expression for the lambda function and 
>> propose that as a PEP. (Right now it’s hardcoded as a dotted name and 
>> optionally a single argument list surrounded by parentheses.)
>> 
>> I suggest that instead of “@before” it’s “@snapshot” and instead of “old” 
>> it’s “snapshot”.
>> 
>> Python does have unary plus/minus syntax as well as stream operators (<<, 
>> >>) and list slicing syntax and the @ operator and operators & and | if you 
>> want to play with syntax. There’s also the line continuation character for 
>> crazy lambdas.
>> 
>> Personally I prefer
>> @post(lambda self, key, old: {{old.self.get(old.key)}} and 
>> {{old.self.length() + 1}} ==
>> self.length())
>> 
>> because it’s explicit about what it does (evaluate the expressions within {{ 
>> }} before the function runs. I also find it elegant.
>> 
>> Alternatively, inside the {{ }} could be a special scope where locals() is 
>> all the arguments @pre could’ve received as a dictionary. For either option 
>> you can remove the old parameter from the lambda. Example:
>> @post(lambda self, key: {{self.get(key)}} and {{self.length() + 1}} ==
>> self.length())
>> 
>> Perhaps the convention should be to write {{ expr }} (with the spaces in 
>> between).
>> 
>> You’d probably have to use the ast module to inspect it instead of the dis 
>> mod

Re: [Python-ideas] "old" values in postconditions

2018-09-25 Thread James Lu

Hmm, I was wrong: there is no reliable way to get the code of a lambda 
function. 

If it was possible to execute all code paths of the function, we could monkey 
patch the builtins so { } used our own custom set class. 

Alternatively, the decorator could also accept a string.

Or maybe we could send a PEP to add the .func_code attribute to lambdas as well 
as normal functions.

There’s also a technique online where they find the lambda’s source by locating 
the file the function was defined in and then removing the irrelevant parts, 
but that just doesn’t sound practical to me.

There’s also MacroPy.

I think the best solution would be to mock the old object and record the 
operations done to the object, like the other replier gave a PoC of. Proposed 
syntax
from icontract import post, old
@post(lambda: ...,
key=old.self.key(),
)

Sent from my iPhone

> On Sep 25, 2018, at 1:49 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> Thanks for the feedback!
> 
> I also thought about decompiling the condition to find its AST and figure out 
> what old values are needed. However, this is not so easily done at the moment 
> as all the decompilation libraries I looked at (meta, ucompyle6) are simply 
> struggling to keep up with the development of the python bytecode. In other 
> words, the decompiler needs to be updated with every new version of python 
> which is kind of a loosing race (unless the python devs themselves don't 
> provide this functionality which is not the case as far as I know).
> 
> There is macropy (https://github.com/lihaoyi/macropy) which was suggested on 
> the other thread 
> (https://groups.google.com/forum/#!topic/python-ideas/dmXz_7LH4GI) that I'm 
> currently looking at.
> 
> Cheers,
> Marko
> 
> 
>> On Tue, 25 Sep 2018 at 00:35, James Lu  wrote:
>> You could disassemble (import dis) the lambda to biew the names of the 
>> lambdas.
>> 
>> @before(lambda self, key, _, length, get: self.length(), self.get(key))
>> 
>> Perhaps you could disassemble the function code and look at all operations 
>> or accesses that are done to “old.” and evaluate those expressions before 
>> the function runs. Then you could “replace” the expression.
>> @post(lambda self, key, old: old.get is None and old.length + 1 ==
>> self.length())
>> 
>> Either the system would grab old.get and old.length or be greedy and grab 
>> old.get is None and old.length + 1. It would then replace the old.get and 
>> old.length with injects that only respond to is None and +1.
>> 
>> Or, a syntax like this 
>> @post(lambda self, key, old: [old.get(old.key)] is None and 
>> [old.self.length() + 1] ==
>> self.length())
>> 
>> Where the stuff inside the brackets is evaluated before the decorated 
>> function runs. It would be useful for networking functions or functions that 
>> do something ephemeral, where data related to the value being accessed 
>> needed for the expression no longer exists after the function. 
>> 
>> This does conflict with list syntax forever, so maybe either force people to 
>> do list((expr,)) or use an alternate syntax like one item set syntax { } or 
>> double set syntax {{ }} or double list syntax [[ ]]. Ditto with having to 
>> avoid the literals for the normal meaning.
>> 
>> You could modify Python to accept any expression for the lambda function and 
>> propose that as a PEP. (Right now it’s hardcoded as a dotted name and 
>> optionally a single argument list surrounded by parentheses.)
>> 
>> I suggest that instead of “@before” it’s “@snapshot” and instead of “old” 
>> it’s “snapshot”.
>> 
>> Python does have unary plus/minus syntax as well as stream operators (<<, 
>> >>) and list slicing syntax and the @ operator and operators & and | if you 
>> want to play with syntax. There’s also the line continuation character for 
>> crazy lambdas.
>> 
>> Personally I prefer
>> @post(lambda self, key, old: {{old.self.get(old.key)}} and 
>> {{old.self.length() + 1}} ==
>> self.length())
>> 
>> because it’s explicit about what it does (evaluate the expressions within {{ 
>> }} before the function runs. I also find it elegant.
>> 
>> Alternatively, inside the {{ }} could be a special scope where locals() is 
>> all the arguments @pre could’ve received as a dictionary. For either option 
>> you can remove the old parameter from the lambda. Example:
>> @post(lambda self, key: {{self.get(key)}} and {{self.length() + 1}} ==
>> self.length())
>> 
>> Perhaps the convention should be to write {{ expr }} (wit

Re: [Python-ideas] "old" values in postconditions

2018-09-25 Thread James Lu

> I'm surprised you haven't found
>inspect.getsource(func)

I did. That’s exactly what I was describing in the paragraph. It wouldn’t work 
in interactive mode and it includes everything on the same line of the lambda 
definition.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

I still prefer snapshot, though capture is a good name too. We could use 
generator syntax and inspect the argument names.

Instead of “a”, perhaps use “_”. Or maybe use “A.”, for arguments. Some people 
might prefer “P” for parameters, since parameters sometimes means the value 
received while the argument means the value passed.

(#A1)

from icontract import snapshot, __
@snapshot(some_func(_.some_argument.some_attr) for some_identifier, _ in __)

Or (#A2)

@snapshot(some_func(some_argument.some_attr) for some_identifier, _, 
some_argument in __)

—
Or (#A3)

@snapshot(lambda some_argument,_,some_identifier: 
some_func(some_argument.some_attr))

Or (#A4)

@snapshot(lambda _,some_identifier: some_func(_.some_argument.some_attr))
@snapshot(lambda _,some_identifier, other_identifier: 
some_func(_.some_argument.some_attr), other_func(_.self))

I like #A4 the most because it’s fairly DRY and avoids the extra punctuation of 

@capture(lambda a: {"some_identifier": some_func(a.some_argument.some_attr)})

> On Sep 26, 2018, at 12:23 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi,
> 
> Franklin wrote:
>> The name "before" is a confusing name. It's not just something that
>> happens before. It's really a pre-`let`, adding names to the scope of
>> things after it, but with values taken before the function call. Based
>> on that description, other possible names are `prelet`, `letbefore`,
>> `predef`, `defpre`, `beforescope`. Better a name that is clearly
>> confusing than one that is obvious but misleading.
> 
> James wrote:
>> I suggest that instead of “@before” it’s “@snapshot” and instead of “old” 
>> it’s “snapshot”.
> 
> 
> I like "snapshot", it's a bit clearer than prefixing/postfixing verbs with 
> "pre" which might be misread (e.g., "prelet" has a meaning in Slavic 
> languages and could be subconsciously misread, "predef" implies to me a 
> pre-definition rather than prior-to-definition , "beforescope" is very clear 
> for me, but it might be confusing for others as to what it actually refers to 
> ). What about "@capture" (7 letters for captures versus 8 for snapshot)? I 
> suppose "@let" would be playing with fire if Python with conflicting new 
> keywords since I assume "let" to be one of the candidates.
> 
> Actually, I think there is probably no way around a decorator that 
> captures/snapshots the data before the function call with a lambda (or even a 
> separate function). "Old" construct, if we are to parse it somehow from the 
> condition function, would limit us only to shallow copies (and be complex to 
> implement as soon as we are capturing out-of-argument values such as globals 
> etc.). Moreove, what if we don't need shallow copies? I could imagine a dozen 
> of cases where shallow copy is not what the programmer wants: for example, 
> s/he might need to make deep copies, hash or otherwise transform the input 
> data to hold only part of it instead of copying (e.g., so as to allow 
> equality check without a double copy of the data, or capture only the value 
> of certain property transformed in some way).
> 
> I'd still go with the dictionary to allow for this extra freedom. We could 
> have a convention: "a" denotes to the current arguments, and "b" denotes the 
> captured values. It might make an interesting hint that we put "b" before "a" 
> in the condition. You could also interpret "b" as "before" and "a" as 
> "after", but also "a" as "arguments".
> 
> @capture(lambda a: {"some_identifier": some_func(a.some_argument.some_attr)})
> @post(lambda b, a, result: b.some_identifier > result + 
> a.another_argument.another_attr)
> def some_func(some_argument: SomeClass, another_argument: AnotherClass) -> 
> SomeResult:
> ...
> "b" can be omitted if it is not used. Under the hub, all the arguments to the 
> condition would be passed by keywords.
> 
> In case of inheritance, captures would be inherited as well. Hence the 
> library would check at run-time that the returned dictionary with captured 
> values has no identifier that has been already captured, and the linter 
> checks that statically, before running the code. Reading values captured in 
> the parent at the code of the child class might be a bit hard -- but that is 
> case with any inherited methods/properties. In documentation, I'd list all 
> the captures of both ancestor and the current class.
> 
> I'm looking forward to reading your opinion on this and alternative 
> suggestions :)
> Marko
> 
>> On Tue, 25 Sep 2018 at 18:12, Franklin? Lee  
>> wrote:
>> On Sun, Sep 23, 2018 at 2:05 AM Marko Ristin-Kaufmann
>>  wrote:
>> >
>> > Hi,
>> >
>> > (I'd like to fork from a previous thread, "Pre-conditions and 
>> > post-conditions", since it got long and we started discussing a couple of 
>> > different things. Let's discuss in this thread the implementation of a 
>> > library for design-by-contract and how to push it forward to hopefully add 
>> > it to the standard library one day.)
>> >
>> > For those unfamiliar with contracts and

Re: [Python-ideas] "while:" for the loop

repeat could be only considered a keyword when it’s used as a loop

Sent from my iPhone

> On Sep 26, 2018, at 8:46 AM, Brice Parent  wrote:
> 
>> Le 26/09/2018 à 14:33, James Lu a écrit :
>> what about “repeat:”?
>> 
>> Sent from my iPhone
> I'm not sure it was on purpose, but you replied to me only, and not the 
> entire list.
> I believe the adding of a new keyword to do something that is already 
> straightforward (`while True:`), and that doesn't add any new functionality, 
> won't probably ever be accepted.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why is design-by-contracts not widely adopted?

> It's easy to say that they're boolean expressions. But that's like
> saying that unit tests are just a bunch of boolean expressions too.
> Why do we have lots of different forms of test, rather than just a big
> fat "assert this and this and this and this and this and this"?
> Because the key to unit testing is not "boolean expressions", it's a
> language that can usefully describe what it is we're testing.
> Contracts aren't just boolean expressions - they're a language (or a
> mini-language) that lets you define WHAT the contract entails.
Please read the earlier discussion from Marko. Contracts are like unit tests 
but acts as documentation that is right next to the function definition. It’s 
also much shorter in number of lines to define. You can write a simple unit 
smoke test to turn a contract into a unit test. Contracts serve unit testing 
and documentation at the same time.

Sent from my iPhone

> On Sep 26, 2018, at 3:18 AM, Chris Angelico  wrote:
> 
> It's easy to say that they're boolean expressions. But that's like
> saying that unit tests are just a bunch of boolean expressions too.
> Why do we have lots of different forms of test, rather than just a big
> fat "assert this and this and this and this and this and this"?
> Because the key to unit testing is not "boolean expressions", it's a
> language that can usefully describe what it is we're testing.
> Contracts aren't just boolean expressions - they're a language (or a
> mini-language) that lets you define WHAT the contract entails.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

> Actually, following on #A4, you could also write those as multiple decorators:
> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
> @snpashot(lambda _, other_identifier: other_func(_.self))

Yes, though if we’re talking syntax using kwargs would probably be better.
Using “P” instead of “_”: (I agree that _ smells of ignored arguments)

@snapshot(some_identifier=lambda P: ..., some_identifier2=lambda P: ...)

Kwargs has the advantage that you can extend multiple lines without repeating 
@snapshot, though many lines of @capture would probably be more intuitive since 
each decorator captures one variable.

> Why uppercase "P" and not lowercase (uppercase implies a constant for me)?

To me, the capital letters are more prominent and explicit- easier to see when 
reading code. It also implies its a constant for you- you shouldn’t be 
modifying it, because then you’d be interfering with the function itself.

Side node: maybe it would be good to have an @icontract.nomutate (probably use 
a different name, maybe @icontract.readonly) that makes sure a method doesn’t 
mutate its own __dict__ (and maybe the __dict__ of the members of its 
__dict__). It wouldn’t be necessary to put the decorator on every read only 
function, just the ones your worried might mutate.

Maybe a @icontract.nomutate(param=“paramname”) that ensures the __dict__ of all 
members of the param name have the same equality or identity before and after. 
The semantics would need to be worked out.

> On Sep 26, 2018, at 8:58 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> 
> Actually, following on #A4, you could also write those as multiple decorators:
> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
> @snpashot(lambda _, other_identifier: other_func(_.self))
> 
> Am I correct?
> 
> "_" looks a bit hard to read for me (implying ignored arguments).
> 
> Why uppercase "P" and not lowercase (uppercase implies a constant for me)? 
> Then "O" for "old" and "P" for parameters in a condition:
> @post(lambda O, P: ...)
> ?
> 
> It also has the nice property that it follows both the temporal and the 
> alphabet order :)
> 
>> On Wed, 26 Sep 2018 at 14:30, James Lu  wrote:
>> I still prefer snapshot, though capture is a good name too. We could use 
>> generator syntax and inspect the argument names.
>> 
>> Instead of “a”, perhaps use “_”. Or maybe use “A.”, for arguments. Some 
>> people might prefer “P” for parameters, since parameters sometimes means the 
>> value received while the argument means the value passed.
>> 
>> (#A1)
>> 
>> from icontract import snapshot, __
>> @snapshot(some_func(_.some_argument.some_attr) for some_identifier, _ in __)
>> 
>> Or (#A2)
>> 
>> @snapshot(some_func(some_argument.some_attr) for some_identifier, _, 
>> some_argument in __)
>> 
>> —
>> Or (#A3)
>> 
>> @snapshot(lambda some_argument,_,some_identifier: 
>> some_func(some_argument.some_attr))
>> 
>> Or (#A4)
>> 
>> @snapshot(lambda _,some_identifier: some_func(_.some_argument.some_attr))
>> @snapshot(lambda _,some_identifier, other_identifier: 
>> some_func(_.some_argument.some_attr), other_func(_.self))
>> 
>> I like #A4 the most because it’s fairly DRY and avoids the extra punctuation 
>> of 
>> 
>> @capture(lambda a: {"some_identifier": some_func(a.some_argument.some_attr)})
>> 
>> On Sep 26, 2018, at 12:23 AM, Marko Ristin-Kaufmann  
>> wrote:
>> 
>>> Hi,
>>> 
>>> Franklin wrote:
>>>> The name "before" is a confusing name. It's not just something that
>>>> happens before. It's really a pre-`let`, adding names to the scope of
>>>> things after it, but with values taken before the function call. Based
>>>> on that description, other possible names are `prelet`, `letbefore`,
>>>> `predef`, `defpre`, `beforescope`. Better a name that is clearly
>>>> confusing than one that is obvious but misleading.
>>> 
>>> James wrote:
>>>> I suggest that instead of “@before” it’s “@snapshot” and instead of “old” 
>>>> it’s “snapshot”.
>>> 
>>> 
>>> I like "snapshot", it's a bit clearer than prefixing/postfixing verbs with 
>>> "pre" which might be misread (e.g., "prelet" has a meaning in Slavic 
>>> languages and could be subconsciously misread, "predef" implies to me a 
>>> pre-definition rather than prior-to-definition , "beforescope" is very 
>>> clear for me, but it m

Re: [Python-ideas] "old" values in postconditions

Why couldn’t we record the operations  done to a special object and replay them?

>>> Actually, I think there is probably no way around a decorator that 
>>> captures/snapshots the data before the function call with a lambda (or even 
>>> a separate function). "Old" construct, if we are to parse it somehow from 
>>> the condition function, would limit us only to shallow copies (and be 
>>> complex to implement as soon as we are capturing out-of-argument values 
>>> such as globals etc.). Moreove, what if we don't need shallow copies? I 
>>> could imagine a dozen of cases where shallow copy is not what the 
>>> programmer wants: for example, s/he might need to make deep copies, hash or 
>>> otherwise transform the input data to hold only part of it instead of 
>>> copying (e.g., so as to allow equality check without a double copy of the 
>>> data, or capture only the value of certain property transformed in some 
>>> way).

from icontract import snapshot, P, thunk
@snapshot(some_identifier=P.self.some_method(P.some_argument.some_attr))

P is an object of our own type, let’s call the type MockP. MockP returns new 
MockP objects when any operation is done to it. MockP * MockP = MockP. 
MockP.attr = MockP. MockP objects remember all the operations done to them, and 
allow the owner of a MockP object to re-apply the same operations 

“thunk” converts a function or object or class to a MockP object, storing the 
function or object for when the operation is done. 

thunk(function)()

Of course, you could also thunk objects like so: thunk(3) * P.number. (Though 
it might be better to keep the 3 after P.number in this case so P.number’s 
__mult__ would be invoked before 3’s __mult__ is invokes.

In most cases, you’d save any operations that can be done on a copy of the data 
as generated by @snapshot in @postcondiion. thunk is for rare scenarios where 
1) it’s hard to capture the state, for example an object that manages network 
state (or database connectivity etc) and whose stage can only be read by an 
external classmethod  2) you want to avoid using copy.deepcopy.

I’m sure there’s some way to override isinstance through a meta class or dunder 
subclasshook.

I suppose this mocking method could be a shorthand for when you don’t need the 
full power of a lambda. It’s arguably more succinct and readable, though YMMV.

I look forward to reading your opinion on this and any ideas you might have.

> On Sep 26, 2018, at 3:56 PM, James Lu  wrote:
> 
> Hi Marko,
> 
>> Actually, following on #A4, you could also write those as multiple 
>> decorators:
>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
>> @snpashot(lambda _, other_identifier: other_func(_.self))
> 
> Yes, though if we’re talking syntax using kwargs would probably be better.
> Using “P” instead of “_”: (I agree that _ smells of ignored arguments)
> 
> @snapshot(some_identifier=lambda P: ..., some_identifier2=lambda P: ...)
> 
> Kwargs has the advantage that you can extend multiple lines without repeating 
> @snapshot, though many lines of @capture would probably be more intuitive 
> since each decorator captures one variable.
> 
>> Why uppercase "P" and not lowercase (uppercase implies a constant for me)?
> 
> To me, the capital letters are more prominent and explicit- easier to see 
> when reading code. It also implies its a constant for you- you shouldn’t be 
> modifying it, because then you’d be interfering with the function itself.
> 
> Side node: maybe it would be good to have an @icontract.nomutate (probably 
> use a different name, maybe @icontract.readonly) that makes sure a method 
> doesn’t mutate its own __dict__ (and maybe the __dict__ of the members of its 
> __dict__). It wouldn’t be necessary to put the decorator on every read only 
> function, just the ones your worried might mutate.
> 
> Maybe a @icontract.nomutate(param=“paramname”) that ensures the __dict__ of 
> all members of the param name have the same equality or identity before and 
> after. The semantics would need to be worked out.
> 
>> On Sep 26, 2018, at 8:58 AM, Marko Ristin-Kaufmann  
>> wrote:
>> 
>> Hi James,
>> 
>> Actually, following on #A4, you could also write those as multiple 
>> decorators:
>> @snpashot(lambda _, some_identifier: some_func(_, some_argument.some_attr)
>> @snpashot(lambda _, other_identifier: other_func(_.self))
>> 
>> Am I correct?
>> 
>> "_" looks a bit hard to read for me (implying ignored arguments).
>> 
>> Why uppercase "P" and not lowercase (uppercase implies a constant for me)? 
>> Then "O" for "old" and "P" for parameters in

[Python-ideas] Add .= as a method return value assignment operator

> items = ["foo", "bar", "quux"]
> items[randrange(3)] .= upper()
>
> Is this equivalent to:
>
> items[randrange(3)] = items[randrange(3)].upper()
>
> ? That would call randrange twice, potentially grabbing one element
> and dropping it into another slot. If it isn't equivalent to that, how
> is it defined?


It would not call randrange twice. Consider existing Python behavior:

def foo():
print("foo")
return 0
l = [7]
l[foo()] += 1
# output: "foo", but only once
print(l)  # l == [8]

Sent from my iPhone

> On Sep 27, 2018, at 4:13 PM, Chris Angelico  wrote:
>
> That would call randrange twice, potentially grabbing one element
> and dropping it into another slot. If it isn't equivalent to that, how
> is it defined?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add .= as a method return value assignment operator

> As I see it, you are mixing very different things. Augmented operators in
Python work on objects, generally trying to mutate them in-place. So
usually after these operations you have the same object (with the same
type, with the same name and etc.) as before these operations. Of course
there are exceptions, for example all immutable types or some promotions
between numbers.

Yep.

>>> a = (5, 2)
>>> a += (3, )
>>> a
(5, 2, 3)

> In your examples some cases imply that you are working on names, others
that you are working on objects. And as for me this ambiguity is not
solvable.

example please? Show an ambiguous case. To me, it's always working on
names.

On Thu, Sep 27, 2018 at 5:32 PM Kirill Balunov 
wrote:

> As I see it, you are mixing very different things. Augmented operators in
> Python work on objects, generally trying to mutate them in-place. So
> usually after these operations you have the same object (with the same
> type, with the same name and etc.) as before these operations. Of course
> there are exceptions, for example all immutable types or some promotions
> between numbers.
>
> In your examples some cases imply that you are working on names, others
> that you are working on objects. And as for me this ambiguity is not
> solvable.
>
> With kind regards,
> -gdg
>
> ср, 26 сент. 2018 г. в 14:14, Jasper Rebane :
>
>> Hi,
>>
>> When using Python, I find myself often using assignment operators, like
>> 'a += 1' instead of 'a = a + 1', which saves me a lot of time and hassle
>>
>> Unfortunately, this doesn't apply to methods, thus we have to write code
>> like this:
>> text = "foo"
>> text = text.replace("foo","bar")
>> # "bar"
>>
>> I propose that we should add '.=' as a method return value assignment
>> operator so we could write the code like this instead:
>> text = "foo"
>> text .= replace("foo","bar")
>> # "bar"
>> This looks cleaner, saves time and makes debugging easier
>>
>> Here are a few more examples:
>> text = " foo "
>> text .= strip()
>> # "foo"
>>
>> text = "foo bar"
>> text .= split(" ")
>> # ['foo', 'bar']
>>
>> text = b'foo'
>> text .= decode("UTF-8")
>> # "foo"
>>
>> foo =  {1,2,3}
>> bar = {2,3,4}
>> foo .= difference(bar)
>> # {1}
>>
>>
>> Rebane
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add .= as a method return value assignment operator

> I agree that this adds ambiguity where we can't be sure whether text .= 
> encode('utf-8') is referring to the function or the method. We can infer 
> that it *ought* to be the method, and maybe even add a rule to force 
> that, but this works only for the simple cases. It is risky and error 
> prone in the hard cases.
I think it would be a good idea to treat all global name lookups as lookups on 
the object on the LHS when you’re on the RHS of .=.

This behavior prevents the worst mistakes and makes it very clear what is 
happening.

Sent from my iPhone

> On Sep 27, 2018, at 8:03 PM, Steven D'Aprano  wrote:
> 
> On Thu, Sep 27, 2018 at 10:44:38PM +0300, Taavi Eomäe wrote:
>>> Do you see how this creates an ambiguous situation?
>> 
>> Name shadowing could create such ambiguous situations anyways even without
>> the new operator?
> 
> How? Can you give an example?
> 
> Normally, there is no way for a bare name to shadow a dotted name, or 
> vice versa, since the dotted name always needs to be fully qualified. In 
> the example below, we can talk about "text.encode" which is always the 
> method, or "encode", which is always the function and never the method.
> 
> I agree that this adds ambiguity where we can't be sure whether text .= 
> encode('utf-8') is referring to the function or the method. We can infer 
> that it *ought* to be the method, and maybe even add a rule to force 
> that, but this works only for the simple cases. It is risky and error 
> prone in the hard cases.
> 
> Think about a more complex assignment:
> 
>text .= encode(spam) + str(eggs)
> 
> This could mean any of:
> 
>text.encode(spam) + text.str(eggs)
>text.encode(spam) + str(eggs)
>encode(spam) + text.str(eggs)
>encode(spam) + str(eggs)
> 
> In a statically typed language, the compiler could work out what is 
> needed at compile-time (it knows that text.str doesn't exist) and either 
> resolve any such ambiguities or refuse to compile. But in a dynamic 
> language like Python, you can't tell whether text.str exists or not 
> until you try it.
> 
> So this proposal suffers from the same problems as Pascal-style "with" 
> blocks, which are a FAQ:
> 
> https://docs.python.org/3/faq/design.html#why-doesn-t-python-have-a-with-statement-for-attribute-assignments
> 
> [...]
 On 2018-09-27 14:13, Calvin Spealman wrote:
 Absolutely -1 on this. Consider the following example:
 
 def encode(s, *args):
 """Force UTF 8 no matter what!"""
 return s.encode('utf8')
 
 text = "Hello, there!"
 text .= encode('latin1')
> 
> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

I am fine with your proposed syntax. It’s certainly lucid. Perhaps it would be 
a good idea to get people accustomed to “non-magic” syntax.

> I still have a feeling that most developers would like to store the state in 
> many different custom ways. 
Please explain. (Expressions like thunk(all)(a == b for a, b in P.arg.meth()) 
would be valid.)
> I'm thinking mostly about all the edge cases which we would not be able to 
> cover (and how complex that would be to cover them).

Except for a > b > c being one flat expression with 5 members, it seems fairly 
easy to recreate an AST, which can then be compiled down to a code object. The 
code object can be fun with a custom “locals()”

Below is my concept code for such a P object.

from ast import *
# not done: enforce Singleton property on EmptySymbolType
class EmptySymbolType(object): ...
EmptySymbol = EmptySymbolType() # empty symbols are placeholders
class MockP(object):
# "^" is xor
@icontract.pre(lambda symbol, astnode: (symbol is None) ^ (astnode is None))
def __init__(self, symbol=None, value=EmptySymbol, astnode=None, 
initsymtable=(,)):
self.symtable = dict(initsymtable)
if symbol:
self.expr = Expr(value=Name(id=symbol, ctx=Load()))
self.symtable = {symbol: value}
else:
self.expr = astnode
self.frozen = False
def __add__(self, other):
wrapped = MockP.wrap_value(other)
return MockP(astnode=Expr(value=BinOp(self.expr, Add(), wrapped.expr), 
initsymtable={**self.symtable, **wrapped.symtable})
def compile(self): ...
def freeze(self):
# frozen objects wouldn’t have an overrided getattr, allowing for 
icontract to manipulate the MockP object using its public interface
self.frozen = True
@classmethod
def wrap_value(cls, obj):
   # create a MockP object from a value. Generate a random identifier and 
set that as the key in symtable, the AST node is the name of that identifier, 
retrieving its value through simple expression evaluation.
   ...

thunk = MockP.wrap_value
P = MockP('P')
# elsewhere: ensure P is only accessed via valid “dot attribute access” inside 
@snapshot so contracts fail early, or don’t and allow Magic like __dict__ to 
occur on P.

> On Sep 27, 2018, at 9:49 PM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> 
> I still have a feeling that most developers would like to store the state in 
> many different custom ways. I see also thunk and snapshot with wrapper 
> objects to be much more complicated to implement and maintain; I'm thinking 
> mostly about all the edge cases which we would not be able to cover (and how 
> complex that would be to cover them). Then the linters need also to work 
> around such wrappers... It might also scare users off since it looks like too 
> much magic. Another concern I also have is that it's probably very hard to 
> integrate these wrappers with mypy later -- but I don't really have a clue 
> about that, only my gut feeling?
> 
> What about we accepted to repeat "lambda P, " prefix, and have something like 
> this:
> 
> @snapshot(
>   lambda P, some_name: len(P.some_property),
>   lambda P, another_name: hash(P.another_property)
> )
> 
> It's not too verbose for me and you can still explain in three-four sentences 
> what happens below the hub in the library's docs.  A pycharm/pydev/vim/emacs 
> plugins could hide the verbose parts. 
> 
> I performed a small experiment to test how this solution plays with pylint 
> and it seems OK that arguments are not used in lambdas.
> 
> Cheers,
> Marko
> 
> 
>> On Thu, 27 Sep 2018 at 12:27, James Lu  wrote:
>> Why couldn’t we record the operations  done to a special object and replay 
>> them?
>> 
>>>>> Actually, I think there is probably no way around a decorator that 
>>>>> captures/snapshots the data before the function call with a lambda (or 
>>>>> even a separate function). "Old" construct, if we are to parse it somehow 
>>>>> from the condition function, would limit us only to shallow copies (and 
>>>>> be complex to implement as soon as we are capturing out-of-argument 
>>>>> values such as globals etc.). Moreove, what if we don't need shallow 
>>>>> copies? I could imagine a dozen of cases where shallow copy is not what 
>>>>> the programmer wants: for example, s/he might need to make deep copies, 
>>>>> hash or otherwise transform the input data to hold only part of it 
>>>>> instead of copying (e.g., so as to allow equality check without a double 
>>>>> copy of the data, or capture only the value of certain property 
>>>&

Re: [Python-ideas] You might find coconut language useful

Hi Marko,

I honestly don’t know how many people are using coconut. Though with a little 
bit of configuring Python’s import functionality, it should have decent inter 
compatibility. Even without Coconut, I still find icontract’s plain Python 
lambda syntax readable and useful. 

James Lu

> On Sep 27, 2018, at 9:32 PM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> 
> It would be super useful, particularly because lambdas can be written more 
> succinctly. You would suggest to write programs in coconut rather than 
> python? How many people are actually using it?
> 
>> On Thu, 27 Sep 2018 at 22:55, James Lu  wrote:
>> The language is a superset of Python that transpiles to Python. Let me know 
>> if you think its syntax would be useful for contracts.
>> 
>> Sent from my iPhone
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

> The problem with readability might be easier to solve than I thought, and 
> your pointer to coconut gave me the idea. What if we make a utility that 
> takes the python source code, examines the decorators pre/post/inv (or 
> whatever we call them) and transforms them back and forth from/to valid 
> python code?
> 
> Pretty much any IDE has after load / before save handlers. When you load a 
> python file, you'd transform it from less readable python code, write using a 
> concise form and when you save it, it's transformed back to valid python.
Hmm yes. Look at my previous email- the proposed syntax there does not require 
changing or transforming Python code. It’s missing many magic methods, but it 
would work something like this:

@snapshot(some_identifier=P.self.another_property + 10)

would internally turn into something like this: lambda P: 
P.self.another_property + 10

Maybe it’s possible to modify PyCharm parsing by altering its Python grammar 
file?

Sent from my iPhone

> On Sep 28, 2018, at 10:27 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> 
> The problem with readability might be easier to solve than I thought, and 
> your pointer to coconut gave me the idea. What if we make a utility that 
> takes the python source code, examines the decorators pre/post/inv (or 
> whatever we call them) and transforms them back and forth from/to valid 
> python code?
> 
> Pretty much any IDE has after load / before save handlers. When you load a 
> python file, you'd transform it from less readable python code, write using a 
> concise form and when you save it, it's transformed back to valid python.
> 
> Then we need to pick the python form that is easiest to implement (and still 
> readable enough for, say, code reviews on github), but writing and reading 
> the contracts in the code would be much more pleasant.
> 
> As long as the "readable" form has also valid python syntax, the tool can be 
> implemented with ast module.
> 
> For example:
> @snapshot(some_identifier=self.another_property + 10)
> @post(self.some_property => old.some_identifier > 100)
> 
> would transform into 
> @snapshot(lambda P, some_identifier: P.self.another_property + 10)
> @post(lambda O, P: not self.some_property and O.some_identifier > 100)
> 
> Cheers,
> Marko
> 
>> On Fri, 28 Sep 2018 at 03:49, Marko Ristin-Kaufmann  
>> wrote:
>> Hi James,
>> 
>> I still have a feeling that most developers would like to store the state in 
>> many different custom ways. I see also thunk and snapshot with wrapper 
>> objects to be much more complicated to implement and maintain; I'm thinking 
>> mostly about all the edge cases which we would not be able to cover (and how 
>> complex that would be to cover them). Then the linters need also to work 
>> around such wrappers... It might also scare users off since it looks like 
>> too much magic. Another concern I also have is that it's probably very hard 
>> to integrate these wrappers with mypy later -- but I don't really have a 
>> clue about that, only my gut feeling?
>> 
>> What about we accepted to repeat "lambda P, " prefix, and have something 
>> like this:
>> 
>> @snapshot(
>>   lambda P, some_name: len(P.some_property),
>>   lambda P, another_name: hash(P.another_property)
>> )
>> 
>> It's not too verbose for me and you can still explain in three-four 
>> sentences what happens below the hub in the library's docs.  A 
>> pycharm/pydev/vim/emacs plugins could hide the verbose parts. 
>> 
>> I performed a small experiment to test how this solution plays with pylint 
>> and it seems OK that arguments are not used in lambdas.
>> 
>> Cheers,
>> Marko
>> 
>> 
>>> On Thu, 27 Sep 2018 at 12:27, James Lu  wrote:
>>> Why couldn’t we record the operations  done to a special object and replay 
>>> them?
>>> 
>>>>>> Actually, I think there is probably no way around a decorator that 
>>>>>> captures/snapshots the data before the function call with a lambda (or 
>>>>>> even a separate function). "Old" construct, if we are to parse it 
>>>>>> somehow from the condition function, would limit us only to shallow 
>>>>>> copies (and be complex to implement as soon as we are capturing 
>>>>>> out-of-argument values such as globals etc.). Moreove, what if we don't 
>>>>>> need shallow copies? I could imagine a dozen of cases where shallow copy 
>>>>>> is not what the programmer wants: for example, s/he might need

[Python-ideas] Exception handling in contracts

Let’s get some ideas for how icontract can say “it should throw an exception if 
this happens.”
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why is design-by-contracts not widely adopted?

Many editors highlight decorators in a different color that makes it easier to 
ignore and can also fold decorators. 

Contracts can also sometimes actively improve the flow of code. 

I personally find a formal contract easier to read than informal documentation.

It also reduces the times where you need to spend time figuring out if 
documentation actually accurate and up to date
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

Going back to your proposal on repeating lambda P as a convention.

I do find 

@snapshot(some_identifier=P -> P.self(P.arg1),
some_identifier2=P -> P.arg1 + P.arg2)

acceptable. 

Should we keep some kind of document to keep track of all the different 
proposals? I’m thinking an editable document like HackMD where we can label all 
the different ideas to keep them straight in our head.

James Lu
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

> If the documentation is clear, I'd expect the user to be able to distinguish 
> the two. The first approach is shorter, and uses magic, but fails in some 
> rare situations. The other method is more verbose, but always works.
I like this idea. 

James Lu
> On Sep 29, 2018, at 1:36 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> If the documentation is clear, I'd expect the user to be able to distinguish 
> the two. The first approach is shorter, and uses magic, but fails in some 
> rare situations. The other method is more verbose, but always works.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

Regarding the “transpile into Python” syntax with with statements: Can I see an 
example of this syntax when used in pathlib? I’m a bit worried this syntax is 
too long and “in the way”, unlike decorators which are before the function 
body. Or do you mean that both MockP and your syntax should be supported?

Would

with requiring: assert arg1 < arg2, “message”

Be the code you type or the code that’s actually run?

James Lu

> On Sep 29, 2018, at 2:56 PM, Marko Ristin-Kaufmann  
> wrote:
> 
> Just
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

I just found the time to reply to these.
> I reread the proposal with MockP. I still don't get the details, but if I 
> think I understand the basic idea. You put a placeholder and whenever one of 
> its methods is called (including dunders), you record it and finally assemble 
> an AST and compile a lambda function to be executed at actual call later.
Precisely.

> But that would still fail if you want to have:
> @snapshot(var1=some_func(MockP.arg1, MockP.arg2))
> , right? Or there is a way to record that?

This would still fail. You would record it like this:
@snapshot(var1=thunk(some_func)(MockP.arg1, MockP.arg2))

thunk stores the function for later and produces another MockP object that 
listens for __call__.

By the way, MockP is the class while P is a virgin instance of MockP. MockP 
instances are immutable, so any operation on a MockP instance creates a new 
object or MockP instance. 

I’m also beginning to lean towards
@snapshot(var1=...)
@snapshot(var2=...)

I suspect this would deal better with VCS.

This syntax does have a a nice visual alignment. I’m not entirely sure what 
kind of indentation PEP 8 recommends and editors give, so the point may be moot 
if the natural indentation also gives the same visual alignment.

Though both should be supported so the best syntax may win.

James Lu

> On Sep 29, 2018, at 3:22 PM, Marko Ristin-Kaufmann  
> wrote:
> 
> I reread the proposal with MockP. I still don't get the details, but if I 
> think I understand the basic idea. You put a placeholder and whenever one of 
> its methods is called (including dunders), you record it and finally assemble 
> an AST and compile a lambda function to be executed at actual call later.
> 
> But that would still fail if you want to have:
> @snapshot(var1=some_func(MockP.arg1, MockP.arg2))
> , right? Or there is a way to record that?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Upgrade to Mailman 3

It has a nice GUI for people who spectate a discussion to read emails
without having to subscribe to the list.

http://docs.mailman3.org/en/latest/migration.html
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] "old" values in postconditions

Hi Marko,

Regarding switching over to GitHub issues:
* I copy-pasted the MockP original code to GitHub issues.
* There's a clunky way to view the discussion at
https://mail.python.org/pipermail/python-ideas/2018-September/subject.html#start
.
* The less clunky way to view the discussion is to subscribe to the mailing
list and use Gmail to move all the messages from python-ideas to python
ideas list and all the messages from the discussions we have to a
"contracts" label and view the discussion with your email client.
* A week earlier I didn't think I'd be saying this, but I like email for
discussion better. It works on mobile and I can send messages offline, and
I send 90% of my messages on my phone and when I'm offline. Unless you know
an alternative (WhatsApp, maybe?) that fits my use cases off the top of
your head, I think we should stick to email.
* My proposal: We split the discussion into a new email thread, we keep the
latest agreed upon proposal on GitHub issues.

On Sun, Sep 30, 2018 at 4:32 PM Marko Ristin-Kaufmann <
marko.ris...@gmail.com> wrote:

> Hi James,
> (I'm just about to go to sleep, so I'll answer the other messages
> tomorrow.)
>
> Should we keep some kind of document to keep track of all the different
>> proposals? I’m thinking an editable document like HackMD where we can label
>> all the different ideas to keep them straight in our head.
>>
>
> I thought github issues would be a suitable place for that:
> https://github.com/Parquery/icontract/issues
>
> It reads a bit easier as a discussion rather than a single document -- if
> anybody else needs to follow. What do you think?
>
> On Sun, 30 Sep 2018 at 22:07, James Lu  wrote:
>
>> Hi Marko,
>>
>> Going back to your proposal on repeating lambda P as a convention.
>>
>> I do find
>>
>> @snapshot(some_identifier=P -> P.self(P.arg1),
>> some_identifier2=P -> P.arg1 + P.arg2)
>>
>> acceptable.
>>
>> Should we keep some kind of document to keep track of all the different
>> proposals? I’m thinking an editable document like HackMD where we can label
>> all the different ideas to keep them straight in our head.
>>
>> James Lu
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Transpiling contracts

2018-10-01 Thread James Lu

Hi Marko,

I’m going to refer to the transpiler syntax as “block syntax.”

1. How does Eiffel do formal contracts?
2. What’s the use case for preferring block syntax over lambda syntax? Is the 
block syntax only better when multiple statements are needed to test a single 
contract condition? 
2. If the last proposition is true, how often do we need multiple statements to 
test or specify a single contract condition?
3. When we do need multiple statements to formally verify/specify, is it 
usually a better idea to factor out major functionality of the contract testing 
to an external function? (So other contracts can use the same external function 
and the intent of the contract is clear trough the function name)
4. If this is true, is it a good idea to include a contracts/ folder at the 
same folder level or a .contracts.py file to list “helper” or verification 
functions for a contract?

I think we should answer questions two and three by looking at real code.
—

If MockP is supported, it could be used for all the contract decorators:
@snapshot(var=P.self.name)
@post(var=R.attr == P.old.var)

The decorator can also check the type of the returned object, and if it’s MockP 
it will use MockP protocol and otherwise it will check the object is callable 
and treat it as a lambda.

Your approach has better type hinting, but I’m not sure if type hinting would 
be that useful if you’re adding a contract whilst writing the function. 

James Lu

> On Oct 1, 2018, at 1:01 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi James,
> 
>> Regarding the “transpile into Python” syntax with with statements: Can I see 
>> an example of this syntax when used in pathlib? I’m a bit worried this 
>> syntax is too long and “in the way”, unlike decorators which are before the 
>> function body. Or do you mean that both MockP and your syntax should be 
>> supported?
>> 
>> Would
>> 
>> with requiring: assert arg1 < arg2, “message”
>> 
>> Be the code you type or the code that’s actually run?
> 
> 
> That's the code you would type. Let me make a full example (I'm omitting 
> "with contracts" since we actually don't need it).
> 
> You would read/write this:
> def some_func(arg1: List[int])->int:
>   with requiring:
> assert len(arg1) > 3, "some description"
> 
>   with oldie as O, resultie as result, ensuring:
> if SLOW:
>   O.var1 = sum(arg1)
>   assert result > sum(arg1) > O.var1
> 
> assert len(result) > 5
> 
> This would run:
> @requires(lambda P: len(P.arg1) > 3, "some description")
> @snapshot(lambda P, var1: sum(P.arg1), enabled=SLOW)
> @ensures(lambda O, P, result: result > sum(arg1) > O.var1, enabled=SLOW)
> @ensures(lambda result: len(result) > 5)
> 
> I omitted in the example how to specify a custom exception to avoid confusion.
> 
> If we decide that transpiler is the way to go, I'd say it's easier to just 
> stick with lambdas and allow no mock-based approach in transpilation.
> 
> Cheers,
> Marko
>  
> 
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Renaming icontract to pcontract

2018-10-01 Thread James Lu

I think the confusion would be minimal and not worth the hassle of renaming to 
pcontract. People would just say “Python icontract” when it’s ambiguous.

Sent from my iPhone

> On Oct 1, 2018, at 3:28 AM, Marko Ristin-Kaufmann  
> wrote:
> 
> Hi Cameron,
> A nerdy way to make it sound like a sentence: "I contract that ...".
> 
> Pcontract would stand for python contract. Pycontract is already taken.
> 
> Cheers,
> Marko
> 
>> Le lun. 1 oct. 2018 à 09:15, Cameron Simpson  a écrit :
>> On 01Oct2018 07:25, Marko Ristin-Kaufmann  wrote:
>> >I'd like to rename icontract into pcontract to avoid name conflict with
>> >java's icontract library.
>> >
>> >Do you have any better suggestion?
>> 
>> No, sounds ok to me. What was the "i" for in the old name?
>> 
>> Cheers,
>> Cameron Simpson 
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Support parsing stream with `re`

2018-10-12 Thread James Lu

The file system is really just a b-tree. If you’re concerned about using memory, you can implement a O(log n) map using the file system, where the entires are the different critical sections.Every node is a folder and every file is a leaf. Many package managers implement maps like this. I’d like to propose a second alternative. You can use a script that’s responsible for creating the file of commands. The file is actually a FIFO pipe. The script streams data to the file, but to save resources it stops when the file/buffer reaches a certain size. The command-running code works in a different process. Python has good capability for handling streams. On Oct 10, 2018, at 1:27 AM, Stephen J. Turnbull wrote:Chris Angelico writes:On Wed, Oct 10, 2018 at 5:09 AM Stephen J. Turnbull wrote:Chris Angelico writes:Both processes are using the virtual memory. Either or both could beusing physical memory. Assuming they haven't written to the pages(which is the case with executables - the system mmaps the binary intoyour memory space as read-only), and assuming that those pages arebacked by physical memory, which process is using that memory?One doesn't know. Clever side-channel attacks aside, I don't care,and I don't see how it could matter.It matters a lot when you're trying to figure out what your systemis doing.Sure, but knowing how your system works is far more important. Eg,create a 1TB file on a POSIX system, delete it while a process stillhas it opened, and it doesn't matter how you process the output of duor ls, you still have 1TB of used file space not accounted for. Thesame applies to swapfiles. But "df" knows and will tell you.In fact, "ps" will tell you how much shared memory a process is using.I just don't see a problem here, on the "I'm not getting the data Ineed" side. You do have access to the data you need.Tell me, which process is responsible for libc being in memory?Other than, like, all of them?Yes. Why would you want a different answer?Because that would mean that I have way more *physical* memory in usethan I actually have chips on the motherboard for.No, that's like saying that because you have multiple links to a fileon disk you're using more physical disk than you have.Actually, that's exactly the same problem, with exactly the sameconsequences. How do you figure out why your disk is full? How do youenforce disk quotas? How can you get any sort of reasonable metricsabout anything when the sum of everything vastly exceeds the actualusage?You add up the right things, of course, and avoid paradoxes.The disk quota enforcement problem is indeed hard. This sounds to melike a special problem studied in cost accounting, a problem which wassolved (a computation that satisfies certain axioms was shown to existand be unique) in a sense by Aumann and Shapley in the 1970s. The A-Sprices have been used by telephone carriers to allocate costs of fixedassets with capacity constraints to individual calls, though I don'tknow if the method is still in use. I'm not sure if the disk quotaproblem fits the A-S theorem (which imposes certain monotonicityconditions), but the general paradigm does.However, the quota problem (and in general, the problem of allocationof overhead) is "hard" even if you have complete information about thesystem, because it's a values problem: what events are bad? whatevents are worse? what events are unacceptable (result in bankruptcyand abandonment of the system)? Getting very complete, accurateinformation about the physical consequences of individual events inthe system (linking to a file on disk, allocating a large quantity ofviritual memory) is not difficult, in the sense that you throw moneyand engineers at it, and you get "df". Getting very complete,accurate information about the values you're trying to satisfy ispossible only for an omniscient god, even if, as in business, they canbe measured in currency units.Steve___Python-ideas mailing listPython-ideas@python.orghttps://mail.python.org/mailman/listinfo/python-ideasCode of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Brainstorm] Testing with Documented ABCs

2018-12-08 Thread James Lu

> Interesting. In the thread you linked on DBC, it seemed like Steve
D'Aprano and David Mertz (and possibly others) were put off by the
verbosity and noisiness of the decorator-based solution you provided with
icontract (though I think there are ways to streamline that solution). It
seems like syntactic support could offer a more concise and less noisy
implementation.

Btw, it would be relatively easy to create a parser for Python. Python
doesn't have any crazy grammar constructs like the lexer hack
 AFAIK. I'm imagining using
Bison:
1. convert python's grammar (
https://github.com/python/cpython/blob/master/Lib/lib2to3/Grammar.txt) to
Bison format.
2. write a lexer to parse tokens and convert indentation to indent/dedent
tokens.
3. extend the grammar however you want it. Call these custom AST nodes
"contract nodes."
4. create a simple AST, really an annotated parse tree. I think we can use
a simple one that's a bunch of nested lists:

["for_stmt", "for i in range(10):",
[
 ["exprlist", "i", [ ... ]],
 ["testlist", "range(10)", [ ... ]]
]]
# ["node_type", "", ]

The AST can be made more detailed on an as-needed basis.
5. traverse the AST, and "rewrite" the the AST by pasting traditional
python AST nodes where contract nodes are.

This example from the Babel handbook may help if you have trouble
understanding what this step means.
https://github.com/jamiebuilds/babel-handbook/blob/master/translations/en/plugin-handbook.md#toc-writing-your-first-babel-plugin

6. turn the AST back into python source. Since we're storing the source
code from the beginning, this should be fairly easy. (Bison lets your lexer
tell the parser the line and column numbers of each token.)

---

I made a joke language with Bison once, it's really flexible and
well-suited for this kind of task. This 6-step p
Tip: I found Bison's C++ mode too complicated, so I used it in C mode with
the C++ Standard Library and C++ references enabled.

---
I'm interested, what contract-related functionality do you think Python's
existing syntax is inadequate for? You could look into using with
statements and a python program that takes the AST and snips
contract-related with statements to produce optimized code, though I
suppose that's one step below the custom-parser method.

On Wed, Nov 28, 2018 at 3:29 PM Abe Dillon  wrote:

> [Marko Ristin-Kaufmann]
>>
>> Have you looked at the recent discussions regarding design-by-contract on
>> this list
>
>
> I tried to read through them all before posting, but I may have missed
> some of the forks. There was a lot of good discussion!
>
> [Marko Ristin-Kaufmann]
>
>> You might want to have a look at static checking techniques such as
>> abstract interpretation. I hope to be able to work on such a tool for
>> Python in some two years from now. We can stay in touch if you are
>> interested.
>
>
> I'll look into that! I'm very interested!
>
> [Marko Ristin-Kaufmann]
>
>> Re decorators: to my own surprise, using decorators in a larger code base
>> is completely practical including the  readability and maintenance of the
>> code. It's neither that ugly nor problematic as it might seem at first look.
>
>
> Interesting. In the thread you linked on DBC, it seemed like Steve
> D'Aprano and David Mertz (and possibly others) were put off by the
> verbosity and noisiness of the decorator-based solution you provided with
> icontract (though I think there are ways to streamline that solution). It
> seems like syntactic support could offer a more concise and less noisy
> implementation.
>
> One thing that I can get on a soap-box about is the benefit putting the
> most relevant information to the reader in the order of top to bottom and
> left to right whenever possible. I've written many posts about this. I
> think a lot of Python syntax gets this right. It would have been easy to
> follow the same order as for-loops when designing comprehensions, but
> expressions allow you some freedom to order things differently, so now
> comprehensions read:
>
> squares = ...
> # squares is
>
> squares = [...
> # squares is a list
>
> squares = [number*number...
> # squares is a list of num squared
>
> squares = [number*number for num in numbers]
> # squares is a list of num squared 'from' numbers
>
> I think decorators sort-of break this rule because they can put a lot of
> less important information (like, that a function is logged or timed)
> before more important information (like the function's name, signature,
> doc-string, etc...). It's not a huge deal because they tend to be
> de-emphasized by my IDE and there typically aren't dozens of them on each
> function, but I definitely prefer Eiffel's syntax
>  over
> decorators for that reason.
>
> I understand that syntax changes have an very high bar for very good
> reasons. Hillel Wayne's PyCon talk got me thinking that we might be close
> enough to a really great solution to

[Python-ideas] Backtick expression: similar to a shorter lambda syntax

2019-01-20 Thread James Lu

Backtick expressions work exactly like lambdas, except that they are bound to 
the instance they are created in every time that class is used to create one. 
To illustrate, this “percent” property is bound to the instance, not to the 
class.
class Example:
percent = property(`self.v*self.v2/100`)

And a few more examples for clarity.

def example():
locals()['a'] = 1
expr = `a+1`
return expr() # error: one variable is required

Any variable names that exist when the backtick expression is created are bound 
to the expression, and the reference to the expression is stored within the 
expression. Names that do not exist when the expresssion is created must be 
passed in as parameters. Such names can also be passed in as keyword arguments. 
Backtick expressions are created when their scope is created.

Variable names that are declared but have not been assigned to will be 
considered to exist for the purposes of the backtick expression.

Directly calling a backtick expression as soon as it’s created is forbidden:

`v+1`(a)

But this is technically allowed but discouraged, like how := works:
(`v+1`)(a)

Use Cases

This can be used anywhere a lambda would feel “heavy” or long. Here are a few 
use cases where using a backtick expression would allow code to be 
significantly mote readable:

If/else chains that would be switch statements.
Creating decorators.
Passing in logging hooks.
Writing design-by-contract contracts. (See icontract on GitHub for an example 
of what DBC looks like in Python.)
Tests and assertions.

Additionally, the instance binding enables:

A shorthand way to create a class that wraps an API to a better or more uniform 
code interface. Previously you’d need to make defs and @property, now each 
wrapped property and method is a single, readable line of code.

Appendix

I propose syntax highlighters show a backtick expression on a different 
background color, a lighter shade of black for a dark theme; dirty white for a 
light thing.

I also propose the following attributes on the backtick expression.

__str__(): the string [parameter names separated by commas] => [the string of 
the backtick expression]
__repr__(): the original string of the backtick expression, surrounded by 
backticks.

I secondarily propose that backtick expressions are only bound to their 
instances when defined within a class when the following syntax is used:

def a = 

—
Now, let’s bikeshed.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Discussion: Duck typing with “concepts”

So here’s an interesting idea, not a proposal yet. 

In C++20, a Concept is a list of Boolean expressions with a name that can be 
used in place of a type in a templated (ie type-generic) function.

from typing import Concept
Iterator = Concept(lambda o: hasattr(o, "__iter__", lambda o: iter(o) != 
NotImplemented)
# Concept inheritance
Iterable = Concept(lambda o: hasattr(o, "__next__"), Iterator)

You could use concepts to define many practical “real-world” duck types.

A concept is like an opt-in duck typing type assertion. Since it’s a part of 
type annotation syntax, assertions can be generated automatically by looking at 
assertions.

You would use a Concept like any other type in type annotations.

Marko, how do you think Concepts might integrate with icontract? (I’m imagining 
overriding an import hook to automatically add contracts to functions with 
concepts.) How frequently do you use duck typing at Parquery? How might 
Concepts affect how often you used duck typing?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Backtick expression: similar to a shorter lambda syntax

> On Jan 21, 2019, at 1:56 AM, Steven D'Aprano  wrote:
> 
> It disturbs me that you believe you get to tell everyone what syntax 
> highlighting they should use for this feature. That's pretty 
> dictatorial, and not in a good BDFL way.

I don’t want to tell anyone how to make their syntax highlighter- just to point 
out that there are good ways to make the code within backtick expressions 
visually distinct from the surrounding code. I told the list because it’s 
helpful for the readability discussion.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Backtick expression: similar to a shorter lambda syntax

> On Jan 21, 2019, at 1:56 AM, Steven D'Aprano  wrote:
> 
> It disturbs me that you believe you get to tell everyone what syntax 
> highlighting they should use for this feature. That's pretty 
> dictatorial, and not in a good BDFL way.

I don’t want to tell anyone how to make their syntax highlighter- just to point 
out that there are good ways to make the code within backtick expressions 
visually distinct from the surrounding code. I told the list because it’s 
helpful for the readability discussion.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Backtick expression: similar to a shorter lambda syntax

Later today I will send a working implementation of backtick expressions as a 
function call.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Backtick expression: similar to a shorter lambda syntax

I’m a little busy recently, so I’ll reply to as much as I can now and reply to 
the rest later.

Scratch the stuff I said about scope. Backtick expressions should inherit the 
scope normally like any other nested function.

> That's different behaviour from regular functions, where names are only 
> resolved when the function is called.

What benefits are there to the late name lookup of normal functions? I’m 
looking to have backtick expressions raise early, not late. 

We can relax the names to be optional if a ternary expression is used within 
the backtick expression.

I realized that the behavior of Backtick Expressions can be silently affected 
by global variables. Example:

x = 1 
def increment_each(l):
 return map(`x+1`, l)

## Explicit expression, implicit usage
Explicit backtick expressions are ones that, for all parameters that the 
created function produces, it uses a caret before the name of the parameter. 
The other names must exist when the backtick expression is evaluated. Example:

parameter = 0
is_integer = `int(^parameter) == ^parameter` # arity: 1 despite the global 
definition

self = 'James'
str = `^self.__class__.__str__(^self)` # arity: 1 despite the global definition
str(type(lambda: 1)) # use our custom str function on the function type; 
output: 

## Implicitly Created Backtick Expression
Implicit baktick expressions, ones that mention undefined parameters without 
using the caret mark are generally discouraged. However they create 
UncastBacktickExpression, which must be cast using the .to_safe(*names) method 
to be used, which takes in a list of parameter names and outputs a normal 
backtick expression.

Even if the variable is defined on a global level, it can be explicitly 
overridden in to_safe.

Example 1
`x+1`.to_safe('x')(1)  # output: 2

Example 2
x = 0
`x+1`.to_safe('x')(1)  # output: 2

If a backtick expression has no unspecified names and has no carets, it is an 
implicit backtick expression.

This allows developers to safely omit the ^ when the code that is using the 
resulting backtick expression is aware of the parameters to be used, given that 
it's obvious to the developer which names are parameters.

> On Jan 21, 2019, at 1:56 AM, Steven D'Aprano  wrote:
> 
> That's different behaviour from regular functions, where names are only 
> resolved when the function is called.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Backtick expression: similar to a shorter lambda syntax

2019-01-23 Thread James Lu

Backtick expressions (now) use the same scoping and same binding rules as
other functions. The only difference is that
class Class:
  stacticmethod = `...`
  staticmethod = lambda: ...
  def instancemethod = `...` # an instancemethod that's called with self
passed in
  def property = property(`...`) # an instancemethod that's called with
self passed in

> The only thing that I can think of is that you want `foo + ^bar` to be
another way of writing lambda bar: foo + bar with some under-specified behavior
for evaluating foo and different under-specified behavior for evaluating bar
.

That is what `lambda bar: foo + ^bar` means.

A caret in a backtick expression indicates that the name after the caret is
a parameter. All names with the same name must have a caret before them.
Mandatory parameters can be passed in as keyword arguments or as positional
ones.

As for the under-specification, I've been working on an example
implementation I'll send soon for backtick expressions.

I've also been doing the "look for use cases in stdlib" thing that
Johnathan and Steve mentioned.


On Wed, Jan 23, 2019 at 3:02 AM Bruce Leban  wrote:

> On Sun, Jan 20, 2019 at 6:43 PM James Lu  wrote:
>
>> Backtick expressions work exactly like lambdas, except that they are
>> bound to the instance they are created in every time that class is used to
>> create one. To illustrate, ...
>
>
> First, if there is a useful procedure I am strongly against using
> backticks because (1) it's been used in the past with an entirely different
> meaning and (2) it looks ugly and is not visually suggestive at all of what
> it does, especially not the subtle difference between other function
> definitions.
>
> Second, I don't understand exactly what this difference or why it would be
> useful. It would help for you to give examples comparing lambda and this
> variation.
>
> Third, you mention using ^ in "explicit" expressions to refer to
> parameters of the "created function" and I do not know what function you
> are referring to or what the exact semantics of this are. Again, a
> comparison of two expressions with and without that ^ would help. An
> expression is not a function and not all expressions are written inside
> functions. (And as to the specific proposed syntax, there already is the ^
> xor operator and the most expected meaning of ^value is ~value. just as
> the unary + and - operators corresponds to the binary operators.
>
> The only thing that I can think of is that you want `foo + ^bar` to be
> another way of writing lambda bar: foo + bar with some under-specified 
> behavior
> for evaluating foo and different under-specified behavior for evaluating
> bar.
>
> Finally, if there is some other useful semantics for references inside a
> function definition, then I would think the best way to do that is to
> implement that, not add a new function difference. For example,
>
> lambda foo: foo + $bar
>
> def sample(foo):
>
> return foo + $foo
>
>
> where I'm arbitrarily using $ to represent the new semantics whatever they
> are (no point in bikeshedding syntax when semantics are yet to be defined).
>
> --- Bruce
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Option of running shell/console commands inside the REPL

I always use ptipython (ptpython shell over the ipython console) for my
REPLs. The built-in python repl is not *batteries included* in the sense
that it already has what you need to explore the language.

I wonder, what do the python committers think about including a
stripped-down version of ipython and ptpython in the built-in python?

On Fri, Feb 1, 2019 at 6:15 AM Lele Gaifax  wrote:

> Further magic comes with https://pypi.org/project/xonsh/
>
> ciao, lele.
> --
> nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
> real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
> l...@metapensiero.it  | -- Fortunato Depero, 1929.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Clearer communication

A lot of the traffic on this email list is people saying “I don’t understand” 
or “that’s not what I meant” or trying to re-explain. A lot of “-1”s are really 
“I don’t see the usefulness of this”.


So I want an open discussion on: How can we communicate clearer?


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] How do ideas on this mailing list turn into features?

How do ideas on this mailing list turn into features? What is the typical 
roadmap? Can this process be documented somewhere?___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Option of running shell/console commands inside the REPL

It’s difficult to learn anything with a body (such as a loop or a class or a 
function) with the built in REPL because you can’t edit lines you’ve already 
written.

Sent from my iPhone

> On Feb 1, 2019, at 1:44 PM, Steven D'Aprano  wrote:
> 
>> On Fri, Feb 01, 2019 at 09:07:14AM -0500, James Lu wrote:
>> 
>> I always use ptipython (ptpython shell over the ipython console) for my
>> REPLs. The built-in python repl is not *batteries included* in the sense
>> that it already has what you need to explore the language.
> 
> What part of the Python language do you think cannot be used in the 
> standard Python REPL?
> 
> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

> I want the discussion to focus not only on technical solutions like +1 or 
> Mailman 3, but also social ones and how to better express oneself.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

It’s very demotivating to hear just negative feedback on this list.

Was starting this thread useful for y’all?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] What factors led Guido to quit?

I want y’all to think about this very carefully. What factors led Guido to 
quit? And I don’t want you to just reply with the first thing that comes off 
your head. The purpose of this question/discussion is to identify problems with 
the Python community so we can fix them.

That is the only real chance of having a successful Python language that 
doesn’t get left behind by other languages, or of getting Guido to come back.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

This list IS hard for newcomers. I wish there was one place where I could read 
up on how to not feel like a noob. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

I think we need to step away from the egalitarian ideal and have some way of 
knowing that Python committees are python committers. It’s really difficult to 
know how well your proposal is doing without having this.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

There is no need for any of you to argue over this small point. Tolerate each 
other’s language.

Sent from my iPhone

> On Feb 2, 2019, at 3:58 AM, Steven D'Aprano  wrote:
> 
>> On Sat, Feb 02, 2019 at 05:10:14AM +, MRAB wrote:
>>> On 2019-02-02 04:32, Steven D'Aprano wrote:
>>> [snip]
>>> 
>>> Of course it makes sense. Even numpy supports inhomogeneous data:
>>> 
>> [snip]
>> 
>> "inhomogeneous"? Who came up with that?
> 
> I don't know, but it has been used since at least the early 1920s
> 
> https://english.stackexchange.com/questions/194906/heterogeneous-vs-inhomogeneous
> 
> and the Oxford dictionary describes "inhomogenity" as being used from 
> the late 19th century. So my guess is, probably people who were more 
> familiar with Latin and Greek than we are.
> 
> There are many words that are derived from both Latin and Greek. There's 
> no rule that says that because a word was derived from Greek, we must 
> use Greek grammatical forms for it. We are speaking English, not Greek, 
> and in English, we can negate words using the "in" prefix.
> 
> 
> 
> -- 
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Consistency in naming [was Re: ...ALL CAPS]



Sent from my iPhone

> On Feb 2, 2019, at 3:41 AM, Steven D'Aprano  wrote:
> 
>> On Sat, Feb 02, 2019 at 12:06:47AM +0100, Anders Hovmöller wrote:
>> 
>>> - the status quo means "no change", so there is no hassle there;
>> 
>> Not quite true. There is a constant hassle of "do I need to write 
>> datetime.datetime.now() or datetime.now()?"
> 
> My point was that there is no hassle from *making a change* if you don't 
> actually make a change. (There may, or may not, be other, unrelated 
> hassles.)
> 
> Besides, I'm not seeing that this is any worse than any other import. Do 
> I call spam.Eggs.make() or Eggs.make()? If you don't remember what you 
> imported, the names don't make much difference.
> 
> I accept that datetime.datetime reads a bit funny and is a bit annoying. 
> If we had the keys to the time machine and could go back a decade to 
> version 3.0, or even further back to 1.5 or whenever the datetime module 
> was first created, it would be nice to change it so that the class was 
> DateTime. But changing it *now* is not free, it has real, serious costs 
> which are probably greater than the benefit gained.
Why can’t we put “now” as a property of the module itself, reccomend that, and 
formally deprecate but never actually remove datetime.datetime.now?
> 
>> I solved this at work by changing all imports to follow the "from 
>> datetime import datetime" pattern and hard banning the other 
>> statically in CI. But before that people suffered for years.
> 
> Oh how they must have suffered *wink*
> 
> I'm surprised that you don't do this:
> 
> from datetime import datetime as DateTime
> 
> 
>> I have a colleague who likes to point that the future is longer than 
>> the past. It's important to keep that perspective.
> 
> Actually, no, on average, the projected lifespan of technologies, 
> companies and cultural memes is about the same as their current age. It 
> might last less, or it might last more, but the statistical expectation 
> is about the same as the current age. So on average, "the future" is 
> about the same as "the past".
> 
> Python has been around not quite 30 years now, so we can expect that it 
> will probably last another 30 years. But chances are not good that it 
> will be around in 300 years.

A big reason why projects last as long as you say they last is that the 
maintainers get un-ambitious, they get used to relaxing in the language they 
know so well, they are no longer keen on change.

This kind of readability issue, datetime.now, is an example of what’s 
contributing to Python’s decline.

Bottom line: if someone submits a PR for this, will anyone merge it?
> 
> -- 
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] What factors led Guido to quit?

Are you a moderator or committed?

I disagree with you. This, like the communication thread, is about improving 
the ability of the Python community to work together. If you don’t think others 
can have a respectful discussion about that, that’s the community’s fault, not 
mine.

Again, the goal of this thread is for constructive discussion.

Sent from my iPhone

> On Feb 3, 2019, at 9:44 AM, Antoine Pitrou  wrote:
> 
> 
> James, please don't post such topics on python-ideas.  This is
> completely off-topic and has strong flamebait potential.  You may write
> on your own blog or Facebook account if you feel so strongly about it.
> 
> Regards
> 
> Antoine.
> 
> 
> On Sat, 2 Feb 2019 11:04:40 -0500
> James Lu  wrote:
>> I want y’all to think about this very carefully. What factors led Guido to 
>> quit? And I don’t want you to just reply with the first thing that comes off 
>> your head. The purpose of this question/discussion is to identify problems 
>> with the Python community so we can fix them.
>> 
>> That is the only real chance of having a successful Python language that 
>> doesn’t get left behind by other languages, or of getting Guido to come back.
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Consistency in naming [was Re: ...ALL CAPS] (off-list)

Python’s decline is in not growing. 

Sent from my iPhone

> On Feb 3, 2019, at 11:20 AM, Ned Batchelder  wrote:
> 
> James, you say below, "This kind of readability issue, datetime.now, is an 
> example of what’s contributing to Python’s decline."
> 
> Do you have any evidence of Python's decline?  Lots of metrics (albeit 
> simplistic ones) point to Python growing in popularity:
> 
> https://www.techrepublic.com/article/fastest-growing-programming-language-pythons-popularity-is-still-climbing/
> https://www.netguru.com/blog/why-python-is-growing-so-quickly-future-trends
> https://www.economist.com/graphic-detail/2018/07/26/python-is-becoming-the-worlds-most-popular-coding-language
> Are there indicators we are missing?
> 
> --Ned.
> 
>> On 2/2/19 11:56 PM, James Lu wrote:
>> Sent from my iPhone
>> 
>>>> On Feb 2, 2019, at 3:41 AM, Steven D'Aprano  wrote:
>>>> 
>>>>> On Sat, Feb 02, 2019 at 12:06:47AM +0100, Anders Hovmöller wrote:
>>>>> 
>>>>> - the status quo means "no change", so there is no hassle there;
>>>> Not quite true. There is a constant hassle of "do I need to write 
>>>> datetime.datetime.now() or datetime.now()?"
>>> My point was that there is no hassle from *making a change* if you don't 
>>> actually make a change. (There may, or may not, be other, unrelated 
>>> hassles.)
>>> 
>>> Besides, I'm not seeing that this is any worse than any other import. Do 
>>> I call spam.Eggs.make() or Eggs.make()? If you don't remember what you 
>>> imported, the names don't make much difference.
>>> 
>>> I accept that datetime.datetime reads a bit funny and is a bit annoying. 
>>> If we had the keys to the time machine and could go back a decade to 
>>> version 3.0, or even further back to 1.5 or whenever the datetime module 
>>> was first created, it would be nice to change it so that the class was 
>>> DateTime. But changing it *now* is not free, it has real, serious costs 
>>> which are probably greater than the benefit gained.
>> Why can’t we put “now” as a property of the module itself, reccomend that, 
>> and formally deprecate but never actually remove datetime.datetime.now?
>>>> I solved this at work by changing all imports to follow the "from 
>>>> datetime import datetime" pattern and hard banning the other 
>>>> statically in CI. But before that people suffered for years.
>>> Oh how they must have suffered *wink*
>>> 
>>> I'm surprised that you don't do this:
>>> 
>>> from datetime import datetime as DateTime
>>> 
>>> 
>>>> I have a colleague who likes to point that the future is longer than 
>>>> the past. It's important to keep that perspective.
>>> Actually, no, on average, the projected lifespan of technologies, 
>>> companies and cultural memes is about the same as their current age. It 
>>> might last less, or it might last more, but the statistical expectation 
>>> is about the same as the current age. So on average, "the future" is 
>>> about the same as "the past".
>>> 
>>> Python has been around not quite 30 years now, so we can expect that it 
>>> will probably last another 30 years. But chances are not good that it 
>>> will be around in 300 years.
>> A big reason why projects last as long as you say they last is that the 
>> maintainers get un-ambitious, they get used to relaxing in the language they 
>> know so well, they are no longer keen on change.
>> 
>> This kind of readability issue, datetime.now, is an example of what’s 
>> contributing to Python’s decline.
>> 
>> Bottom line: if someone submits a PR for this, will anyone merge it?
>>> -- 
>>> Steve
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication

Well, the question wasn’t about any specific proposal but improving 
communication in general. I don’t have a specific straw man.


Sent from my iPhone

> On Feb 2, 2019, at 7:00 PM, Steven D'Aprano  wrote:
> 
>> On Sat, Feb 02, 2019 at 10:59:36AM -0500, James Lu wrote:
>> 
>> I think we need to step away from the egalitarian ideal and have some 
>> way of knowing that Python committees are python committers. It’s 
>> really difficult to know how well your proposal is doing without 
>> having this.
> 
> Speaking of clearer communication (see the subject line), who are you 
> referring to and which proposal do you mean?
> 
> 
> -- 
> Steven
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Clearer communication


>> On Feb 2, 2019, at 7:17 PM, Steven D'Aprano  wrote:
>> 
>> On Sat, Feb 02, 2019 at 11:12:02AM -0500, James Lu wrote:
>> 
>> This list IS hard for newcomers. I wish there was one place where I 
>> could read up on how to not feel like a noob.
> 
> It has become unfashionable to link to this, because it is allegedly too 
> elitest and unfriendly and not welcoming enough, but I still think it is 
> extremely valuable:
> 
> http://www.catb.org/esr/faqs/smart-questions.html
> 
> (Its also somewhat unpopular because the maintainer has become 
> something of a politically Right-wing extremist.)
> 
> If you think of *proposals* as a kind of question:
> 
>   "What are the Pros and Cons of this suggestion?"
> 
> rather than
> 
>   "We must do this. I have spoken, make it so!"
> 
> then the Smart Questions document is relevant. Do your research first. 
> What have you tried? Have you tried to knock holes in your own proposal 
> or are you so excited by the Pros that you are blind to the Cons? What 
> do other languages do? Do they differ from Python in ways which matter 
> to your proposal?
> 
> Did you make even a feeble attempt to search the archives, 
> Stackoverflow, etc, or just post the first ill-formed thought that came 
> to your mind?
> 
> If your proposal been asked before, unless you are bringing something 
> new to the discussion, don't waste everyone's time covering old ground.


That’s some great stuff, can we document it somewhere? I think it would benefit 
future proposers.

I’ve been subconsciously aware of what you said but haven’t been able to 
express it.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Mention more alternative implementations on the PSF website

https://www.python.org/download/alternatives/ should possibly mention:

- Cython and Nuitka
- Mention the possibility of compiling Python to WASM
   - WASM allows Web and Mobile use of Python at possibly native speed.
Though not mature yet, Pyodide is a working implementation.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Consistency in naming [was Re: ...ALL CAPS] (off-list)

I agree with everything all of you have said in reply to me.

Sent from my iPhone

> On Feb 3, 2019, at 7:34 PM, Ned Batchelder  wrote:
> 
>> On 2/3/19 6:01 PM, Steven D'Aprano wrote:
>> (1) Taking the group discussion off-list should be done rarely, and
>> usually only for personal messages that shouldn't be shared
>> publically, or by mutual agreement for off-topic discussions.
>> 
>> I can't see Ned's original comment in either my inbox, or at the
>> archives, so it seems that he took the discussion off-list. I don't know
>> why. It doesn't strike me as either personal or off-topic, so taking it
>> off-list seems to be both unnecessary and a little rude to the rest of
>> the group -- why were the rest of us excluded?
>> 
>> But since Ned apparently intended to take it off-list, it is only polite
>> to respect that your reply.
>> 
>> 
>> (2) The long-standing tradition is to put "OFFLIST" at the *start* of
>> the subject line, not the end where it is easy to overlook.
>> 
>> To make it even more clear, we should explicitly state that the
>> message is off-list at the top of the message.
>> 
>> (Especially if you intend the message to be confidential.)
> 
> I replied to James privately because I felt that the question of "Python's 
> decline," and James' reasons for believing it to be so, were not on-topic for 
> a list about suggestions for improving Python.  It also seemed to me that it 
> could easily devolve into an unproductive discussion for such a large group.  
> But, it was probably also of interest to the group, many of whom were 
> probably wondering the same things I was, so I can see how it could have 
> stayed in the list (where it now is.)
> 
> It also seemed to me to be a topic which could easily result in James feeling 
> at a disadvantage, being on the other side of a subjective debate from the 
> bulk of the group. I had hoped to discuss it with him in a setting that was 
> less likely to get heated.
> 
> I didn't mean to be rude to anyone.
> 
> --Ned.
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

2019-02-07 Thread James Lu

Here are some alternate syntaxes.

These are all equivalent to len(print(list)).

(len | print)(list)
(len |> print)(list)
(print <| len)(list)
print <| len << list
list >> print <| len
list >> len |> print


## Traditional argument order 
print <| len << list

## Stored functions 
print_lengths = len | print
print_lengths = len |> print
print_lengths = print <| len

These can be called using callable syntax.
These can be called using << syntax.
These can be called using >> syntax.
## Lightweight traditional syntax order
(print | len)()

# Explanation
The pipeline operator (|, |>, <|) create an object.

That object implements, depending on the chosen implementation, some 
combination of the __call__ operator, the __rshift__ operator, and/or the 
__lshift__ operator.
—
I am not proposing Python has all these operators at the same time, just 
putting these ideas out there for discussion. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Keeping discussion relevant

2019-02-08 Thread James Lu

Sometimes I see threads briefly go into topics that are unrelated to new 
features in Python. For example: talking about a writer’s use of 
“inhomogeneous” vs “heterogenous” vs “anhomogenous.” We get what the original 
author meant, there is no need to fiddle with the little details of language at 
this point, even if it is fun.

These extra emails, though harmless, impose a visual, time, and navigational 
burden on the readers of the thread. It’s okay if the problem is little but not 
if it’s big.

Two questions:
How often does off-topic discussion occur? Does it need to be find ways to 
reduce the amount?
How can we find ways to reduce the amount?
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Vectorization [was Re: Add list.join() please]

2019-02-08 Thread James Lu

Has anyone thought about my proposal yet? I think because it allows chained 
function calls to be stored, which is probably something that is a common; if 
imagine people turning the same series of chained functions into a lambda of 
its own once it’s used more than once in a program.

Arguably, the lambda syntax is more readable and puts on less visual burden.

Sent from my iPhone

> On Feb 8, 2019, at 3:35 PM, David Mertz  wrote:
> 
>> On Fri, Feb 8, 2019 at 3:17 PM Christopher Barker  
>> wrote:
> 
>> >vec_seq = Vector(seq)
>> >(vec_seq * 2).name.upper()
>> ># ... bunch more stuff
>> >seq = vec_seq.unwrap()
>> 
>> what type would .unwrap() return?
> 
> The idea—and the current toy implementation/alpha—has .unwrap return whatever 
> type went into the Vector creation.  Might be a tuple, list, set, deque, or 
> it might be an iterator.  It might even be some custom collection that isn't 
> in the standard library.
> 
> But you can also explicitly make a Vector into something else by using that 
> constructor.  Pretty much as I gave example before:
> 
> set(Vector(a_list)) # Get a set
> Vector(a_list)).unwrap()# Get a list (without needing to know type to 
> call .unwrap())
>  
> -- 
> Keeping medicines from the bloodstreams of the sick; food 
> from the bellies of the hungry; books from the hands of the 
> uneducated; technology from the underdeveloped; and putting 
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 8 update on line length

2019-02-22 Thread James Lu

A general rule of thumb is, if Python feels inconvenient or awkward, you’re 
doing something wrong. 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dict joining using + and +=

2019-02-28 Thread James Lu

I agree with Storchaka here. The advantage of existing dict merge syntax is 
that it will cause an error if the object is not a dict or dict-like object, 
thus preventing people from doing bad things.


> On Feb 28, 2019, at 2:16 AM, Serhiy Storchaka  wrote:
> 
> 27.02.19 20:48, Guido van Rossum пише:
>> On Wed, Feb 27, 2019 at 10:42 AM Michael Selik > > wrote > The dict subclass collections.Counter 
>> overrides the update method
>>for adding values instead of overwriting values.
>>
>> https://docs.python.org/3/library/collections.html#collections.Counter.update
>>Counter also uses +/__add__ for a similar behavior.
>> >>> c = Counter(a=3, b=1)
>> >>> d = Counter(a=1, b=2)
>> >>> c + d # add two counters together:  c[x] + d[x]
>> Counter({'a': 4, 'b': 3})
>>At first I worried that changing base dict would cause confusion for
>>the subclass, but Counter seems to share the idea that update and +
>>are synonyms.
>> Great, this sounds like a good argument for + over |. The other argument is 
>> that | for sets *is* symmetrical, while + is used for other collections 
>> where it's not symmetrical. So it sounds like + is a winner here.
> 
> Counter uses + for a *different* behavior!
> 
> >>> Counter(a=2) + Counter(a=3)
> Counter({'a': 5})
> 
> I do not understand why we discuss a new syntax for dict merging if we 
> already have a syntax for dict merging: {**d1, **d2} (which works with *all* 
> mappings). Is not this contradicts the Zen?
> 
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

2019-03-03 Thread James Lu

I propose that the + sign merge two python dictionaries such that if there are 
conflicting keys, a KeyError is thrown. 

This way, d1 + d2 isn’t just another obvious way to do {**d1, **d2}. The second 
syntax makes it clear that a new dictionary is being constructed and that d2 
overrides keys from d1.

One can reasonably expect or imagine a situation where a section of code that 
expects to merge two dictionaries with non-conflicting keys commits a semantic 
error if it merges two dictionaries with conflicting keys.

To better explain, imagine a program where options is a global variable storing 
parsed values from the command line.

def verbose_options():
 if options.quiet
 return {'verbose': True}

def quiet_options():
 if options.quiet:
 return {'verbose': False}

If we were to define an options() function, return {**quiet_options(), 
**verbose_options()} implies that verbose overrules quiet; whereas return 
quiet_options() + verbose_options() implies that verbose and quiet cannot be 
used simultaneously. I am not aware of another easy way in Python to merge 
dictionaries while checking for non-conflicting keys.

Compare:

def settings():
 return {**quiet_options(), **verbose_options()}

def settings():
 try:
 return quiet_options() + verbose_options() 
 except KeyError:
 print('conflicting options used', sys.stderr')
 sys.exit(1)

***

This is a simple scenario, but you can imagine more complex ones as well. Does 
—quiet-stage-1 loosen —verbose? Does —quiet-stage-1 conflict with 
—verbose-stage-1?Does —verbosity=5 override —verbosity=4 or cause an error?

Having {**, **} and + do different things provides a convenient and Pythonic 
way to model such relationships in code. Indeed, you can even combine the two 
syntaxes in the same expression to show a mix of overriding and exclusionary 
behavior.

Anyways, I think it’s a good idea to have this semantic difference in behavior 
so Python developers have a good way to communicate what is expected of the two 
dictionaries being merged inside the language. This is like an assertion 
without 

Again, I propose that the + sign merge two python dictionaries such that if 
there are conflicting keys, a KeyError is thrown, because such “non-conflicting 
merge” behavior would be useful in Python. It gives clarifying power to the + 
sign. The + and the {**, **} should serve different roles. 

In other words, explicit + is better than implicit {**, **#, unless explicitly 
suppressed.  Here + is explicit whereas {**, **} is implicitly allowing 
inclusive keys, and the KeyError is expressed suppressed by virtue of not using 
the {**, **} syntax. People expect the + operator to be commutative, while the 
{**, **} syntax prompts further examination by virtue of its “weird” syntax.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction




> On Mar 4, 2019, at 3:41 AM, Stefan Behnel  wrote:
> 
> James Lu schrieb am 04.03.19 um 03:28:
>> I propose that the + sign merge two python dictionaries such that if there 
>> are conflicting keys, a KeyError is thrown.
> 
> Please, no. That would be really annoying.
> 
> If you need that feature, it can become a new method on dicts.
> 
> Stefan
If you want to merge it without a KeyError, learn and use the more explicit 
{**d1, **d2} syntax.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

>> On Mar 4, 2019, at 4:51 AM, Stefan Behnel  wrote:
> 
> Jimmy Girardet schrieb am 04.03.19 um 10:12:
>> I'm not old on this list but every time there is a proposal, the answer
>> is "what are you trying to solve ?".
>> 
>> Since
>> 
>> |z ={**x,**y} and z.update(y) Exists, I can"t find the answer.
> 
> I think the main intentions is to close a gap in the language.
> 
>[1,2,3] + [4,5,6]
> 
> works for lists and tuples,
> 
>{1,2,3} | {4,5,6}
> 
> works for sets, but joining two dicts isn't simply
> 
>{1:2, 3:4} + {5:6}
> 
> but requires either some obscure syntax or a statement instead of a simple
> expression.
> 
> The proposal is to enable the obvious syntax for something that should be
> obvious.

Rebutting my “throw KeyError on conflicting keys for +” proposal:
Indeed but + is never destructive in those contexts: duplicate list items are 
okay because they’re ordered, duplicated set items are okay because they mean 
the same thing (when two sets contain the same item and you merge the two the 
“containing” means the same thing), but duplicate dict keys mean different 
things. 

How many situations would you need to make a copy of a dictionary and then 
update that copy and override old keys from a new dictionary?

It’s better to have two different syntaxes for different situations. 

The KeyError of my proposal is a feature, a sign that something is wrong, a 
sign an invariant is being violated. Yes, {**, **} syntax looks abnormal and 
ugly. That’s part of the point– how many times have you needed to create a copy 
of a dictionary and update that dictionary with overriding keys from a new 
dictionary? It’s much more common to have non-conflicting keys. The ugliness of 
the syntax makes one pause and think and ask: “Why is it important that the 
keys from this dictionary override the ones from another dictionary?”

PROPOSAL EDIT: I think KeyError should only be thrown if the same keys from two 
dictionaries have values that are not __eq__.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction



> On Mar 4, 2019, at 10:02 AM, Stefan Behnel  wrote:
> 
> INADA Naoki schrieb am 04.03.19 um 11:15:
>> Why statement is not enough?
> 
> I'm not sure I understand why you're asking this, but a statement is "not
> enough" because it's a statement and not an expression. It does not replace
> the convenience of an expression.
> 
> Stefan
There is already an expression for key-overriding merge. Why do we need a new 
one? 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

> On Mon, Mar 04, 2019 at 10:01:23AM -0500, James Lu wrote:
> 
> If you want to merge it without a KeyError, learn and use the more explicit 
> {**d1, **d2} syntax.

On Mar 4, 2019, at 10:25 AM, Steven D'Aprano  wrote:

> In your previous email, you said the {**d ...} syntax was implicit:
> 
>In other words, explicit + is better than implicit {**, **#, unless 
>explicitly suppressed.  Here + is explicit whereas {**, **} is 
>implicitly allowing inclusive keys, and the KeyError is expressed 
>suppressed by virtue of not using the {**, **} syntax.
> 
> It is difficult to take your "explicit/implicit" argument seriously when 
> you cannot even decided which is which.
I misspoke. 
> In your previous email, you said the {**d ...} syntax was implicit:
> 
>In other words, explicit + is better than implicit {**, **#, unless 
>explicitly suppressed.  Here + is explicit whereas {**, **} is 
>implicitly allowing inclusive keys, and the KeyError is expressed 
>suppressed by virtue of not using the {**, **} syntax.
> 
> It is difficult to take your "explicit/implicit" argument seriously when 
> you cannot even decided which is which.

Yes, + is explicit. {**, **} is implicit. 

My argument:

We should set the standard that + is for non-conflicting merge and {**, **} is 
for overriding merge. That standard should be so that + explicitly asserts that 
the keys will not conflict whereas {**d1, **d2} is ambiguous on why d2 is 
overriding d1.^

^Presumably you’re making a copy of d1 so why should d3 have d2 take priority? 
The syntax deserves a comment, perhaps explaining that items from d2 are newer 
in time or that the items in d1 are always nonces. 

The + acts as an implicit assertion and an opportunity to catch an invariant 
violation or data input error.

Give me an example of a situation where you need a third dictionary from two 
existing dictionaries and having conflict where a key has a different value in 
both is desirable behavior. 

The situation where non-conflicting merge is what’s desired is more common and 
in that case throwing an exception in the case of a conflicting value is a good 
thing, a way to catch code smell.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction

On Mar 4, 2019, at 11:25 AM, Steven D'Aprano  wrote:

>> How many situations would you need to make a copy of a dictionary and 
>> then update that copy and override old keys from a new dictionary?
>> 
> 
> Very frequently.
> 
> That's why we have a dict.update method, which if I remember correctly, 
> was introduced in Python 1.5 because people were frequently re-inventing 
> the same wheel:
> 
>def update(d1, d2):
>for key in d2.keys():
>d1[key] in d2[key]
> 
> 
> You should have a look at how many times it is used in the standard 
> library:
> 
> [steve@ando cpython]$ cd Lib/
> [steve@ando Lib]$ grep -U "\.update[(]" *.py */*.py | wc -l
> 373
> 
> Now some of those are false positives (docstrings, comments, non-dicts, 
> etc) but that still leaves a lot of examples of wanting to override old 
> keys. This is a very common need. Wanting an exception if the key 
> already exists is, as far as I can tell, very rare.
It is very rare when you want to modify an existing dictionary. It’s not rare 
at all when you’re creating a new one.
> 
> It is true that many of the examples in the std lib involve updating an 
> existing dict, not creating a new one. But that's only to be expected: 
> since Python didn't provide an obvious functional version of update, 
> only an in-place version, naturally people get used to writing 
> in-place code.
My question was “How many situations would you need to make a copy of a 
dictionary and then update that copy and override old keys from a new 
dictionary?” Try to really think about my question, instead of giving answering 
with half of it to dismiss my point.___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction


> On Mar 4, 2019, at 11:25 AM, Steven D'Aprano  wrote:
> 
> The PEP gives a good example of when this "invariant" would be 
> unnecessarily restrictive:
> 
>For example, updating default configuration values with 
>user-supplied values would most often fail under the 
>requirement that keys are unique::
> 
>prefs = site_defaults + user_defaults + document_prefs
> 
> 
> Another example would be when reading command line options, where the 
> most common convention is for "last option seen" to win:
> 
> [steve@ando Lib]$ grep --color=always --color=never "zero" f*.py
> fileinput.py: numbers are zero; nextfile() has no effect.
> fractions.py: # the same way for any finite a, so treat a as zero.
> functools.py: # prevent their ref counts from going to zero during
> 
Indeed, in this case you would want to use {**, **} syntax. 


> and the output is printed without colour.
> 
> (I've slightly edited the above output so it will fit in the email 
> without wrapping.)
> 
> The very name "update" should tell us that the most useful behaviour is 
> the one the devs decided on back in 1.5: have the last seen value win. 
> How can you update values if the operation raises an error if the key 
> already exists? If this behaviour is ever useful, I would expect that it 
> will be very rare.

> An update or merge is effectively just running through a loop setting 
> the value of a key. See the pre-Python 1.5 function above. Having update 
> raise an exception if the key already exists would be about as useful as 
> having ``d[key] = value`` raise an exception if the key already exists.
> 
> Unless someone can demonstrate that the design of dict.update() was a 
> mistake
You’re making a logical mistake here. + isn’t supposed to have .update’s 
behavior and it never was supposed to.

> , and the "require unique keys" behaviour is more common,
I just have. 99% of the time you want to have keys from one dict override 
another, you’d be better off doing it in-place and so would be using .update() 
anyways.

> then 
> I maintain that for the very rare cases you want an exception, you can 
> subclass dict and overload the __add__ method:
Well, yes, the whole point is to define the best default behavior.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP: Dict addition and subtraction