Re: [Python-ideas] Retire or reword the "Beautiful is better than ugly" Zen clause

[Stephan Houben]

Op do 13 sep. 2018 12:03 schreef Antoine Pitrou :

> On Thu, 13 Sep 2018 11:55:40 +0200
> "Giampaolo Rodola'" 
> wrote:
> >
> > This is simply ridiculous. I'm not sure if this is political
> > correctness pushed to its limits or just trolling.
>
> Indeed she might be trolling.  Though the fact we're hesitating on the
> diagnosis shows how far reality has come on the matter...
>


Poe's Law?

https://en.m.wikipedia.org/wiki/Poe%27s_law

Stephan

>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Python dialect that compiles into python

[Stephan Houben]

Op di 11 sep. 2018 om 10:42 schreef Brice Parent :

> Le 11/09/2018 à 02:15, Abe Dillon a écrit :
>
> [Steven D'Aprano]
>
>> It would be great for non-C coders to be able to prototype proposed
>> syntax changes to get a feel for what works and what doesn't.
>
>
> I think it would be great in general for the community to be able to try
> out ideas and mull things over.
>
> If there was something like a Python Feature Index (PyFI) and you could
> install mods to the language,
> it would allow people to try out ideas before rejecting them or
> incorporating them into the language
> (or putting them on hold until someone suggests a better implementation).
>
> That would be an almost-Python to Python transpiler, I love it! It would
> surely help a lot to explain and try out new ideas, as well as for domain
> specific needs. And having an index of those features could help a lot.
> It would surely also help a lot with backward compatibility of new
> functionalities. Someone who would want to use a Python 3.9 functionality
> in 3.8 (whatever his reasons) could use a shim from the index. Such shim
> wouldn't have to be as optimal or fast as the version that would be used in
> 3.9, but it could be functionally equivalent to it.
>
>
For what it's worth, the Ocaml community has something like that: Campl5
https://camlp5.github.io/doc/html/

Despite the name "preprocessor" this actually communicates with the Ocaml
compiler proper through an AST.
So you get proper source code location, etc.

I think you could actually already hack something like this in Python today,
by creating custom import hooks, which then run your own compile step on a
file, which produces an AST, which
is then passed to compile().

However keeping your custom Python++ parser in sync with Python is probably
a pain.


> I'm just a bit afraid of the popularity that some of these experiments
> could get (like `from __future__ import braces` -> No problem!) and the
> pressure that could be made upon core devs to push the most popular changes
> into Python (popularity not being equivalent to sanity for the language and
> its future. There are many devs that want to code in one language the same
> way they do in others, which is often wrong, or at least not optimal).
>

I fully expect the core devs to resist such pressure, especially for the
braces ;-)

Stephan


> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Keyword only argument on function call

[Stephan Houben]

My 3 cents:

1. My most objective objection against the f(*, foo, bar, baz) syntax is
that it looks like positional arguments, and the syntactic marker *
which
   dissuades you of that can be arbitrarily far apart from the keyword.

2. The syntax f(=foo, =bar, =baz) at least solves that problem.
 Otherwise I find it quite ugly with the unbalanced = but that is
obviously more subjective.

3. I still am not convinced it is needed at all.
   IMHO, if your code is filled with
   f(foo=foo, bar=bar, baz=baz)
then perhaps Python is telling you that foo, bar and baz want to become
fields in a new object which you should pass around.

4. (Bonus cent) Somewhat tongue-in-cheek I offer the following Vim mapping
for those who
find themselves typing longword=longword all the time.

:inoremap  =hyiwt=lpa

Now you can just do longword.

Stephan


Op di 11 sep. 2018 om 08:55 schreef Jacco van Dorp :

>
>
> Op di 11 sep. 2018 om 06:48 schreef Steve Barnes :
>
>>
>>
>> On 10/09/2018 22:00, Ethan Furman wrote:
>> > On 09/10/2018 12:52 PM, Chris Barker via Python-ideas wrote:
>> >
>> >> I've spent this whole thread thinking: "who in the world is writing
>> >> code with a lot of spam=spam arguments? If you are transferring that
>> >> much state in a function call, maybe you should have a class that
>> >> holds that state? Or pass in a **kwargs dict?
>> >
>> >> So still looking for a compelling use-case
>> >
>> > In my day job I spend a lot of time writing/customizing modules for a
>> > framework called OpenERP (now Odoo*).  Those modules are all
>> subclasses,
>> > and most work will require updating at least a couple parent metheds --
>> > so most calls look something like:
>> >
>> >def a_method(self, cr, uid, ids, values, context=None):
>> >  ...
>> >  super(self, parent).a_method(cr, uid, ids, values, context=context)
>> >
>> > Not a perfect example as these can all be positional, but it's the type
>> > of code where this syntax would shine.
>> >
>> > I think, however, that we shouldn't worry about a lead * to activate
>> it,
>> > just use a leading '=' and let it show up anywhere and it follows the
>> > same semantics/restrictions as current positional vs keyword args:
>> >
>> >def example(filename, mode, spin, color, charge, orientation):
>> >pass
>> >
>> >example('a name', 'ro', =spin, =color, charge=last, =orientation)
>> >
>> > So +0 with the above proposal.
>> >
>> > --
>> > ~Ethan~
>> > ___
>> > Python-ideas mailing list
>> > Python-ideas@python.org
>> > https://mail.python.org/mailman/listinfo/python-ideas
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>>
>> Couldn't just about all of the use cases mentioned so far be met in
>> quite a neat manner by providing access to a method, or dictionary,
>> called __params__ which would give access, as a dictionary, to the
>> parameters as supplied in the call, (or filled in by the defaults).
>>
>> If this was accessible externally, as fn.__defaults__ is then examples
>> such as:
>>
>>  >def a_method(self, cr, uid, ids, values, context=None):
>>  >  ...
>>  >  super(self, parent).a_method(cr, uid, ids, values,
>> context=context)
>>
>> would become:
>>
>>
>>  def a_method(self, cr, uid, ids, values, context=None):
>>...
>>params = {k:v for k,v in __params__ if k in parent.a_method.keys()}
>># Possibly add some additional entries here!
>>super(self, parent).a_method(**params)
>
>
> So...deep black magic ? That's what this looks like. Having =spam for
> same-named kwargs sounds easier to comprehend for new people than a
> __magic__ object you can only access in function bodies and will give
> headaches if you have to write decorators:
>
> def other_function_defaults(*args, **kwargs):
> outer_params = __params__.copy()
> def deco(func):
> def inner(self, yo_momma):
> return func(self, **outer_params, **__params__)  # overwrite with
> specifically provided arguments
> return deco
>
>
> I think that magic objects like that aren't really pythonic - if it were,
> "self" would be the same kind of magic, instead of us having to name it on
> every function call (A decision im really a fan of, tbh)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Fwd: Add Unicode-aware str.reverse() function?

[Stephan Houben]

Op za 8 sep. 2018 13:33 schreef Paddy3118 :

>
> I would like to propose that Python add a Unicode-aware *str.reverse *method.
> The problem is, I'm a Brit, who only speaks English and only very rarely
> dips into Unicode.* I don't know how useful this would be!*
>


To be honest, quite apart from the Unicode issue, I never had a need to
reverse a string in real code.

.ytilibigel edepmi ot sdnet yllareneg tI

Stephan


> Cheers, Paddy.
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Executable space protection: NX bit,

[Stephan Houben]

I am pretty sure that on systems which support it, Python's stack and data
are already NX.

NX is basically the default on modern systems.

Stephan

Op ma 3 sep. 2018 09:00 schreef Wes Turner :

> Rationale
> =
> - Separation of executable code and non-executable data is a good thing.
> - Additional security in Python is a good idea.
> - Python should support things like the NX bit to separate code and
> non-executable data.
>
> Discussion
> ==
> How could Python implement support for the NX bit? (And/or additional
> modern security measures; as appropriate).
>
> What sort of an API would C extensions need?
>
> Would this be easier in PyPy or in CPython?
>
> - https://en.wikipedia.org/wiki/NX_bit
> - https://en.wikipedia.org/wiki/Executable_space_protection
>
> Here's one way to identify whether an executable supports NX:
> https://github.com/longld/peda/blob/e0eb0af4bcf3ee/peda.py#L2543
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fix some special cases in Fractions?

[Stephan Houben]

I would also like to point out that the current behavior of Fraction
is consistent with other parts of the numeric system, e.g.
1/1 produces 1.0 (rather than 1)
math.sqrt(4) produces 2.0 (rather than 2)
1j-1j produces 0j (rather than 0.0 or 0)

So in general the type of the output is determined by what
the operation would in general return for that input type,
as opposed to a more specific type which only applies
to the specific input value.

Stephan

Op do 30 aug. 2018 om 15:03 schreef Steven D'Aprano :

> On Wed, Aug 29, 2018 at 09:39:05PM -0700, Neil Girdhar wrote:
>
> > Would there be any problem with changing:
> >
> > In [4]: Fraction(1, 1) ** Fraction(2, 3)
> > Out[4]: 1.0
> >
> > In [5]: Fraction(-1, 1) ** Fraction(2, 3)
> > Out[5]: (-0.4998+0.8660254037844387j)
> >
> > In [6]: Fraction(0, 1) ** Fraction(2, 3)
> > Out[6]: 0.0
> >
> > I'd like these to be Fraction(1), Fraction(1), and Fraction(0).
>
> If anyone else has mentioned the backward-compatibility issue by now, I
> haven't see it. I believe that would make it a fairly big problem.
>
> I expect that there is code out in the wild which (for good or ill) now
> expects 1**Fraction(2, 3) to return 1.0, rather than Fraction(1), and
> similarly for the other examples. Changing that could break people's
> code.
>
> Even if we had consensus that this was a good idea, or at least
> consensus from the maths-folk who care about this sort of thing (I'd
> like to know what Uncle Timmy and Mark think of this idea), it would
> still probably need a "__future__" import to activate it, or a
> deprecation period. Or some other annoyance.
>
> Possibly the simpler approach would be to add a subclass that does what
> you want. UnitRootFraction or something.
>
> Then the only argument will be whether such a subclass ought to go into
> the fractions module, or your own personal toolkit :-)
>
> I must admit though, I'm a bit curious as to what you are doing that
> having 1**Fraction(2,3) return 1.0 is an annoyance, but having
> 27**Fraction(2,3) return 8.998 instead of Fraction(9) isn't.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Pre-conditions and post-conditions

[Stephan Houben]

I
Op wo 29 aug. 2018 07:53 schreef Greg Ewing :

> Wes Turner wrote:
> > I'm going to re-write that in a pseudo-Eiffel like syntax:
>
> Maybe some magic could be done to make this work:
>
>   def __init__(self, img: np.ndarray, x: int, y: int, width: int,
>   height: int) -> None:
>
>   def __require__():
>   x >= 0
>   y >= 0
>   width >= 0
>   height >= 0
>   x + width <= pqry.opencv.width_of(img)
>   y + height <= pqry.opencv.height_of(img)
>
>   def __ensure__():
>   (self.x, self.y) in self
>   (self.x + self.width - 1, self.y + self.height - 1) in self
>   (self.x + self.width, self.y + self.height) not in self
>
>   # body of __init__ goes here...
>


I have often wished we could get at the AST of a function object.

Then we could inspect the AST and extract these magic functions.

Stephan

-- 
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A GUI for beginners and experts alike

[Stephan Houben]

Op za 25 aug. 2018 02:28 schreef Chris Barker - NOAA Federal via
Python-ideas :

>
>
> Too bad all the cool kids are doing web dev these days — hard to get
> help with a desktop GUI project :-(
>

Pywebview seems interesting, uses a platform  webview to put up the UI;

https://github.com/r0x0r/pywebview

But not so newbie-friendly I am afraid...

Stephan


> > Pyjamas seems to be something like that:
> >
> > https://pypi.org/project/Pyjamas/
>
> Or it was 6 years ago :-(
>
> -CHB
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A simple proposal concerning lambda

[Stephan Houben]

Op do 23 aug. 2018 09:06 schreef Jacco van Dorp :

> I think it would have been better to use def from the start instead of
> lambda. The only thing JS does right is using the same "function" keyword
> for both of these.
>


Seriously?

Consider how the following code

function f() { return 42; }

may or may not bind the function to f, depending on if it appears in
expression or statement context. Just Google and find all the newbies who
get confused about this.

And the proposed def syntax is almost as terrible:

def greet: print("hello'')

This would then be legal, and leave the newbie confused why greet remains
undefined.

Please don't repurpose def.
Repurposing for and if inside comprehensions is already confusing to
newbies.

Stephan

>
> However, changing it now doesn't seem that important to me.
>
> (And I've used lambda's as default argument - I was moving data from one
> database to another, and wanted a generic method to switch out column names
> that defaulted to the same header. So it became a lambda x:x function by
> default, but you can provide any function that takes and returns a string
> and change the column names. This was last week.)
>
> And lets be real - adding a new keyword is something to be done extremely
> sparingly. lambda could have been avoided by re-using def. And no, it being
> the same keyword is an advantage, as they do the exact same - they create a
> function. It's the same reason as we use if condition:
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Does jargon make learning more difficult?

[Stephan Houben]

Fwiw, I usually don't do

def foo():
   if False: yield None

But simply:

def foo():
return ()

That the returned iterable is not a generator seldom matters.

Stephan

Op wo 22 aug. 2018 21:17 schreef Chris Angelico :

> On Thu, Aug 23, 2018 at 3:56 AM, Rhodri James 
> wrote:
> > On 22/08/18 14:38, Jonathan Fine wrote:
> > def fn():
> >>
> >> ... if None:
> >> ... yield
> >> ...
> >
> > list(fn()) # Fails, unless fn is a generator function.
> >>
> >> []
>
> Actually, it fails unless fn returns some sort of iterable.
>
> >> I think what's happening is this. Even though the None-guarded yield
> >> has been optimised away, it leaves a residue. Namely, that there's a
> >> yield in the function. Hence fn() is an iterator.
> >>
> >> Having explained the surprise, here's how it can help you. Suppose you
> >> have a long function body, with a single yield at the bottom. If you
> >> write like this:
> >>
> >>  def my_very_long_function_with_one_yield_point(...):
> >>  if None: yield # This is a generator function.
> >>
> >> then the next programmer can know immediately that it's a generator
> >> function.
> >
> >
> > Ew.
> >
> > I'd prefer this way of doing it:
> >
> > def my_very_long_function_with_one_yield_point(...):
> > # This is a generator
> >
>
> Or you could use a return type annotation.
>
> Why should generator objects be special? A generator function is very
> similar to a function that returns a list, except that it can yield
> values as it gets them, rather than batching them up to the end. A
> generator function is a function which returns an iterable object of
> the type "generator". How is it different from a function that returns
> an integer? They return different things. How do you show what a
> function returns? You annotate it, put it in the docstring, put it in
> external docs... It's a function. It does something, it returns
> something. That's it.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Does jargon make learning more difficult?

[Stephan Houben]

Op wo 22 aug. 2018 18:40 schreef Steven D'Aprano :

> Chris' conclusion is that anonymous functions are inherently hard for
> many beginners to learn, regardless of whether the syntax is called
> "lambda" or "function".
>

Civilization itself had trouble with the concept. Functions as mathematical
objects in their own right were only conceived (late) in the 19th century.
Church's lambda notation was the first way to write down a function without
naming it, in the 1930's. And a proven consistent model for lambda calculus
was only provided by Dana Scott in the 1970's!

So clearly these mathematical developments haven't yet entered high school
mathematics, which remains firmly set in an 18th century mindset, where we
can interchangeably write down f or f(x).

Unfortunately modern computer languages require more, so it is up to the
comp sci teacher to fill the gap.

But let's not blame Church's notation for that.

 λ f.(λ x. f(x x)) (λ x. f(x x))
lambda f: (lambda x: f(x(x))) (lambda x: f(x(x)))

(Yeah I know it doesn't work in eagerly-evaluated Python).

Stephan


> That matches my own observations, interacting with newbies on various
> mailing lists, as well as my own personal experience.
>
> I believe that many people have a lot of trouble grasping the concept of
> functions as first-class values capable of being passed to other
> functions as data. It requires a major rethink of the way we think of
> functions. We go from treating them purely as verbs:
>
> sort the list
>
> to nouns in their own right:
>
> process the sort  # the what now?
>
>
> Not "process the sorted list", but reify the sort verb into an actual
> thing (an object or value) and then process that thing itself.
>
> This is mind-bending when you think about it, far more mind-blowing than
> the normal linguistic process of nouning verbs and verbing nouns. No
> wonder people take a while to get fully comfortable with the concept. It
> took me a long time to stop writing code like this:
>
> map(lambda x: len(x), sequence)
>
> instead of simply map(len, sequence). I doubt it would have taken any
> less time if it were
>
> map(function x: len(x), sequence)
>
> instead. Aside from having to learn the correct spelling ("lamdba"?) it
> was never the *name* that gave me trouble.
>
> Of course there is a cost for beginners to having to learn a name, and
> the less often the name is used, the longer it takes (unless it is
> extremely memorable). But that cost is paid for, with interest, later,
> as a journeyman or journeywoman programmer.
>
>
> > In my own experience teaching, I find that many concepts are easier to
> > introduce if I avoid the Python jargon until after I've explained what it
> > does. This is supported by education research. Some light Googling found
> a
> > study on the topic [0] that is consistent with my own observations.
>
> I agree. When I teach my students function transformations in maths, any
> time I use the technical terms translation, dilation and reflection, I
> follow up with the common terms shift, stretch, and flip, and vice
> versa. I make a point of always associating the imprecise plain English
> words with the precise technical terms.
>
> Nevertheless, it is important that I teach them the technical terms too.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] A simple proposal concerning lambda

[Stephan Houben]

Let me stand up and say that I personally like lambda. It's the standard
terminology and not easily confused.

I doubt that "many Python users" have a problem with it. Evidence?

I dislike the def proposal strongly.
It is too similar to a normal def.
At least get a new keyword then

Some other languages:

lambda: scheme, lisp
\ : Haskell
fn : Standard ml
function : JavaScript
[] : C++

Stephan



Op wo 22 aug. 2018 18:16 schreef Mike Miller :

> I've often thought the same about "lambda."
>
> Although I've long since gotten used to the word, "def" without a function
> name
> seemed like a better choice, especially for beginners.
>
> +0.25 for proposal ONE
>
> However, parentheses should be allowed and lambda put on a long
> deprecation
> schedule.
>
> -Mike
>
>
> On 2018-08-22 08:58, Jonathan Fine wrote:
> >
> > ONE.  Wherever you could have used 'lambda', you now have a choice.
> > You can still use 'lambda', or you can use 'def' instead. The
> > semantics are unchanged.
> >
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Revisiting dedicated overloadable boolean operators

[Stephan Houben]

I use these Vim abbreviations,
which are derived from LaTeX

https://gist.github.com/stephanh42/fc466e62bfb022a890ff2c4643eaf3a5

Stephan

Op za 4 aug. 2018 20:03 schreef David Mertz :

> On Sat, Aug 4, 2018, 1:24 PM Steven D'Aprano  wrote:
>
>> If you think the uproar over PEP 572 was vicious, imagine what would
>> happen if we introduced new operators like ∉ ∥ ∢ ∽ ⊎ etc instead.
>>
>> - keyboard support for entering the bulk of Unicode characters is
>>   non-existent or poor;
>>
>> - without keyboard support, editor support for entering Unicode
>>   characters is as best clunky, requiring the memorization of
>>   obscure names, hex codes, or a GUI palette;
>>
>
> This is the essential problem. I write this as someone who has the vim
> conceal plugin configured to change my Python code into something with many
> of those funny characters Steven users as examples.
>
> But while I like looking at those, although recognizing it's quirky,
> entering any of them is enormously cumbersome. I did it once to configure
> my substitution macros (which are purely visual... What I type in is just
> ASCII and that's what is saved in soak, but it appears on screen with some
> replacements).
>
>> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

Op wo 1 aug. 2018 11:10 schreef Jonathan Fine :

>
>
> You're right to be cautious. My understanding of PEP 505 is that
> #13.  a ?. b ?. __str__
> #14. (a ?. b) ?. __str__
> are not equivalent. The first is None, and the other is None.__str__.
> That looks like a gotcha.
>


No.
None.?__str__
produces None, even though None has a __str__ attribute.

I am pretty sure
a?.b?.c == (a?.b)?.c

and more generically

chain_A ?. chain_B == (chain_A) ?. chain_B

Stephan


> (By the way, it was not my intention to catch you out. I'm simply
> looking for clarity. I wasn't aware of the gotcha until I started
> answering myself the question I had asked you.)
>
> However, the None object is somewhat special, in that all it's methods
> and double-under (dunder) methods. And one of them is __bool__.  And
> we can't add or change the attributes of None.
>
> Chris, you don't have to reply to this. But I would be pleased if an
> expert could either tell me that my neck is safe, or produce a value
> of 'a' that cuts it off (so to speak).
>
> --
> Jonathan
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

Op wo 1 aug. 2018 10:50 schreef Chris Angelico :

> On Wed, Aug 1, 2018 at 6:45 PM, Jonathan Fine  wrote:
> > Hi Chris
> >
> > Thank you for your reply. I think we're making good progress.
> >
> > You wrote
> >
> >>> 10) a ?. b ?. c
> >>> 11) (a ?. b) ?. c
> >>
> >> I would parse those differently, but you may be right that they'll
> >> always have the same final result.
> >
> > I'd like to get some certainty on this. I'm not aware of any value of
> > 'a' for which #10 and #11 give different values. Can you (or anyone
> > else) think of any such value?
> >
> >> Technically they should result in different code, though.
> >
> > Maybe. We need to think. Should can be a difficult word. Elsewhere you
> > have, as I recall, pointed out that
> > if None:
> > do_something()
> > generates no code.
> >
> > Perhaps the compiler should collapse #11 to #10, if they are
> > equivalent. But this is a side issue.
> >
> > So, are there any values of 'a' for which #10 and #11 don't give the
> > same result?
>
> I'm not prepared to put my neck out and say "They are absolutely
> identical" and have people jump on me with some technicality.


Let me stand up and claim that if a chain consists *only* of
None-coalescing operations, then breaking up the chain by adding
parentheses does not matter, ever.

 So a?.b?.c is equivalent to (a?.b)?.c


What is
> your point here?
>

It is useful to establish rules under which a chain can be factored.

Stephan




> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

Op di 31 jul. 2018 20:49 schreef Jonathan Fine :

> David Mertz wrote:
>
> > `spam?.eggs?.cheese?.aardvark` is NOT redundant for
> > `spam?.eggs.cheese.aardvark`.  The two expressions simply do different
> > things [...]
>
> I agree, assuming ?. is a binary operator.


It isn't.


Given this, in Python (+
> PEP 505) one can write
>
> tmp = spam ?. eggs
> val1 = tmp ?. cheese ?. aardvark# For spam?.eggs?.cheese?.aardvark
> val2 = tmp . cheese . aardvark# For spam?.eggs.cheese.aardvark
>

Nope, the introduction of the tmp variable changed the semantics. It isn't
a "chain" anymore so it breaks shortcutting.

To be honest I didn't get this either until it was pointed out to me

>
> No special knowledge of PEP 505 is needed. If val1 is always equal to
> val2, then the dot and None-dot operators must be the same. From the
> assumptions, this is something that can be mathematically proved.
>

And false.


> By the way, there's a widely used programming language in which
> val = a.method()
> and
> tmp = a.method
> val = tmp()
> are not always equivalent. Can you guess which language it is?


Javascript.
I suppose in the same way as x+2 and x*2 are " not always" equivalent.

Stephan



> The answer is in:
>
> https://www.slideshare.net/jonathanfine/javascript-the-easiest-quiz-in-the-world-ever
> (question 6: Dot binds).
>
> I'll now go back to following the example of Steve Bower and Raymond
> Hettinger, which in my words is to wait until we have proper cover for
> the BDFL's vacation.
>
> --
> Jonathan
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Redefining method

[Stephan Houben]

Here is an example of how it could be done.

https://gist.github.com/stephanh42/97b47506e5e416f97f5790c070be7878


Stephan

Op di 31 jul. 2018 01:29 schreef Steven D'Aprano :

> On Tue, Jul 31, 2018 at 10:10:32AM +1200, Greg Ewing wrote:
> > Jamesie Pic wrote:
> > >def o.bar(self): ...
> >
> > You could get almost the same effect with
> >
> >from functools import partial
> >
> >def bar(self, other_args):
> >   ...
> >
> >o.bar = partial(bar, o)
>
> Why are you using functools.partial instead of types.MethodType? I'm
> wondering if there is some advantage to partial that I don't recognise.
>
> I'm not sure if there's a functional difference between the two
> approaches, but it makes o.bar a different kind of callable and that
> will probably make a difference to somebody.
>
>
> > But IMO this is nowhere near being a common enough thing to
> > do to justify having special syntax for it.
>
> This sort of thing isn't common because there's no neat, easy, obvious,
> built-in way to do it. If we allowed people to extend classes using the
> syntax
>
> def classobj.methodname(...): ...
>
> def instance.methodname(...): ...
>
> people would use the technique more. For good or ill.
>
> I don't question the utility of this technique, but I suspect we prefer
> to *slightly* discourage it by *not* providing a Batteries Included
> solution for this. If you want to do this, we won't stop you, but
> neither will we encourage it by supporting it in syntax or providing a
> standard decorator for it.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

Let me just address this point:

2018-07-19 20:36 GMT+02:00 Brendan Barnwell :

> As far as I can see, these null-coalescing operators would break
> that model.  The PEP doesn't seem to provide for a "real" magic method
> allowing users to override the actual behavior of the method.  (You can
> only override __has_value__ to hook into it, but not define by fiat what A
> ?? B does, as you can with other operators.)  And I think the reason for
> this is that the operator itself is too specific, much more specific in
> semantics than other operators.  (I had similar doubts about adding the
> matrix-multiplication operator @.)
>

I think the actual reason is that it is a short-cutting operator, and none
of
the shortcutting operators (and, or,  if/else) have an associated method.
They cannot have, since they induce a non-standard evaluation order,
hence their effect cannot be emulated with a method invocation.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Performance improvements via static typing

[Stephan Houben]

You are aware of numba?

https://numba.pydata.org/

Stephan

Op do 19 jul. 2018 16:03 schreef Eric Fahlgren :

> On Thu, Jul 19, 2018 at 6:52 AM Michael Hall 
> wrote:
>
>> While I am aware of projects like Cython and mypy, it seems to make sense
>> for CPython to allow optional enforcement of type hints, with compiler
>> optimizations related to it to be used. While this would not receive the
>> same level of performance benefits as using ctypes directly, there do
>> appear to be various gains available here.
>>
>
> ​Just to make sure I understand:  In other words, they would no longer be
> "hints" but "guarantees".  This would allow an optimizer pass much greater
> latitude in code generation, somehow or other.​
>
> For purposes of illustration (this is not a proposal, just for
> clarification):
>
> @guaranteed_types
> def my_sqrt(x:c_double) -> c_double:
> ...
>
> would tell the compiler that it's now possible to replace the general
> PyObject marshalling of this function with a pure-C one that only accepts
> doubles and woe be unto those who use it otherwise.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 505: None-aware operators

[Stephan Houben]

FWIW, I like ??

It is short and distinctive.
There is prior art in this spelling in c#.
It requires no new keyword, nor does it give new meaning to an existing one.

I understand why  ?[ needs to be spelled using only a single ?, but I am
afraid it will be used infrequently, and people will accidentally write
 a??[x]
which is legal but different.

I found the example code in the PEP using ?. and ?[ hard to read.
?? and ??= are compelling, though.

One more question: what does this do?

del x
x ??= 42

Stephan


Op do 19 jul. 2018 15:00 schreef Judah Levy :

> On Thu, Jul 19, 2018 at 8:47 AM Rhodri James  wrote:
>
>> On 19/07/18 09:33, Antoine Pitrou wrote:
>> > There is a use case I sympathize with: the argument-is-None case.  For
>> > that I would suggest a simpler form:  "A else B" which would evaluate
>> > to A if A is not None, otherwise to B (parentheses may be mandatory).
>> >
>> > So e.g. one of the examples would read:
>> >
>> >   def insort_right(a, x, lo=0, hi=None):
>> >   # ...
>> >   hi = hi else len(a)
>> >   # ...
>>
>> Much as I would like a keyword, "else" is the wrong one.  It implies we
>> are dealing with truthiness, which we aren't, and lays a subtle semantic
>> trap as a consequence.
>>
>> If anyone can think of a good word for "if it isn't None, otherwise",
>> I'd be all for it :-)
>>
>> --
>> Rhodri James *-* Kynesim Ltd
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> I think that it may look better with the order switched and the word
> unless, as in
>
> def insort_right(a, x, lo=0 hi=None):
> # ...
> hi = len(a) unless hi
> # ...
>
> Unfortunately, this does maybe feel more like checking for truthiness than
> non-Null value
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Stephan Houben]

Hi Nathaniel,

2018-07-19 1:33 GMT+02:00 Nathaniel Smith :

> Note that this everything you said here also exactly describes the
> programming model for the existing 'multiprocessing' module:
> "structured clone" is equivalent to how multiprocessing uses pickle to
> transfer arbitrary objects, or you can use multiprocessing.Array to
> get a shared view on raw "C"-style data.
>


This is true. In fact, I am a big fan of multiprocessing and I think it is
often
overlooked/underrated. Experience with multiprocessing is also
what has me convinced that share-nothing or share-explicit approach
to concurrency is a useful programming model.

The main limitation of multiprocessing comes when you need to go outside
Python, and you need to interact with C/C++ libraries or operating services
from multiple processes.
The support for this generally varies from "extremely weak" to "none at
all".

For example, things I would like to in parallel with a main thread/process:

* Upload data to the GPU using OpenGL or OpenCL
* Generate a picture in pyqt QImage, then hand over zero-copy to main thread
* interact with a complex scenegraph in C++ (shared with main thread)

This is impossible right now but would be possible if the interpreters were
all in-process.

In addition, there are things which are now hard with "multiprocessing" but
could be fixed.
For example, sharing a Numpy array is possible but very inconvenient.
You need to first allocate the raw data segment, communicate that, then
create in each process
an array which uses this data segment.

Ideally, this would rather work like this:

   ar = numpy.zeros((30, 30), shared=True)

and then "ar" would automatically be shared.

This is fixable but given the other limitations above the question is if it
is worthwhile
to fix it now. It would be a lot simpler to fix if we had the in-process
model.

But yeah, I am actually also very open to ideas on how multiprocessing
could be
made more convenient and powerful. Perhaps there are ways, and I am just
not seeing them.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Multi-core reference count garbage collection

[Stephan Houben]

Hi Jonathan,

2018-07-19 8:33 GMT+02:00 Jonathan Fine :

> I call any such scheme BUFFERED multi-core reference count garbage
> collection. The worker processes know nothing about how garbage collection
> is managed. Instead, they pass over to the GC process sufficient
> information to allow it to manage the garbage.
>

This is actually a known idea (which is GOOD, unless you wanted to apply
for a patent ;-) ).

It is described, among other places, in
"The Garbage Collection Handbook: The Art of Automatic Memory Management",
by Richard Jones, Antony Hosking, Eliot Moss.

In fact, they also call it buffered reference counting.
Section 18.2: Buffered Reference Counting:
"...DeTreville[1990] had log mutators the old and new referents of each
pointer update to a buffer"

By the way, this book describes a ton of other ideas to speed up
reference counting in general and in the concurrent case, so
it may be worthwhile to get it.

I should warn you that many of these advanced refcounting ideas have been
proposed in the past already, although I am unaware of this particular
technique
having been implemented.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Stephan Houben]

Hi Eric, Antoine, all

Antoine said that what I proposed earlier was very similar to what Eric
is trying to do, but from the direction the discussion has taken so far
that appears not to be the case.

I will therefore try to clarify my proposal.

Basically, what I am suggesting is a direct translation of Javascript's
Web Worker API (
https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API)
to Python.

The Web Worker API is generally considered a "share-nothing" approach,
although
as we will see some state can be shared.

The basic principle is that any object lives in a single Worker (Worker =
subinterpreter).
If a message is send from Worker A to Worker B, the message is not shared,
rather the so-called "structured clone" algorithm is used to create
recursively a NEW message
object in Worker B. This is roughly equivalent to pickling in A and then
unpickling in B,

Of course, this may become a bottleneck if large amounts of data need to be
communicated.
Therefore, there is a special object type designed to provide a view upon a
piece
of shared memory:  SharedArrayBuffer. Notable, this only provides a view
upon
raw "C"-style data (ints or floats or whatever), not on Javascript objects.

To translate this to the Python situation: each Python object is owned by a
single
subinterpreter, and may only be manipulated by a thread which holds the GIL
of that particular subinterpreter. Message sending between subinterpreters
will
require the message objects to be "structured cloned".

Certain C extension types may override what structured cloning means for
them.
In particular, some C extension types may have a two-layer structure where
the Py_Object contains a refcounted pointer to the actual data.
The structured cloning on such an object may create a second Py_Object which
references the same underlying object.
This secondary refcount will need to be properly atomic, since it may be
manipulated
from multiple subinterpreters.

In this way, interpreter-shared data structures can be implemented.
However, all the "normal" Python objects are not shared and can continue
to use the current, non-atomic refcounting implementation.

Hope this clarifies my proposal.

Stephan


2018-07-18 19:58 GMT+02:00 Eric Snow :

> On Wed, Jul 18, 2018 at 1:37 AM Barry Scott 
> wrote:
> > Let me try a longer answer. The inc+test and dec+test do not require a
> > lock if coded correctly. All OS and run times have solved this to provide
> > locks. All processors provide the instructions that are the building
> blocks
> > for lock primitives.
> >
> > You cannot mutate a mutable python object that is not protected with the
> GIL as
> > the change of state involves multiple parts of the object changing.
> >
> > If you know that an object is immutable then you could only do a check
> on the
> > ref count as you will never change the state of the object beyond its
> ref count.
> > To access the object you only have to ensure it will not be deleted,
> which the
> > ref count guarantees. The delete of the immutable object is then the
> only job
> > that the original interpreter must do.
>
> Perhaps we're agreeing?  Other than the single decref at when
> "releasing" the object, it won't ever be directly modified (even the
> refcount) in the other interpreter.  In effect that interpreter holds
> a reference to the object which prevents GC in the "owning"
> interpreter (the corresponding incref happened in that original
> interpreter before the object was "shared").  The only issue is how to
> "release" the object in the other interpreter so that the decref
> happens in the "owning" interpreter.  As earlier noted, I'm planning
> on taking advantage of the exiting ceval "pending calls" machinery.
>
> So I'm not sure where an atomic int would factor in.  If you mean
> switching the exiting refcount to an atomic int for the sake of the
> cross-interpreter decref then that's not going to happen, as Ronald
> suggested.  Larry could tell you about his Gilectomy experience. :)
>
> Are you suggesting something like a second "cross-interpreter
> refcount", which would be atomic, and add a check in Py_DECREF?  That
> would imply an extra cross-interpreter-oriented C-API to parallel
> Py_DECREF.  It would also mean either adding another field to PyObject
> (yikes!) or keeping a separate table for tracking cross-interpreter
> references.  I'm not sure any of that would be better than the
> alternative I'm pursuing.  Then again, I've considered tracking which
> interpreters hold a "reference" to an object, which isn't that
> different.
>
> -eric
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofco

Re: [Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Stephan Houben]

What about the following model: you have N Python interpreters, each with
their own GIL. Each *Python* object belongs to precisely one interpreter.

However, the interpreters share some common data storage: perhaps a shared
Numpy array, or a shared Sqlite in-memory db. Or some key-value store where
the key and values are binary data. The interpreters communicate through
that.

Stephan

Op ma 16 jul. 2018 06:25 schreef Chris Angelico :

> On Mon, Jul 16, 2018 at 1:21 PM, Nathaniel Smith  wrote:
> > On Sun, Jul 15, 2018 at 6:00 PM, Chris Angelico 
> wrote:
> >> On Mon, Jul 16, 2018 at 10:31 AM, Nathaniel Smith 
> wrote:
> >>> On Sun, Jul 8, 2018 at 11:27 AM, David Foster 
> wrote:
>  * The Actor model can be used with some effort via the
> “multiprocessing”
>  module, but it doesn’t seem that streamlined and forces there to be a
>  separate OS process per line of execution, which is relatively
> expensive.
> >>>
> >>> What do you mean by "the Actor model"? Just shared-nothing
> >>> concurrency? (My understanding is that in academia it means
> >>> shared-nothing + every thread/process/whatever gets an associated
> >>> queue + queues are globally addressable + queues have unbounded
> >>> buffering + every thread/process/whatever is implemented as a loop
> >>> that reads messages from its queue and responds to them, with no
> >>> internal concurrency. I don't know why this particular bundle of
> >>> features is considered special. Lots of people seem to use it in
> >>> looser sense though.)
> >>
> >> Shared-nothing concurrency is, of course, the very easiest way to
> >> parallelize. But let's suppose you're trying to create an online
> >> multiplayer game. Since it's a popular genre at the moment, I'll go
> >> for a battle royale game (think PUBG, H1Z1, Fortnite, etc). A hundred
> >> people enter; one leaves. The game has to let those hundred people
> >> interact, which means that all hundred people have to be connected to
> >> the same server. And you have to process everyone's movements,
> >> gunshots, projectiles, etc, etc, etc, fast enough to be able to run a
> >> server "tick" enough times per second - I would say 32 ticks per
> >> second is an absolute minimum, 64 is definitely better. So what
> >> happens when the processing required takes more than one CPU core for
> >> 1/32 seconds? A shared-nothing model is either fundamentally
> >> impossible, or a meaningless abstraction (if you interpret it to mean
> >> "explicit queues/pipes for everything"). What would the "Actor" model
> >> do here?
> >
> > "Shared-nothing" is a bit of jargon that means there's no *implicit*
> > sharing; your threads can still communicate, the communication just
> > has to be explicit. I don't know exactly what algorithms your
> > hypothetical game needs, but they might be totally fine in a
> > shared-nothing approach. It's not just for embarrassingly parallel
> > problems.
>
> Right, so basically it's the exact model that Python *already* has for
> multiprocessing - once you go to separate processes, nothing is
> implicitly shared, and everything has to be done with queues.
>
> >> Ideally, I would like to be able to write my code as a set of
> >> functions, then easily spin them off as separate threads, and have
> >> them able to magically run across separate CPUs. Unicorns not being a
> >> thing, I'm okay with warping my code a bit around the need for
> >> parallelism, but I'm not sure how best to do that. Assume here that we
> >> can't cheat by getting most of the processing work done with the GIL
> >> released (eg in Numpy), and it actually does require Python-level
> >> parallelism of CPU-heavy work.
> >
> > If you need shared-memory threads, on multiple cores, for CPU-bound
> > logic, where the logic is implemented in Python, then yeah, you
> > basically need a free-threaded implementation of Python. Jython is
> > such an implementation. PyPy could be if anyone were interested in
> > funding it [1], but apparently no-one is. Probably removing the GIL
> > from CPython is impossible. (I'd be happy to be proven wrong.) Sorry I
> > don't have anything better to report.
>
> (This was a purely hypothetical example.)
>
> There could be some interesting results from using the GIL only for
> truly global objects, and then having other objects guarded by arena
> locks. The trouble is that, in CPython, as soon as you reference any
> read-only object from the globals, you need to raise its refcount.
> ISTR someone mentioned something along the lines of
> sys.eternalize(obj) to flag something as "never GC this thing, it no
> longer has a refcount", which would then allow global objects to be
> referenced in a truly read-only way (eg to call a function). Sadly,
> I'm not expert enough to actually look into implementing it, but it
> does seem like a very cool concept. It also fits into the "warping my
> code a bit" category (eg eternalizing a small handful of key objects,
> and paying the price of "well, now they can never be

Re: [Python-ideas] Add the imath module

[Stephan Houben]

Should we call sort then probable_sort, since the non-zero probability
exists of it going wrong due to a stray cosmic ray?

It's pointless to worry about failure modes which are order of magnitudes
unlikelier than hardware failure.

Stephan

Op vr 13 jul. 2018 14:45 schreef Jeroen Demeyer :

> On 2018-07-13 14:11, Steven D'Aprano wrote:
> > What it *actually* does is:
> >
> >
> is_almost_certainly_prime_except_for_a_ludicrously_microscopic_chance_of_error_thousands_of_times_less_likely_than_a_stray_cosmic_ray_flipping_a_bit_in_memory_and_causing_the_wrong_result_to_be_returned()
>
> That's just a long variant of is_probable_prime()
>
> > If your bank is satisfied with "mere probable prime number" to transfer
> > billions of dollars around the world, then I'm sure that the users of
> > Python's std library should be too.
>
> That's besides the point. I agree that probable primes are good enough,
> just don't call them "prime".
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] anyone need a frozenset or bytearray literal?

[Stephan Houben]

Hi all,

While we are at it, could

{"a":1, "b":2}.freeze()

perhaps create a MappingProxyType?

I've a gist at
https://gist.github.com/stephanh42/d277170dd8a3a2f026c272a4fda15396
with a stand-alone freeze function which attempts to convert objects to a
read-only
version, and dict -> MappingProxyType is one of the transforms.

Stephan


2018-07-12 7:33 GMT+02:00 Gregory P. Smith :

>
>
> On Wed, Jul 11, 2018 at 6:13 PM Guido van Rossum  wrote:
>
>> I know of many use cases for frozenset({...}) and I think a hack along
>> those lines is fine -- but what's the common use for bytearray(b"...") or
>> bytearray((...))? Almost invariably a bytearray is created empty or as a
>> given number of zeros. I certainly wouldn't want to burden the tuple type
>> with a to_bytearray() method, the types are unrelated (unlike set and
>> frozenset).
>>
>
> Agreed, bytearray(b'...') should be way less common.  I don't immediately
> have a use for that beyond merely than disliking the copy from temporary
> bytes object and gc behind the scenes.  I could find a practical use for it
> in micropython where ram is extremely limited, but that VM could already
> implement such a compile time optimization on its own.  The concept of the
> optimization that'd be required just seemed similar to that of frozenset to
> me.
>
> frozenset is the one that led me down this train of thought as I was
> looking at code declaring a bunch on constants.
>
> -gps
>
>
>> On Wed, Jul 11, 2018 at 6:03 PM, Robert Vanden Eynde <
>> robertvandeney...@hotmail.com> wrote:
>>
>>> {1,2,7}.freeze() or {1,2,7}.to_frozenset() seem very natural and if this
>>> can be optimized to avoid the copy, it's perfect.
>>> For bytearray, one use case would be to optimise bytearray([1,2,7,2]) in
>>> something like [1,2,7,2].to_byterray().
>>> About bytes, one could have (1,2,7,2).to_bytes() instead of
>>> bytes((1,2,7,2)) because b'\x01\x02\x07\x02' is long and boring.
>>> What about variables in the values {1,2,x}.freeze() should work too ?
>>> bytes((1,2,7,x)) is not writable as a b string and creates a copy.
>>>
>>>
>>> Le jeu. 12 juil. 2018 à 02:24, Chris Angelico  a
>>> écrit :
>>>
 On Thu, Jul 12, 2018 at 10:13 AM, Gregory P. Smith 
 wrote:
 >
 > On Wed, Jul 11, 2018 at 4:45 PM Jelle Zijlstra <
 jelle.zijls...@gmail.com>
 > wrote:
 >> This could be done safely and without too much craziness if
 .freeze() on a
 >> set returned a new frozenset. The compiler could then safely
 optimize {a,
 >> set, literal}.freeze() into a frozenset literal, because methods on
 builtin
 >> types cannot be monkeypatched. There's been talk of a similar
 optimization
 >> on calls to .format() on string literals (not sure whether it's been
 >> implemented).
 >>
 >> Whether implementing that is a good use of anyone's time is a
 different
 >> question.
 >
 >
 > Neat optimization.  I hadn't considered that.  We do know for sure it
 is a
 > builtin type at that point.
 >
 > If that were implemented, bytes objects could gain a to_bytearray()
 (along
 > the lines of the int.to_bytes() API) method that could be optimized
 away in
 > literal circumstances.

 Be careful: a bytearray is mutable, so this isn't open to very many
 optimizations. A .freeze() method on sets would allow a set display to
 become a frozenset "literal", stored as a constant on the
 corresponding function object, the way a tuple is; but that's safe
 because the frozenset doesn't need to concern itself with identity,
 only value. Example:

 def f(x):
 a = (1, 2, 3) # can be optimized
 b = (x, 4, 5) # cannot
 c = [6, 7, 8] # cannot

 Disassemble this or look at f.__code__.co_consts and you'll see (1, 2,
 3) as a single constant; but the others have to be built.

 +1 on set.freeze(); +0 on bytes.to_bytearray().

 ChrisA
 ___
 Python-ideas mailing list
 Python-ideas@python.org
 https://mail.python.org/mailman/listinfo/python-ideas
 Code of Conduct: http://python.org/psf/codeofconduct/

>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___

Re: [Python-ideas] Secure string disposal (maybe other inmutable seq types too?)

[Stephan Houben]

Would it not be much simpler and more secure to just disable core dumps?

/etc/security/limits.conf on Linux.

If the attacker can cause and read a core dump, the game seems over anyway
since sooner or later he will catch the core dump at a time the string was
not yet deleted.

Stephan


Op za 23 jun. 2018 02:32 schreef Ezequiel Brizuela [aka EHB or qlixed] <
qli...@gmail.com>:

> As all the string in python are immutable, is impossible to overwrite the
> value or to make a "secure disposal" (overwrite-then-free) of a string
> using something like:
>
> >>> a = "something to hide"
> >>> a =  "x"*len(a)
>
> This will lead on the process memory "something to hide" and "x" repeated
> len(a) times.
>
> - Who cares? Why is this relevant?
>   Well if you handle some sensitive information like CC numbers,
> Passwords, PINs, or other kind of information you wanna minimize the chance
> of leaking any of it.
>
> - How this "leak" can happen?
>   If you get a core/memory dump of an app handling sensitive information
> you will get all the information on that core exposed!
>
> - Well, so what we can do about this?
>   I propose to make the required changes on the string objects to add an
> option to overwrite the underlying buffer. To do so:
>
>   * Add a wiped as an attribute that is read-only to be set when the
> string is overwrited.
>   * Add a wipe() method that overwrite the internal string buffer.
>
> So this will work like this:
>
> >>> pwd =getpass.getpass('Set your password:') # could be other sensitive
> data.
> >>> encrypted_pwd = crypt.crypt(pwd)  # crypt() just as example.
> >>> pwd.wiped  # Check if pwd was wiped.
> False
> >>> pwd.wipe()  # Overwrite the underlying buffer
> >>> pwd.wiped  # Check if pwd was wiped.
> True
> >>> print(pwd)  # Print noise (or empty str?)
> >>> del pwd  # Now is in hands of the GC.
>
> The wipe method immediately overwrite the underlying string buffer,
> setting wiped as True for reference so if the string is further used this
> can be checked to confirm that the change was made by a wipe and not by
> another procedure. Also initially the idea is to use unicode NULL datapoint
> to overwrite the string, but this could be change to let the user
> parametrize it over wipe() method.
> An alternative to this is to add a new exception "WipedError" that could
> be throw where the string is accessed again, but I found this method too
> disruptive to implement for a normal/standard string workflow usage.
>
> Quick & Dirty FAQ:
>
> - You do it wrong!, the correct code to do that in a secure way is:
> >>> pwd = crypt.crypt(getpass.getpass('Set your password'))
> Don't you know that fool?
>
>   Well no, the code still generate a temporary string in memory to pass to
> crypt. But now this string is lying there and can't be accessed for an
> overwrite with wipe()
>
>
> - Why not create a new type like in C# or Java?
>
>   I see that this tend to disrupt the usual workflow of string usage. Also
> the idea here is not to offer secure storage of string in memory because
> there is already a few mechanism to achieve with the current Python base. I
> just want to have the hability to overwrite the buffer.
>
>
> - Why don't use one of the standard algorithms to overwrite like DoD5220
> or MIL-STD-414?
>
>   This kind of standard usually are oriented for usage on persistent
> storage, specially on magnetic media for where the data could be "easily"
> recoverd. But this could ve an option that could be implemented adding the
> option to plug a function that do the overwrite work inside the wipe method.
>
>
> - This is far beyond of the almost implementation-agnostic definition of
> the python lang. How about to you make a module with this functionality and
> left the lang as is?
>
>   Well I already do it:
>
> https://github.com/qlixed/python-memwiper/
>
>   But i hit a lot of problems in the road, I was working on me free time
> over the last year on this and make it "almost" work, but that is not
> relevant to the proposal.
>   I think that this kind of security things needs to be tackled from
> within the language itself specially when the lang have GC. I firmly
> believe that the security and protections needs to be part of the "with
> batteries" offer of Python. And I think that this is one little thing that
> could help a lot to secure our apps.
>   Let me know what do you think!
>
> ~ Ezequiel (Ezekiel) Brizuela [ aka Qlixed ] ~
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

Op wo 13 jun. 2018 13:12 schreef Richard Damon :

> My first comment is that special casing values like this can lead to
> some very undesirable properties when you use the function for numerical
> analysis. Suddenly your sind is no longer continuous (sind(x) is no
> longer the limit of sind(x+d) as d goes to 0).
>


The deviations introduced by the special casing are on the order of one ulp.

At that level of detail the sin wasn't continuous to begin with.

>
> As I stated in my initial comment on this, if you are going to create a
> sind function with the idea that you want 'nice' angles to return
> 'exact' results, then what you need to do is have the degree based trig
> routines do the angle reduction in degrees, and only when you have a
> small enough angle, either use the radians version on the small angle or
> directly include an expansion in degrees.
>


Yes that is what my code does.
It reduces degrees to [0,90].

>
> Angle reduction would be based on the identity that sin(x+y) = sin(x) *
> cos(y) + cos(x) * sin(y) and cos(x+y) = cos(x)*cos(y) - sin(x) * sin(y).
>
> If you want to find sin(z) for an arbitrary value z, you can reduce it
> to and x+y where x is some multiple of say 15 degrees, and y is in the
> range -7.5 to 7.5 degrees. You can have stored exact values of sin/cos
> of the 15 degree increments (and only really need them between 0 and 90)
> and then compute the sin and cos of the y value.


This is not how sine functions are calculated. They are calculated by
reducing angle to some interval, then evaluating a polynomial which
approximates the true sine within that interval.

Stephan


> On 6/13/18 6:07 AM, Stephan Houben wrote:
> > 2018-06-13 12:00 GMT+02:00 Robert Vanden Eynde  > <mailto:robertv...@gmail.com>>:
> >
> > What was wrong with my initial implementation with a lookup table
> > ? :D
> >
> > def sind(x):
> > if x % 90 == 0:
> > return (0, 1, 0, -1)[int(x // 90) % 4]
> > else:
> > return sin(radians(x))
> >
> >
> > I kinda missed it, but now you ask:
> >
> > 1. It's better to reduce the angle while still in degrees since one of
> > the advantages
> >of degrees is that the reduction can be done exactly. Converting
> > very large angles
> >first to radians and then taking the sine can introduce a large error,
> >
> > 2. I used fmod instead of % on advice in this thread.
> >
> > 3. I also wanted to special case, 30, 45, and 60.
> >
> >
> >
> > If you want to support multiples of 30, you can do % 30 and // 30.
> >
> >
> > Sure, but I also wanted to special-case 45.
> >
> > Stephan
> >
> >
> >
> > Le mer. 13 juin 2018 à 09:51, Stephan Houben  > <mailto:stephan...@gmail.com>> a écrit :
> >
> > Op di 12 jun. 2018 12:41 schreef Nathaniel Smith
> > mailto:n...@pobox.com>>:
> >
> > On Tue, Jun 12, 2018, 00:03 Stephan Houben
> > mailto:stephan...@gmail.com>> wrote:
> >
> > Hi all,
> >
> > I wrote a possible implementation of sindg:
> >
> >
> https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd
> > <
> https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd>
> >
> > This code first reduces the angle to the [0,90] interval.
> > After doing so, it can be observed that the simple
> > implementation
> >   math.sin(math.radians(angle))
> > produces exact results for 0 and 90, and a result
> > already rounded to nearest for
> > 60.
> >
> >
> > You observed this on your system, but math.sin uses the
> > platform libm, which might do different things on other
> > people's systems.
> >
> >
> >
> > Ok, I updated the code to treat all the values 0, 30, 45, 60
> > and 90 specially.
> >
> > Stephan
> >
> >
> >
> > For 30 and 45, this simple implementation is one ulp
> > too low.
> > So I special-case those to return the
> > correct/correctly-rounded value instead.
> > Note that this does not affect monotonicity around
> > those values.
> >
> >
> > Ag

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

2018-06-13 12:08 GMT+02:00 Robert Vanden Eynde :

> Then of you also want 45, you could do % 15 ? :D
>

Sure, but how the lookup is done in the Python reference code is
ultimately not so important, since it will need to be rewritten in C
if it is to be included in the math package (math is C-only).

And then we'll probably end up with a bunch of if-checks against
the common values.

Stephan


>
>
> Le mer. 13 juin 2018 à 12:07, Stephan Houben  a
> écrit :
>
>> 2018-06-13 12:00 GMT+02:00 Robert Vanden Eynde :
>>
>>> What was wrong with my initial implementation with a lookup table ? :D
>>>
>>> def sind(x):
>>> if x % 90 == 0:
>>> return (0, 1, 0, -1)[int(x // 90) % 4]
>>> else:
>>> return sin(radians(x))
>>>
>>
>> I kinda missed it, but now you ask:
>>
>> 1. It's better to reduce the angle while still in degrees since one of
>> the advantages
>>of degrees is that the reduction can be done exactly. Converting very
>> large angles
>>first to radians and then taking the sine can introduce a large error,
>>
>> 2. I used fmod instead of % on advice in this thread.
>>
>> 3. I also wanted to special case, 30, 45, and 60.
>>
>>
>>>
>>> If you want to support multiples of 30, you can do % 30 and // 30.
>>>
>>
>> Sure, but I also wanted to special-case 45.
>>
>> Stephan
>>
>>
>>>
>>> Le mer. 13 juin 2018 à 09:51, Stephan Houben  a
>>> écrit :
>>>
>>>> Op di 12 jun. 2018 12:41 schreef Nathaniel Smith :
>>>>
>>>>> On Tue, Jun 12, 2018, 00:03 Stephan Houben 
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I wrote a possible implementation of sindg:
>>>>>>
>>>>>> https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd
>>>>>>
>>>>>> This code first reduces the angle to the [0,90] interval.
>>>>>> After doing so, it can be observed that the simple implementation
>>>>>>   math.sin(math.radians(angle))
>>>>>> produces exact results for 0 and 90, and a result already rounded to
>>>>>> nearest for
>>>>>> 60.
>>>>>>
>>>>>
>>>>> You observed this on your system, but math.sin uses the platform libm,
>>>>> which might do different things on other people's systems.
>>>>>
>>>>
>>>>
>>>> Ok, I updated the code to treat all the values 0, 30, 45, 60 and 90
>>>> specially.
>>>>
>>>> Stephan
>>>>
>>>>
>>>>>
>>>>>> For 30 and 45, this simple implementation is one ulp too low.
>>>>>> So I special-case those to return the correct/correctly-rounded value
>>>>>> instead.
>>>>>> Note that this does not affect monotonicity around those values.
>>>>>>
>>>>>
>>>>> Again, monotonicity is preserved on your system, but it might not be
>>>>> on others. It's not clear that this matters, but then it's not clear that
>>>>> any of this matters...
>>>>>
>>>>> -n
>>>>>
>>>> ___
>>>> Python-ideas mailing list
>>>> Python-ideas@python.org
>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>>
>>>
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>>
>>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

2018-06-13 12:00 GMT+02:00 Robert Vanden Eynde :

> What was wrong with my initial implementation with a lookup table ? :D
>
> def sind(x):
> if x % 90 == 0:
> return (0, 1, 0, -1)[int(x // 90) % 4]
> else:
> return sin(radians(x))
>

I kinda missed it, but now you ask:

1. It's better to reduce the angle while still in degrees since one of the
advantages
   of degrees is that the reduction can be done exactly. Converting very
large angles
   first to radians and then taking the sine can introduce a large error,

2. I used fmod instead of % on advice in this thread.

3. I also wanted to special case, 30, 45, and 60.


>
> If you want to support multiples of 30, you can do % 30 and // 30.
>

Sure, but I also wanted to special-case 45.

Stephan


>
> Le mer. 13 juin 2018 à 09:51, Stephan Houben  a
> écrit :
>
>> Op di 12 jun. 2018 12:41 schreef Nathaniel Smith :
>>
>>> On Tue, Jun 12, 2018, 00:03 Stephan Houben  wrote:
>>>
>>>> Hi all,
>>>>
>>>> I wrote a possible implementation of sindg:
>>>>
>>>> https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd
>>>>
>>>> This code first reduces the angle to the [0,90] interval.
>>>> After doing so, it can be observed that the simple implementation
>>>>   math.sin(math.radians(angle))
>>>> produces exact results for 0 and 90, and a result already rounded to
>>>> nearest for
>>>> 60.
>>>>
>>>
>>> You observed this on your system, but math.sin uses the platform libm,
>>> which might do different things on other people's systems.
>>>
>>
>>
>> Ok, I updated the code to treat all the values 0, 30, 45, 60 and 90
>> specially.
>>
>> Stephan
>>
>>
>>>
>>>> For 30 and 45, this simple implementation is one ulp too low.
>>>> So I special-case those to return the correct/correctly-rounded value
>>>> instead.
>>>> Note that this does not affect monotonicity around those values.
>>>>
>>>
>>> Again, monotonicity is preserved on your system, but it might not be on
>>> others. It's not clear that this matters, but then it's not clear that any
>>> of this matters...
>>>
>>> -n
>>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

Op di 12 jun. 2018 12:41 schreef Nathaniel Smith :

> On Tue, Jun 12, 2018, 00:03 Stephan Houben  wrote:
>
>> Hi all,
>>
>> I wrote a possible implementation of sindg:
>>
>> https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd
>>
>> This code first reduces the angle to the [0,90] interval.
>> After doing so, it can be observed that the simple implementation
>>   math.sin(math.radians(angle))
>> produces exact results for 0 and 90, and a result already rounded to
>> nearest for
>> 60.
>>
>
> You observed this on your system, but math.sin uses the platform libm,
> which might do different things on other people's systems.
>


Ok, I updated the code to treat all the values 0, 30, 45, 60 and 90
specially.

Stephan


>
>> For 30 and 45, this simple implementation is one ulp too low.
>> So I special-case those to return the correct/correctly-rounded value
>> instead.
>> Note that this does not affect monotonicity around those values.
>>
>
> Again, monotonicity is preserved on your system, but it might not be on
> others. It's not clear that this matters, but then it's not clear that any
> of this matters...
>
> -n
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

Hi all,

I wrote a possible implementation of sindg:

https://gist.github.com/stephanh42/336d54a53b31104b97e46156c7deacdd

This code first reduces the angle to the [0,90] interval.
After doing so, it can be observed that the simple implementation
  math.sin(math.radians(angle))
produces exact results for 0 and 90, and a result already rounded to
nearest for
60.

For 30 and 45, this simple implementation is one ulp too low.
So I special-case those to return the correct/correctly-rounded value
instead.
Note that this does not affect monotonicity around those values.

So I am still unsure if this belong in the stdlib, but if so, this is how
it could be done.

Stephan




2018-06-12 8:50 GMT+02:00 Chris Angelico :

> On Tue, Jun 12, 2018 at 4:40 PM, Greg Ewing 
> wrote:
> > Tim Peters wrote:
> >
> >> 1. Python's float "%" is unsuitable for argument reduction; e.g.,
> >>
> >>  >>> -1e-14 % 360.0
> >> 360.0
> >>
> >> `math.fmod` is suitable, because it's exact:
> >>
> >>  >>> math.fmod(-1e-14, 360.0)
> >> -1e-14
> >
> >
> > So why doesn't float % use math.fmod?
>
> https://docs.python.org/3/reference/expressions.html#
> binary-arithmetic-operations
> https://docs.python.org/3/reference/expressions.html#id17
> https://docs.python.org/3/reference/expressions.html#id18
>
> (the latter two being footnotes from the section in the first link)
>
> With real numbers, divmod (and thus the // and % operators) would
> always return values such that:
>
> div, mod = divmod(x, y):
> 1) div*y + mod == x
> 2) sign(mod) == sign(y)
> 3) 0 <= abs(mod) < abs(y)
>
> But with floats, you can't guarantee all three of these. The divmod
> function focuses on the first, guaranteeing the fundamental arithmetic
> equality, but to do so, it sometimes has to bend the third one and
> return mod==y.
>
> There are times when it's better to sacrifice one than the other, and
> there are other times when it's the other way around. We get the two
> options.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Trigonometry in degrees

[Stephan Houben]

2018-06-11 19:33 GMT+02:00 Michael Selik :

> Whoops, it turns out Euler's formula does work! I expected imprecision,
> but at least one test matched.
>
> x = 42
> cos(x) + 1j * sin(x) == e ** (1j * x)
>

I think you will find it holds for any x (except inf, -inf and nan).
The boat is less leaky than you think; IEEE floating-point arithmetic goes
out of its way to produce exact answers whenever possible.
(To great consternation of hardware designers who felt that
requiring 1.0*x == x was too expensive.)


> I suppose that's because it's radians.
>

Well, the formula obviously only holds in exact arithmetic
if cos and sin are the versions taking radians.

Stephan


>
>
> On Mon, Jun 11, 2018, 10:24 AM Michael Selik  wrote:
>
>> Would sind and cosd make Euler's formula work correctly?
>>
>> sind(x) + i * sind(x) == math.e ** (i * x)
>>
>> I suspect that adding these functions is kind of like those cartoons
>> where the boat is springing leaks and the character tried to plug them with
>> their fingers. Floating point is a leaky abstraction.
>>
>> Perhaps you'd prefer an enhancement to the fractions module that provides
>> real (not float) math?
>>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Trigonometry in degrees

[Stephan Houben]

2018-06-09 8:18 GMT+02:00 Robert Vanden Eynde :

> For the naming convention, scipy using sindg (therefore Nor sind nor
> sindeg) will make the sind choice less obvious. However if Matlab and Julia
> chooses sind that's a good path to go, Matlab is pretty popular, as other
> pointed out, with Universities giving "free" licences and stuff. With that
> regards, scipy wanting to "be a replacement to Matlab in python and open
> source" it's interesting they chose sindg and not the Matlab name sind.
>


I would suggest that compatibility with a major Python library such as
SciPy is more important than compatibility
with other programming languages.

I would go even further and argue that scipy.special.sindg and its friends
cosdg and tandg
can serve as the reference implementation for this proposal.

Stephan



>
> For the "d" as suffix that would mean "d" as "double" like in opengl.
> Well, let's remember that in Python there's only One floating type, that's
> a double, and it's called float... So python programmers will not think
> "sind means it uses a python float and not a python float32 that C99 sinf
> would". Python programmers would be like "sin takes float in radians, sind
> takes float in degrees or int, because int can be converted to float when
> there's no overflow".
>
> Le sam. 9 juin 2018 à 04:09, Wes Turner  a écrit :
>
>> # Python, NumPy, SymPy, mpmath, sage trigonometric functions
>> https://en.wikipedia.org/wiki/Trigonometric_functions
>>
>> ## Python math module
>> https://docs.python.org/3/library/math.html#trigonometric-functions
>> - degrees(radians): Float degrees
>> - radians(degrees): Float degrees
>>
>> ## NumPy
>> https://docs.scipy.org/doc/numpy/reference/routines.math.htm
>> l#trigonometric-functions
>> - degrees(radians) : List[float] degrees
>> - rad2deg(radians): List[float] degrees
>> - radians(degrees) : List[float] radians
>> - deg2rad(degrees): List[float] radians
>>
>> https://docs.scipy.org/doc/numpy/reference/generated/numpy.sin.html
>>
>>
>> ## SymPy
>> http://docs.sympy.org/latest/modules/functions/elementary.ht
>> ml#sympy-functions-elementary-trigonometric
>> http://docs.sympy.org/latest/modules/functions/elementary.ht
>> ml#trionometric-functions
>>
>> - sympy.mpmath.degrees(radians): Float degrees
>> - sympy.mpmath.radians(degrees): Float radians
>>
>> - https://stackoverflow.com/questions/31072815/cosd-and-sind-with-sympy
>>   - cosd, sind
>>   - https://stackoverflow.com/questions/31072815/cosd-and-sind
>> -with-sympy#comment50176770_31072815
>>
>> > Let x, theta, phi, etc. be Symbols representing quantities in
>> radians. Keep a list of these symbols: angles = [x, theta, phi]. Then, at
>> the very end, use y.subs([(angle, angle*pi/180) for angle in angles]) to
>> change the meaning of the symbols to degrees"
>>
>>
>> ## mpmath
>> http://mpmath.org/doc/current/functions/trigonometric.html
>> - sympy.mpmath.degrees(radians): Float degrees
>> - sympy.mpmath.radians(degrees): Float radians
>>
>>
>> ## Sage
>> https://doc.sagemath.org/html/en/reference/functions/sage/fu
>> nctions/trig.html
>>
>>
>>
>> On Friday, June 8, 2018, Robert Vanden Eynde <
>> robertvandeney...@hotmail.com> wrote:
>>
>>> - Thanks for pointing out a language (Julia) that already had a name
>>> convention. Interestingly they don't have a atan2d function. Choosing the
>>> same convention as another language is a big plus.
>>>
>>> - Adding trig function using floats between 0 and 1 is nice, currently
>>> one needs to do sin(tau * t) which is not so bad (from math import tau, tau
>>> sounds like turn).
>>>
>>> - Julia has sinpi for sin(pi*x), one could have sintau(x) for sin(tau*x)
>>> or sinturn(x).
>>>
>>> Grads are in the idea of turns but with more problems, as you guys said,
>>> grads are used by noone, but turns are more useful. sin(tau * t) For The
>>> Win.
>>>
>>> - Even though people mentionned 1/6 not being exact, so that advantage
>>> over radians isn't that obvious ?
>>>
>>> from math import sin, tau
>>> from fractions import Fraction
>>> sin(Fraction(1,6) * tau)
>>> sindeg(Fraction(1,6) * 360)
>>>
>>> These already work today by the way.
>>>
>>> - As you guys pointed out, using radians implies knowing a little bit
>>> about floating point arithmetic and its limitations. Integer are more
>>> simple and less error prone. Of course it's useful to know about floats but
>>> in many case it's not necessary to learn about it right away, young
>>> students just want their player in the game move in a straight line when
>>> angle = 90.
>>>
>>> - sin(pi/2) == 1 but cos(pi/2) != 0 and sin(3*pi/2) != 1 so sin(pi/2) is
>>> kind of an exception.
>>>
>>>
>>>
>>>
>>> Le ven. 8 juin 2018 à 09:11, Steven D'Aprano  a
>>> écrit :
>>>
 On Fri, Jun 08, 2018 at 03:55:34PM +1000, Chris Angelico wrote:
 > On Fri, Jun 8, 2018 at 3:45 PM, Steven D'Aprano 
 wrote:
 > > Although personally I prefer the look of d as a prefix:
 > >
 > > dsin, dcos, dtan
 > >
 > > That's more obviou

Re: [Python-ideas] Let try-except check the exception instance

[Stephan Houben]

Current documentation says:

"An object is compatible with an exception if it is the class or a base
class of the exception object or a tuple containing an item compatible with
the exception."

https://docs.python.org/3/reference/compound_stmts.html#the-try-statement

It is, in my opinion, not very clear from this that the __instancecheck__
mechanism is bypassed.

Should the documentation perhaps be adapted to explain that the class needs
to actually occur in the MRO
and that virtual base classes are not considered for matching purposes?

"An object is compatible with an exception if it is the class or a
non-virtual base class of the exception object or a tuple containing an
item compatible with the exception.
The exception matching machinery ignores the __instancecheck__ mechanism."


Stephan


2018-05-31 9:19 GMT+02:00 Terry Reedy :

> On 5/31/2018 12:47 AM, Danilo J. S. Bellini wrote:
>
>> Hi!
>> I was working on handling some exceptions from external software
>> (e.g. database constraint triggers)
>> switching the handler based on the messages that had been sent.
>> Today we can do something like (running on Python 3.6.5):
>>
>>
>> try:
>
 ... # [...]
>> ... session.commit() # Here it raises!
>> ... # [...]
>> ... except DatabaseError as exc:
>> ... msg = get_db_error_msg_from_exception(exc)
>> ... if msg == "beyond_limit":
>> ... # [...]
>> ... elif msg == "no_funds":
>> ... # [...]
>> ... else:
>> ... raise
>>
>>
>> That works,
>>
>
> Yes, it works perfectly well, AND it exposes the fact that your code
> depends on the message, which I think is a good thing.
>
> As Stephen said, messages are intentionally not part of the defined API.
> As a matter of curtesy, we usually restrict message changes to new versions
> and do not backport the change. An exception may be made if we decide that
> a message is sufficiently erroneous that is likely misleads people.  In any
> case, message dependent code may be version dependent.
>
> --
> Terry Jan Reedy
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

2018-05-18 15:37 GMT+02:00 Steven D'Aprano :

>
> Earlier you described this suggestion as "a silly joke".
>
> https://mail.python.org/pipermail/python-ideas/2018-May/050861.html



The joke proposal was to write all keywords in Python using bold font
variation,
as a reaction to a similar joke proposal to precede all keywords in Python
with  \.

In contrast this isn't even a proposal, it is merely a description of
an existing feature.

Practically speaking, suppose "spam" becomes a keyword in 3.8, and I
have a module which I want to make compatible with 3.8 AND I want
to preserve the API for pre-3.8 versions, then I will first update my module
to use some alternative spelling spam_ throughout, and then, in a single
place,
write:

𝐬𝐩𝐚𝐦 = spam_  # exploit NFKC normalization to set identifier "spam" for
backward compatibility

Even if this single line shows up as mojibake in somebody's editor, it
shouldn't inconvenience them too much.



> I think you were right then.
>
>
> > I am merely defending the status quo.
> > I demonstrate how the intended behavior can be achieved using features
> > available in current Python versions.
>
> Aside from the fact that font, editor and keyboard support for such
> non-BMP Unicode characters is very spotty, it isn't the intended
> behaviour.
>

I am not sure from what you conclude that.

There seem to be three design possibilities here:
1.  𝐢𝐟 is an alternative spelling for the keyword if
2.  𝐢𝐟 is an identifier
3.  𝐢𝐟 is an error

I am pretty sure option 1 (non-ASCII spelling of keywords) was not intended
(doc says about keywords: "They must be spelled exactly as written here:")

So it is either 2 or 3.  Option 3 would only make sense if we conclude that
it is
a bad idea to have an identifier with the same name as a keyword.
Whereas this whole thread so far has been about introducing such a feature.

So that leaves 2, which happens to be the implemented behavior.

As an aside:
A general observation of PEP-3131 and Unicode identifiers in Python:
from the PEP it becomes clear that there have been several proposals
of making it more restricted (e.g. requiring source code to be already in
NFKC normal form, which would make 𝐢𝐟 illegal, disallowing confusables,
etc.)

Ultimately this has been rejected and the result is that we have a rather
liberal
definition of Unicode identifiers in Python. I feel that 𝐢𝐟  being a valid
identifier fits into that pattern, just as various confusable spellings of
if
would be legal identifiers. In theory this could lead to all kinds of
sneaky attacks where code appears to do one thing but does another,
but it just doesn't seem so big an issue in practice.


> As you point out, the intended behaviour is that obj.𝐢𝐟 and
> obj.if ought to be identical. Since the later is a syntax error, so
> should be the former.
>

NFKC normalization is restricted to identifiers.
Keywords "must be spelled exactly as written here."


>
>
> > It is guaranteed to work by PEP-3131:
> > https://www.python.org/dev/peps/pep-3131
> >
> > "All identifiers are converted into the normal form NFKC while parsing;
> > comparison of identifiers is based on NFKC."
> >
> > NFKC normalization means spam must be considered the same identifier as
> > 𝐬𝐩𝐚𝐦 .
>
>
> It's not the NFKC normalization that I'm questioning. Its the fact that
> it is done too late to catch the use of a keyword.
>
>
See above.

Stephan


>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

2018-05-18 13:37 GMT+02:00 Steven D'Aprano :

> On Fri, May 18, 2018 at 11:17:13AM +0200, Stephan Houben wrote:
>
> > And the alternative is to replace all occurrences of
> > spam with 𝐬𝐩𝐚𝐦 , which has the same effect and also is
> > backward-compatible with 3.x for x < 8.
> >
> > So there is already a kind of solution available, albeit an ugly one.
>
> You are kidding, I hope.
>


I am not kidding; I am merely defending the status quo.
I demonstrate how the intended behavior can be achieved using features
available in current Python versions.

The approach has at least the following two technical advantages.
1. It requires no change to Python
2. It provides backwards compatibility all the way back to 3.0.

The spelling is arguably ugly, but this should be weighted against
the, IMHO, extremely rare use of this feature.


>
> If that works at all, I don't think its something we want to guarantee
> will work.


It is guaranteed to work by PEP-3131:
https://www.python.org/dev/peps/pep-3131

"All identifiers are converted into the normal form NFKC while parsing;
comparison of identifiers is based on NFKC."

NFKC normalization means spam must be considered the same identifier as
𝐬𝐩𝐚𝐦 .

Note that the choice for NFKC normalization was apparently explicitly
discussed and decided upon at the time.
Since the difference between NFC and NFKC is exactly that identifiers like
spam and  𝐬𝐩𝐚𝐦 are different
under the former and identical under the latter, I take it this is all
quite intentional.



> And for what it's worth, what I see is eight empty boxes
> (missing glyph symbols).
>
>
I am afraid that mostly shows that your mailer has a bug in handling
non-BMP unicode
characters; you should be seeing FOUR missing glyph symbols.

Stephan


>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

2018-05-18 8:05 GMT+02:00 Greg Ewing :

> Steven D'Aprano wrote:
>
>> It's Python 3.8, and I learn that in 4.0 "spam" is going to become a
>> keyword. I simply take my code and change all the references spam to \spam,
>> and I've future-proofed the code for 4.0 while still keeping compatibility
>> with 3.8 and 3.9.
>>
>
> Okay, maybe it helps a little bit, but not very much. There
> will still be a lot of reluctance to add new keywords, because
> of the disruption it will cause to existing code


And the alternative is to replace all occurrences of
spam with 𝐬𝐩𝐚𝐦 , which has the same effect and also is
backward-compatible with
3.x for x < 8.

So there is already a kind of solution available, albeit an ugly one.

Stephan

If we've learned nothing else from the Python 3 changeover,
> it's that many people work in an environment where it's
> extremely difficult to update working code.
>
> --
> Greg
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

OK, that was a silly joke, but
did you realize that the following already WORKS TODAY:

>>> class Foo:
...  pass
>>> f = Foo()
>>> f.__dict__["if"] = 42
>>> f.𝐢𝐟
42

That's right, I can access a member whose name is a keyword
by simply using Unicode bold.

Because Python normalizes Unicode fonts but  this is
apparently AFTER keywords have been recognized.

Stephan


2018-05-17 19:53 GMT+02:00 Stephan Houben :

> Fortunately we have Unicode bold characters nowadays
>
> 𝐢𝐟 if 𝐢𝐧 in:
> 𝐫𝐞𝐭𝐮𝐫𝐧 return
>
> Look ma! No syntactic ambiguity!
>
> Stephan
>
> 2018-05-17 19:10 GMT+02:00 Chris Barker via Python-ideas <
> python-ideas@python.org>:
>
>> On Wed, May 16, 2018 at 2:09 PM, Carl Smith  wrote:
>>
>>> If your position is that Guido shouldn't introduce keywords that are
>>> currently used as names at all,
>>>
>>
>> Exactly -- which is why I'm wondering my no one (that I've seen -- long
>> thread)  is presenting the backwards option:
>>
>> Any new keywords introduced will be non-legal as regular names.
>>
>> \new_key_word
>>
>> for instance.
>>
>> Makes me think that it may have been good to have ALL keywords somehow
>> non-legal as user-defined names -- maybe ugly syntax, but it would make a
>> clear distinction.
>>
>> how ugly would this be?
>>
>> \for i in range(n):
>> \while \True:
>> ...
>>
>> pretty ugly :-(
>>
>> But maybe not so much if only a handful of new ones
>>
>> Or is there another currently illegal character that could be used that
>> would be less ugly?
>>
>> I'm actually confused as to what the point is to the \ prefix idea for
>> names:
>>
>> * It would still require people to change their code when a new keyword
>> was introduced
>>
>> * It would be no easier / harder than adding a conventional legal
>> character -- trailing underscore, or ???
>>
>> * but now the changed code would no longer run on older versions of
>> python.
>>
>> I guess it comes down to why you'd want to call out:
>>
>> "this is a name that is almost like a keyword"
>>
>> Seems like a meh, meh, lose proposal to me.
>>
>> OK, I see one advantage -- one could have code that already has BOTH word
>> and word_ names in it. So when word becomes a keyword, a tool that
>> automatically added an underscore would break the code. whereas if it
>> automatically added an currently illegal character, it wouldn't shadow
>> anything.
>>
>> But a sufficiently smart tool could get around that, too.
>>
>> -CHB
>>
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R(206) 526-6959   voice
>> 7600 Sand Point Way NE   (206) 526-6329   fax
>> Seattle, WA  98115   (206) 526-6317   main reception
>>
>> chris.bar...@noaa.gov
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

Fortunately we have Unicode bold characters nowadays

𝐢𝐟 if 𝐢𝐧 in:
𝐫𝐞𝐭𝐮𝐫𝐧 return

Look ma! No syntactic ambiguity!

Stephan

2018-05-17 19:10 GMT+02:00 Chris Barker via Python-ideas <
python-ideas@python.org>:

> On Wed, May 16, 2018 at 2:09 PM, Carl Smith  wrote:
>
>> If your position is that Guido shouldn't introduce keywords that are
>> currently used as names at all,
>>
>
> Exactly -- which is why I'm wondering my no one (that I've seen -- long
> thread)  is presenting the backwards option:
>
> Any new keywords introduced will be non-legal as regular names.
>
> \new_key_word
>
> for instance.
>
> Makes me think that it may have been good to have ALL keywords somehow
> non-legal as user-defined names -- maybe ugly syntax, but it would make a
> clear distinction.
>
> how ugly would this be?
>
> \for i in range(n):
> \while \True:
> ...
>
> pretty ugly :-(
>
> But maybe not so much if only a handful of new ones
>
> Or is there another currently illegal character that could be used that
> would be less ugly?
>
> I'm actually confused as to what the point is to the \ prefix idea for
> names:
>
> * It would still require people to change their code when a new keyword
> was introduced
>
> * It would be no easier / harder than adding a conventional legal
> character -- trailing underscore, or ???
>
> * but now the changed code would no longer run on older versions of python.
>
> I guess it comes down to why you'd want to call out:
>
> "this is a name that is almost like a keyword"
>
> Seems like a meh, meh, lose proposal to me.
>
> OK, I see one advantage -- one could have code that already has BOTH word
> and word_ names in it. So when word becomes a keyword, a tool that
> automatically added an underscore would break the code. whereas if it
> automatically added an currently illegal character, it wouldn't shadow
> anything.
>
> But a sufficiently smart tool could get around that, too.
>
> -CHB
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] String and bytes bitwise operations

[Stephan Houben]

Seems you want numpy:

>>> import numpy
>>> numpy.frombuffer(b"Hello", dtype=numpy.uint8) ^
numpy.frombuffer(b"World", dtype=numpy.uint8)
array([31, 10, 30,  0, 11], dtype=uint8)

Stephan

2018-05-17 12:53 GMT+02:00 Ken Hilton :

> Hi all,
>
> We all know the bitwise operators: & (and), | (or), ^ (xor), and ~ (not).
> We know how they work with numbers:
>
> 420 ^ 502
>
> 110100100
> 10110
> == XOR ==
> 001010010
> = 82
>
> But it might be useful in some cases to (let's say) xor a string (or
> bytestring):
>
> HELLO ^ world
>
> 01001000 01000101 01001100 01001100 0100
> 01110111 0110 01110010 01101100 01100100
> === XOR 
> 0011 00101010 0010 0010 00101011
> = ?*> +
>
> Currently, that's done with this expression for strings:
>
> >>> ''.join(chr(ord(a) ^ ord(b)) for a, b in zip('HELLO', 'world'))
> '?*> +'
>
> and this expression for bytestrings:
>
> >>> bytes(a ^ b for a, b in zip(b'HELLO', b'world'))
> b'?*> +'
>
> It would be much more convenient, however, to allow a simple xor of a
> string:
>
> >>> 'HELLO' ^ 'world'
> '?*> +'
>
> or bytestring:
>
> >>> b'HELLO' ^ b'world'
> b'?*> +'
>
> (All of this applies to other bitwise operators, of course.)
> Compatibility issues are a no-brainer - currently, bitwise operators for
> strings raise TypeErrors.
>
> Thanks.
>
> Suggesting,
> Ken
> ​ Hilton​
> ;
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Verbatim names (allowing keywords as names)

[Stephan Houben]

Hi all,

One problem already alluded to with the \identifier syntax is that it only
works
if the old Python version is sufficiently recent to understand \.

What about using parentheses to allow a keyword to be used as an identifier:
(where)(x, y)

This then in combination with allowing keywords in the following
unambiguous locations:
1. After dot ("numpy.where")
2. After def and class ("def where")
3. After "as".


This should make it possible to write code which works in a hypothetical
future Python
version where "where" is a keyword, and which also works with current
Python versions.

Stephan


2018-05-16 11:03 GMT+02:00 Paul Moore :

> On 16 May 2018 at 09:56, Eric V. Smith  wrote:
> > On 5/16/18 4:47 AM, Eric V. Smith wrote:
> >>
> >> On 5/16/18 4:13 AM, Paul Moore wrote:
> >
> >
> >>> Can you give a worked example of how this would
> >>> help if we wanted to introduce a new keyword? For example, if we
> >>> intended to make "where" a keyword, what would numpy and its users
> >>> need to do to continue using `numpy.where`?
> >>
> >>
> >> I think they'd have to change to `numpy.\where` when `where` became a
> >> keyword.
> >
> >
> > To be clear: this would apply to any code that uses numpy.where, not just
> > the code that defines it.
> >
> > The only way to bullet-proof your code so that it would never need any
> > modifications in the future would be to put a backslash in front of every
> > identifier. Or maybe just all-lowercase identifiers, since we're
> unlikely to
> > make a keyword with uppercase chars in it.
> >
> > And since no one in their right mind would do that, there's still the
> risk
> > of your code breaking in the future. But at least there would be a way of
> > fixing it in a way that would work both with old versions of python where
> > the identifier isn't a keyword, and for versions where it is. That is,
> once
> > "old versions" include ones that support verbatim names.
>
> That's about what I thought - thanks.
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Crazy idea: allow keywords as names in certain positions

[Stephan Houben]

Note that Javascript does something similar,
in that reserved keywords are allowed in member access.

var x = {if : 42};
console.log(x.if);

Stephan


2018-05-13 20:42 GMT+02:00 Eric Fahlgren :

> On Sun, May 13, 2018 at 11:20 AM Guido van Rossum 
> wrote:
>
>> For example, we could allow keywords after 'def' and after a period, and
>> then the following would become legal:
>>
>
> ​Our modeling database overloads getattr/setattr (think SQLAlchemy) to
> allow us to access to database fields as if they were Python data members.
> Nothing new here, but we do have problems with keyword collisions on some
> of the objects, as we are wrapping an already-existing modeling language
> (MSC Adams Solver dataset) with our objects.  We were pleased with 'print'
> became a function, because it removed the restriction from that one, but
> one of the remaining ones is 'return', like this:
>
> class Sensor:
> def __init__(self):
> setattr(self, "print", 0)
> setattr(self, "return", 0)
>
> s = Sensor()
> s.print  # Works now, didn't in Python 2.
> s.return  # Bork.
>
> I have a decades old +1 on this.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Move optional data out of pyc files

[Stephan Houben]

There are libraries out there like this:

https://docopt.readthedocs.io/en/0.2.0/

which use docstrings for runtime info.

Today we already have -OO which allows us to create docstring-less bytecode
files in case we have, after careful consideration, established that it is
safe to do so.

I think the current way (-OO) to avoid docstring loading is the correct one.
It pushes the responsibility on whoever did the packaging to decide if -OO
is appropriate.

The ability to remove the docstrings after bytecode generation would be
kinda nice
(similar to Unix "strip" command)
but given how fast bytecode compilation is, frankly I don't think it is
very important.

Stephan

2018-04-10 19:54 GMT+02:00 Zachary Ware :

> On Tue, Apr 10, 2018 at 12:38 PM, Chris Angelico  wrote:
> > A deployed Python distribution generally has .pyc files for all of the
> > standard library. I don't think people want to lose the ability to
> > call help(), and unless I'm misunderstanding, that requires
> > docstrings. So this will mean twice as many files and twice as many
> > file-open calls to import from the standard library. What will be the
> > impact on startup time?
>
> What about instead of separate files turning the single file into a
> pseudo-zip file containing all of the proposed files, and provide a
> simple tool for removing whatever parts you don't want?
>
> --
> Zach
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Sets, Dictionaries

[Stephan Houben]

Perhaps one day we will be able to use

∅

for the empty set.
That would actually match conventional notation.

Note that this is not valid syntax today (not a legal Unicode identifier).

Stephan

Op do 29 mrt. 2018 17:49 schreef Chris Angelico :

> On Fri, Mar 30, 2018 at 2:42 AM, Julia Kim 
> wrote:
> > My suggestion is to change the syntax for creating an empty set and an
> empty dictionary as following.
> >
> > an_empty_set = {}
> > an_empty_dictionary = {:}
> >
> >
> > Compatibility issues could be resolved with a program which takes a
> Python program (codes) as a text and edits it.
> >
>
> Unfortunately, that's not sufficient for backward compatibility.
> Imagine reading something on Stack Overflow that has this line of code
> in it:
>
> words = {}
>
> Does that make an empty set or an empty dict? How would anyone know?
> Are you going to go through every piece of code on Stack Overflow and
> change it? What about all the blogs out there? Printed books?
>
> Simply transforming code doesn't work. Having the same syntax perform
> drastically different things on different versions of the interpreter
> is almost certainly not going to fly.
>
> The only way that this might be usable is if you use a __future__
> directive. And if it were done that way, I would expect most code out
> there to NOT use the directive, and therefore to keep today's
> semantics - which means the change effectively hasn't happened for
> most people.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Syntax idea for 2D lists\arrays

[Stephan Houben]

I'd just do this, which works today:

==
import numpy
import io

ar = numpy.loadtxt(io.StringIO("""
1   5   9   155
53  44  44  34
"""))
==

Of course, this is only worth the trouble if you
are somehow loading a very large matrix.
(And then, are you sure you want to embed it in your code?)

Stephan


2018-03-15 6:15 GMT+01:00 Steven D'Aprano :

> On Thu, Mar 15, 2018 at 01:32:35AM +0100, Mikhail V wrote:
>
> > Idea is a concept for 2D arrays/lists syntax, which should simplify
> > some editing boilerplate while working with arrays and improve
> > readability for bigger arrays.
>
> I don't understand; we already have perfectly good syntax for working
> with 2D arrays.
>
> > Lets start with a simple list example :
> >
> > L ===
> > 1   5   9   155
> > 53  44  44  34
> >
> > returns a 2d list:
> > [[1, 5, 9, 155], [53, 44, 44, 34]]
>
> We already have:
>
> L = [[1, 5, 9, 155], [53, 44, 44, 34]]
>
> which is more compact (one line rather than two) and explicitly delimits
> the start and end of each list. Like everything else in Python, it uses
> commas to separate items, not whitespace. If you prefer:
>
> L = [[1, 5, 9, 155],
>  [53, 44, 44, 34]]
>
>
> Using spaces to separate items has the fatal flaw that it cannot
> distinguish
>
> x - y 0  # two items, the expression `x - y` and the integer 0
>
> from:
>
>x - y 0  # three items, `x`, `-y`, and 0
>
>
> making it ambiguous. I stopped reading your post once I realised that.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Descouraging the implicit string concatenation

[Stephan Houben]

Op 14 mrt. 2018 15:23 schreef "Facundo Batista" :


I propose the discouragement of the idiom.



What does that mean?

Stephan


Regards,

--
.Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
Twitter: @facundobatista
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Descouraging the implicit string concatenation

[Stephan Houben]

Note that this has already been proposed and rejected before:

https://www.python.org/dev/peps/pep-3126/

Stephan

Op 14 mrt. 2018 15:01 schreef "Oleg Broytman" :

> On Thu, Mar 15, 2018 at 12:15:52AM +1100, Steven D'Aprano <
> st...@pearwood.info> wrote:
> > On Wed, Mar 14, 2018 at 09:18:30AM -0300, Facundo Batista wrote:
> > > We should write the following, instead:
> > >
> > > long_string = (
> > > "some part of the string " +
> > > "with more words, actually is the same " +
> > > "string that the compiler puts together")
> >
> > Should we? I disagree.
> >
> > Of course you're welcome to specify that in your own style-guide for
> > your own code, but I won't be following that recommendation.
> >
> >
> > > I know that "no change to Python itself" is needed, but having a
> > > formal discouragement of the idiom will help in avoiding people to
> > > fall in mistakes like:
> > >
> > > fruits = {
> > > "apple",
> > > "orange"
> > > "banana",
> > > "melon",
> > > }
> >
> > People can make all sorts of mistakes through carlessness. I wrote
> >
> > {y, x*3}
> >
> > the other day instead of {y: x**3}. (That's *two* errors in one simple
> > expression. I wish I could say it was a record for me.) Should we
> > "discourage" exponentiation and dict displays and insist on writing
> > dict((y, x*x*x)) to avoid the risk of errors? I don't think so.
>
>We should fix what causes real problems, not convoluted ones. And
> this particular misfeature caused problems for me.
>
> > --
> > Steve
>
> Oleg.
> --
>  Oleg Broytmanhttp://phdru.name/p...@phdru.name
>Programmers don't die, they just GOSUB without RETURN.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 572: Statement-Local Name Bindings

[Stephan Houben]

Hi all,

I would like to observe that there is already a way to bind names in an
expression:

a = (lambda b: [b, b])(b=f())

Currently this is not so performant, but there is nothing
stopping the compiler from special-casing such an
IIFE (Immediate Invoked Function Expression),
to use the Javascript terminology.

It would then be fast in, say, Python 3.8, but still work in earlier
versions.

Stephan



2018-03-02 14:26 GMT+01:00 Paul Moore :

> On 2 March 2018 at 12:48, Nick Coghlan  wrote:
> > The design intent related rationale stems from the fact that closed over
> > references can live for an arbitrarily long time (as can regular function
> > locals in a generator or coroutine), and I think statement locals should
> be
> > as reliably ephemeral as we can reasonably make them if they're going to
> be
> > different enough from function locals to be worth the hassle of adding
> them.
> >
> > So that means asking a bunch of questions and deliberately giving the
> > *opposite* answer for statement locals than I would for function locals:
> >
> > * Visible in locals()? Yes for function locals, no for statement locals
> > * Visible in frame.f_locals? Yes for function locals, no for statement
> > locals
> > * Prevents access to the same name in outer scopes? Yes for function
> locals,
> > no for statement locals
> > * Can be closed over? Yes for function locals, no for statement locals
> >
> > Answer "no" to all those questions is only reasonable if statement local
> > references *look* different from regular variable names. But I also
> think we
> > need to proposing answering "No" to all of them to make statement locals
> > potentially interesting enough to be worth considering.
>
> OK, that argument makes sense to me. At least in the sense that it
> makes a good case for how statement-local names should work *if* we
> adopt them. I reserve the right to change my mind, as I haven't
> thought this through fully, but at least superficially I'm with you
> this far.
>
> I will note that I don't see this as a compelling reason for *having*
> them, just a good analysis of how they might behave if we do.
>
> > Neither PEP 527 nor 3150 *needs* the syntactic markers - the compiler can
> > figure out what is going on because it does the symbol table analysis
> pass
> > before the code generation pass, and hence can tag names appropriately
> based
> > on what it finds.
> >
> > My concern is for people reading the code, where omitting a syntactic
> marker
> > also forces *humans* to read things in two passes to make sure they
> > understand them correctly. Consider this example:
> >
> > x = 10
> > data = [x*x for i in range(10)]
> >
> > This will give you a 10 item list, where each item is 100.
> >
> > But now consider a world with statement locals, where statement locals
> use
> > the same reference syntax as regular locals:
> >
> > x = 10
> > data = [x*x for i in range(10) if (12 as x)]
> >
> > That's a deliberately pathological example (and you can already write
> > something similarly misleading using the "for x in [12]" trick), but the
> > fact remains that we allow out of order evaluation in expressions in a
> way
> > that we don't permit for statements.
>
> OK. That's my concern as well. But my conclusion is different - I view
> this as a pretty strong argument that the complexity cost is too high
> to justify the feature, rather than as a difficulty we need to
> mitigate in order to make the feature usable.
>
> > With a syntactic marker though, there's typically going to be less
> ambiguity
> > about where a name comes from:
> >
> > x = 10
> > data = [.x*.x for i in range(10) if (12 as .x)]
> >
> > It's not a panacea (since you may still be shadowing a name from an outer
> > statement), and adapting it for PEP 3150 isn't without it's problems
> > (specifically, you need to allow ".x = 12" in the given clause to make
> the
> > names match up), but it should be less confusing than allowing a new
> > subexpression to interfere with name resolution semantics folks have been
> > relying on for years.
>
> On my screen right now, I can barely see the dots. That's in email
> with a proportional font, so not "real coding", but it's surprising
> (and somewhat depressing :-() to think that probably the majority of
> code I read these days is in emails.
>
> So I don't think that this is a good enough fix to warrant making
> Tim's screen look gritty.
>
> > If statement locals behave just like function locals, then there's no
> reason
> > to add them to the language in the first place - "(expr as name)" would
> just
> > become a confusing second way to spell "name = expr".
>
> Well, the whole reason this debate keeps coming up is because people
> keep finding places they want to type "name = expr" but they can't,
> because their context isn't a statement. We're basically trying to
> design a replacement for "name = expr" that can be used in an
> expression. And the status quo is "refactor your expressi

Re: [Python-ideas] Temporary variables in comprehensions

[Stephan Houben]

Note that you can already do:

 [y + g(y) for x in range(10) for y in [f(x)]]

i.e. for y  in [expr]
does exactly what the OP wants.
No new syntax needed.

If you hang out on python-list , you'll soon notice
that many newbies struggle already with the list comprehension
syntax.
It's a mini-language which is almost, but not entirely,
exactly unlike normal Python code.
Let's not complicate it further.

Stephan


2018-02-15 10:53 GMT+01:00 Evpok Padding :

> For simple cases such as `[y + g(y) for y in [f(x) for x in range(10)]]`,
> I don't really see what the issue is, if you really want to make it
> shorter,
> you can ``[y + g(y) for y in map(f,range(10))]` which is one of the rare
> case where I like `map` more than comprehensions.
>
> For more complex case, just define a intermediate generator along the lines
> ```
> f_samples = (f(x) for x in range(10))
> [y+g(y) for y in f_samples]
> ```
> Which does exactly the same thing but
>   - Is more readable and explicit
>   - Has no memory overhead thanks to lazy evaluation
> (btw, you should consider generators for your nested comprenshions)
>
> While I am sometimes in the same state of mind, wishing for variables in
> comprehensions seems to me like a good indicator that your code needs
> refactoring.
>
> Best,
>
> E
>
> On 15 February 2018 at 10:32, Jamie Willis 
> wrote:
> >
> > I +1 this at surface level; Both Haskell list comprehensions and Scala
> for comprehensions have variable assignment in them, even between iterating
> and this is often very useful. Perhaps syntax can be generalised as:
> >
> > [expr_using_x_and_y
> >  for i in is
> >   x = expr_using_i
> >  for j in is
> >   y = expr_using_j_and_x]
> >
> > This demonstrates the scope of each assignment; available in main result
> and then every clause that follows it.
> >
> > Sorry to op who will receive twice, forgot reply to all
> >
> > On 15 Feb 2018 7:03 am, "fhsxfhsx"  wrote:
> >>
> >> As far as I can see, a comprehension like
> >> alist = [f(x) for x in range(10)]
> >> is better than a for-loop
> >> for x in range(10):
> >>   alist.append(f(x))
> >> because the previous one shows every element of the list explicitly so
> that we don't need to handle `append` mentally.
> >>
> >> But when it comes to something like
> >> [f(x) + g(f(x)) for x in range(10)]
> >> you find you have to sacrifice some readableness if you don't want two
> f(x) which might slow down your code.
> >>
> >> Someone may argue that one can write
> >> [y + g(y) for y in [f(x) for x in range(10)]]
> >> but it's not as clear as to show what `y` is in a subsequent clause,
> not to say there'll be another temporary list built in the process.
> >> We can even replace every comprehension with map and filter, but that
> would face the same problems.
> >>
> >> In a word, what I'm arguing is that we need a way to assign temporary
> variables in a comprehension.
> >> In my opinion, code like
> >> [y + g(y) for x in range(10) **some syntax for `y=f(x)` here**]
> >> is more natural than any solution we now have.
> >> And that's why I pro the new syntax, it's clear, explicit and readable,
> and is nothing beyond the functionality of the present comprehensions so
> it's not complicated.
> >>
> >> And I hope the discussion could focus more on whether we should allow
> assigning temporary variables in comprehensions rather than how to solve
> the specific example I mentioned above.
> >>
> >>
> >>
> >>
> >>
> >> ___
> >> Python-ideas mailing list
> >> Python-ideas@python.org
> >> https://mail.python.org/mailman/listinfo/python-ideas
> >> Code of Conduct: http://python.org/psf/codeofconduct/
> >>
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> >
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Possible Enhancement to py Launcher - set default

[Stephan Houben]

Hi all,

Just want to point out that if tools can accept a config file in
a cross-platform standard location
(perhaps in addition to a platform-specific one),
then that is incredibly useful.

Just search on Github for "dotfiles", and see how many people
store their configuration in a git repo, so they can just go to
a fresh machine, "git clone dotfiles" in their $HOME and be fully set up.

A platform-independent location also simplifies life for us who
use Linux, Windows and MacOs on a daily basis.
(And then there is the fact that these platform "standards" are
only followed very haphazardly anyway, e.g.
Windows Vim uses ~\.vimrc on Windows and not somewhere in $LOCALAPPDATA .)

As an aside, I don't agree with the "appdirs" package on Linux:
XDG != Linux. That may seem pedantry, but while ~/.local is perhaps
not too bad for user-specific config, /etc/xdg is almost certainly the
wrong location for global config any application that is not part of
a desktop environment.
(In fact, such a directory may not exist on a typical headless Linux
install.)

Stephan


2018-02-06 16:44 GMT+01:00 Paul Moore :

> On 6 February 2018 at 15:23, Eric Fahlgren  wrote:
> > Right, different planets, but orbiting the same star.  I was thinking
> about
> > the consolidation of the Windows registry layout a year or two ago, don't
> > recall who spearheaded that (Steve Dower?).  In any case, if the various
> > tools either followed that convention, or we came up with an ini-based
> one
> > that was consistent with it and usable on Unix (.pyconf or something)...
>
> Yep, that would be an informational PEP, defining standards we expect
> Python applications to follow. There's a lot more Python
> *applications* than there are Python *distributions*, and I'm not
> convinced a standard for applications would get much traction (even
> ignoring the need they'd have for backward compatibility) but if
> someone wants to try to get consensus on something, then have fun!
>
> Actually, the `appdirs` project (https://pypi.python.org/pypi/appdirs)
> does exactly this - provides a portable interface for applications to
> store config data in platform-specific locations. The correct answer
> is probably to persuade application developers to use appdirs rather
> than their own schemes.
>
> Pip and py both use appdirs-compatible schemes (py doesn't use appdirs
> itself, as it's not written in Python, but pip does).
>
> pip: appdirs.user_config_dir('pip', appauthor=False, roaming=True)
> py: appdirs.user_config_dir()
>
> You could argue that appdirs offers too many options - but if all
> applications used appdirs, you could have that debate once with the
> appdirs authors, rather than having to persuade every application in
> turn.
>
> > Yeah, our Windows dev environment uses Cygwin, so I'm constantly
> confused.
> > :)
>
> Yuk, Cygwin. I'll refrain from commenting further :-)
>
> > Not sure how to make pip cough up similar verbose output, but when it
> > started complaining about legacy formats, I just followed its directions
> and
> > this works:
> >
> >> ll $USERPROFILE/pip/pip.ini
> > -rw-r--r-- efahlgren 2017-04-30 15:51 'C:/Users/efahlgren/pip/pip.ini'
>
> Backward compatibility. When we moved to the Windows-standard
> location, we left in fallbacks to the old locations. I've no idea
> whether pip sees Cygwin as Windows-like or Unix-like, so anything
> could be going on beyond that.
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Format mini-language for lakh and crore

[Stephan Houben]

What about something like:

f"{x:₹d}"

₹ = Indian Rupees symbol

I realize it is not ASCII but ₹ would be, for the target audience,
 both easy to type
(Ctrl-Alt-4 on Windows Indian English keyboard layout)
 and be mnemonic
("format number like you would format an amount in rupees").

Stephan


2018-02-01 6:11 GMT+01:00 Nick Coghlan :

> On 1 February 2018 at 14:11, Nick Coghlan  wrote:
> > On 1 February 2018 at 08:14, Eric V. Smith  wrote:
> >> On 1/29/2018 2:13 AM, Nick Coghlan wrote:
> >>> Given the example, I think a more useful approach would be to allow an
> >>> optional digit grouping specifier after the comma separator, and allow
> >>> the separator to be repeated to indicate non-uniform groupings in the
> >>> lower order digits.
> >>>
> >>> If we did that, then David's example could become:
> >>>
> >>>  >>> print(f"In European format x is {x:,.2f}, in Indian format it
> >>> is {x:,2,3.2f}")
> >>
> >>
> >> This just seems too complicated to me, and is overgeneralizing. How
> many of
> >> these different formats would ever really be used? Can you really expect
> >> someone to remember what that means by looking at it?
> >
> > Sure - "," and "_" both mean "digit grouping", the numbers tell you
> > how large the groups are from left to right (with the leftmost group
> > size repeated as needed), and a single "," means the same thing as
> > ",3," for decimal digits, and the same thing as ",4," for binary,
> > octal, and hexadecimal digits.
>
> Slight correction here, since the comma-separator is decimal only:
>
> - "," would be short for ",3," with decimal digits
> - "_" would be short for "_3_" with decimal digits
> - "_" would be short for "_4_" with binary/octal/hexadecimal digits
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Format mini-language for lakh and crore

[Stephan Houben]

Hi David,

Perhaps the "n" locale-dependent number formatting specifier
should accept a , to have locale-appropriate formatting of thousand
separators?

f"{x:,n}"

would Do The Right Thing(TM) depending on the locale.

Today it is an error.

Stephan

2018-01-28 7:25 GMT+01:00 David Mertz :

> In South Asia, a different style of digit delimiters for large numbers is
> used than in Europe, North America, Australia, etc.  With some minor
> spelling differences, the term lakh is used for a hundred-thousand, and it
> is generally written as '1,00,000'.
>
> In turn, a crore is 100 lakh, and is written as '1,00,00,000'.  Extending
> this pattern, larger numbers continue to use two digits in groups (other
> than the smallest grouping of three digits.  So, e.g. 1e12 is written
> as 10,00,00,00,00,000.
>
> It's nice that we now have the optional underscore in numeric literals.
> So we could write a number as either `12_34_56_78_00_000` or
> `1_234_567_800_000` depending on what region of the world and which
> convention was more familiar.
>
> However, in *formatting* those numbers, the format mini-language only
> allows the European convention.  So e.g.
>
> In [1]: x = 12_34_56_78_00_000
> In [2]: "{:,d}".format(x)
> Out[2]: '1,234,567,800,000'
> In [3]: f"{x:,d}"
> Out[3]: '1,234,567,800,000'
>
>
> In order to get Indian number delimiters, you'd have to write a custom
> formatting function, notwithstanding that something like 1.5 billion people
> use the three-then-two delimiting convention.
>
> I propose that Python should have an additional grouping option, or some
> other way to specify this grouping convention.  Oddly, the '_' grouping
> symbol is available, even though no one actually uses that grouper outside
> of programming languages like Python, e.g.:
>
> In [4]: f"{x:_d}"
> Out[4]: '1_234_567_800_000'
>
>
> I guess this is nice for something like round-tripping numbers used in
> code, but it's not a symbol anyone uses "natively" (I understand why comma
> or period cannot be used in numeric literals since they mean something else
> in Python already).
>
> I'm not sure what symbol or combination I would recommend, but finding
> something suitable shouldn't be so hard.  Perhaps now that backtick no
> longer has any other meaning in Python, it could be used since it looks
> similar to a comma.  E.g. in Python 3.8 we might have:
>
> >>> f"{x:`d}"
> '12,34,56,78,00,000'
>
> (actually, this probably isn't any parser issue even in Python 2 since
> it's already inside quotes; but the issue is moot).
>
> Or maybe a two character version like:
>
> >>> f"{x:2,d}"
> '12,34,56,78,00,000'
>
>
> Or:
>
> >>> f"{x:,,d}"
> '12,34,56,78,00,000'
>
>
> Even if `2,` was used, that wouldn't preclude giving an additional length
> descriptor after it.  Now we can have:
>
> >>> f"{x:,.2f}"
>
> '1,234,567,800,000.00'
>
> Perhaps in the future this would work:
>
> >>> f"{x:2,.2f}"
> '12,34,56,78,00,000.00'
>
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why CPython is still behind in performance for some widely used patterns ?

[Stephan Houben]

Hi all,

I would like to remark that, in my opinion, the question of CPython's
performance cannot
be decoupled from the extremely wide selection of packages which provide
optimized
code for almost any imaginable task.

For example: Javascript may be faster than (C)Python on simple benchmarks,
but
as soon as the task is somewhat amenable to scypi, and I can use scipy in
Python,
the resulting performance will completely cream Javascript in a way that
isn't funny anymore.
And scipy is just an example, there are tons of such libraries for all
kinds of tasks

I am not aware of any language ecosystem with a similar wide scope of
packages;
at least Java and Node both fall short. (Node may have more packages by
number
but the quality is definitely less and there is tons of overlap).

Stephan

2018-01-27 7:42 GMT+01:00 Nick Coghlan :

> On 27 January 2018 at 07:35, Pau Freixes  wrote:
>
>> This could be considered an unimportant thing, but its more relevant
>> than someone could expect, at least IMHO. If the default code that you
>> can write in a language is by default slow and exists an alternative
>> to make it faster, this language is doing something wrong.
>>
>
> Not really, as we've seen with the relatively slow adoption of PyPy over
> the past several years.
>
> CPython, as an implementation, emphasises C/C++ compatibility, and
> internal interpreter simplicity. That comes at a definite cost in runtime
> performance (especially where attribute access and function calls are
> concerned), but has also enabled an enormous orchestration ecosystem,
> originally around C/C++/FORTRAN components, but now increasingly around
> Rust components within the same process, as well as out-of-process Java,
> C#, and JavaScript components. In this usage model, if Python code becomes
> the throughput bottleneck, it's only because something has gone wrong at
> the system architecture level.
>
> PyPy, by contrast, emphasises raw speed, sacrificing various aspects of
> CPython's C/C++ interoperability in order to attain it. It's absolutely the
> implementation you want to be using if your main concern is the performance
> of your Python code in general, and there aren't any obvious hotspots that
> could be more selectively accelerated.
>
> To date, the CPython model of "Use (C)Python to figure out what kind of
> problem you have, then rewrite your performance bottlenecks in a language
> more specifically tailored to that problem space" has proven relatively
> popular. There's likely still more we can do within CPython to make typical
> code faster without increasing the interpreter complexity too much (e.g.
> Yury's idea of introducing an implicit per-opcode result cache into the
> eval loop), but opt-in solutions that explicit give up some of Python's
> language level dynamism are always going to be able to do less work at
> runtime than typical Python code does.
>
> Cheers,
> Nick.
>
> P.S. You may find https://www.curiousefficiency.
> org/posts/2015/10/languages-to-improve-your-python.html#
> broadening-our-horizons interesting in the context of considering some of
> the many factors other than raw speed that may influence people's choice of
> programming language. Similarly, https://www.curiousefficiency.
> org/posts/2017/10/considering-pythons-target-audience.html provides some
> additional info on the scope of Python's use cases (for the vast majority
> of which, "How many requests per second can I serve in a naive loop in a
> CPU bound process?" isn't a particularly relevant characteristic)
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Support WHATWG versions of legacy encodings

[Stephan Houben]

Op 11 jan. 2018 10:56 schreef "Serhiy Storchaka" :

09.01.18 23:15, Rob Speer пише:

>
>
> For the sake of discussion, let's call this encoding "web-1252". WHATWG
> calls it "windows-1252",


I'd suggest to name it then
"whatwg-windows-152".

and in general

"whatwg-" + whatgwgs_name_of_encoding

Stephan



but

notice that it's subtly different from Python's "windows-1252" encoding..
> Python's windows-1252 has bytes that are undefined:
>
>
>  >>> b'\x90'.decode('windows-1252')
> UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 0:
> character maps to 
>
> In web-1252, the bytes that are undefined according to windows-1252 map to
> the control characters in those positions in iso-8859-1 -- that is, the
> Unicode codepoints with the same number as the byte. In web-1252, b'\x90'
> would decode as '\u0090'.
>
> This may seem like a silly encoding that encourages doing horrible things
> with text. That's pretty much the case. But there's a reason every Web
> browser implements it:
>
> - It's compatible with windows-1252
> - Any sequence of bytes can be round-tripped through it without losing
> information
>
> It's not just this one encoding. WHATWG's encoding standard (
> https://encoding.spec.whatwg.org/ )
> contains modified versions of windows-1250 through windows-1258 and
> windows-874.
>

The way of solving this issue in Python is using an error handler. The
"surrogateescape" error handler is specially designed for lossless
reversible decoding. It maps every unassigned byte in the range 0x80-0xff
to a single character in the range U+dc80-U+dcff. This allows you to
distinguish correctly decoded characters from the escaped bytes, perform
character by character processing of the decoded text, and encode the
result back with the same encoding.

>>> b'\x90\x91\x92\x93'.decode('windows-1252', 'surrogateescape')
'\udc90‘’“'
>>> '\udc90‘’“'.encode('windows-1252', 'surrogateescape')
b'\x90\x91\x92\x93'

If you want to map unassigned bytes to other characters, you should just
create a new error handler. There are caveats, since such characters are
not distinguished from correctly decoded characters.

The same problem with the UTF-8 encoding. WHATWG allows encoding and
decoding surrogate characters in the range U+d800-U+dcff. This is contrary
to the Unicode Standard and raises an error by default in Python. But you
can allow encoding and decoding of surrogate characters by explicitly
specifying the "surrogatepass" error handler.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add an UML class diagram to the collections.abc module documentation

[Stephan Houben]

Hi Yahya,

I like the full.png diagram, however, I see some issues with it.

Most seriously, the methods it lists don't match the documentation.

E.g. if you check MappingView:

https://docs.python.org/3/library/collections.abc.html#collections.abc.MappingView

you see it has only a __len__ mixin method.
The other methods in the diagram are implementation details
and should be removed.

Some presentation points (all IMHO of course):
* Get rid of the empty boxes.
* Get rid of the trailing (). Since all methods have this, it adds no info.
* There is no visual distinction between the abstract methods
  and the mixin methods. I'd suggest making the abstract methods italic
  or something like that.

Stephan



2017-12-30 17:11 GMT+01:00 Yahya Abou 'Imran via Python-ideas <
python-ideas@python.org>:

> We can find very usefull class diagramm to understand the hierarchy of the
> builtin Collection abstract class and interface in java.
>
> Some examples:
> http://www.falkhausen.de/Java-8/java.util/Collection-Hierarchy-simple.html
> http://www.falkhausen.de/Java-8/java.util/Collection-List.html
>
> But when I search about python's ABC, The more detailed I can find are
> those from the book of Luciano Ramalho Fluent Python:
> https://goo.gl/images/8JGjvM
> https://goo.gl/images/6xZqcA
>
> (I think they're done with pyreverse of pylint)
>
> They are fine, but I think we could provide some other more detailed in
> this page:
> https://docs.python.org/3/library/collections.abc.html
>
> The table could be difficult to understand, a diagram help visualize
> things.
>
> I've began working on it with plantuml and pyreverse, I'm joining to this
> mail what I've done so far so you can tell me what you think.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Immutable dictionaries

[Stephan Houben]

2017-11-30 2:16 GMT+01:00 Rob Cliffe :

> This is sort of a subset of an idea I once posted on Python-Ideas:
> Dictionaries, sets and lists (etc. ?) could have a mutable flag, and once
> it was set to False you could neither change it back nor change the
> object.  (OK there might be some backdoor hacks, this is Python.)  This
> would avoid having to explain to beginners "Why does Python have lists and
> tuples?" because instead of a tuple, all you need is an immutable list.
> (Or if you prefer, instead of a list, all you need is a mutable tuple.)
>


Here is a concept implementation of a freeze() function which does
something similar.
Rather than mutating the mutable object to become immutable, it
makes an immutable copy.

>>> freeze({"x": [1,2,{3,4}]})
mappingproxy({'x': (1, 2, frozenset({3, 4}))})

https://gist.github.com/stephanh42/d277170dd8a3a2f026c272a4fda15396

Stephan


> Rob Cliffe
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Stephan Houben]

Op 29 nov. 2017 22:35 schreef "Greg Ewing" :


It would read better with some kind of pronoun in there:

   A if it is not None else C

Hypercard's Hypertalk had a special variable "it" that
worked sort of like that.


I  considered that,  but there are two issues.

1. Backward-incompatible change

2. The semantics of
A if B else C
now depends on if B contains 'it',
in which case A gets evaluated unconditionally and prior to B.
What if evaluation of 'it' is itself conditional, e.g.

print("hello") if a or it else print("goodbye")

Note that in the

A if  B else C

proposal the evaluation of the implicit 'it' is never conditional in that
way.

Stephan



-- 
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Stephan Houben]

What about a more general:

A if  B else C

which would allow

A if is not None else C

but also e.g.

A if >= 1 else 0

Stephan

Op 29 nov. 2017 13:41 schreef "Nick Coghlan" :

> On 29 November 2017 at 22:38, Stephan Houben  wrote:
> > What about more English-like syntax:
> >
> > X or else Y
>
> The problem with constructs like this is that they look like they
> should mean the same thing as "X or Y".
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] [Python-Dev] What's the status of PEP 505: None-aware operators?

[Stephan Houben]

What about more English-like syntax:

X or else Y

E.g.

cache.get(foo) or else expensive_call(foo)

Stephan


Op 29 nov. 2017 12:54 schreef "Serhiy Storchaka" :

29.11.17 11:45, Steven D'Aprano пише:

On Wed, Nov 29, 2017 at 09:14:12AM +0200, Serhiy Storchaka wrote:
>
>> 29.11.17 08:08, Steven D'Aprano пише:
>>
>>> Perl is hardly the only language with null-coalescing operators -- we
>>> might better describe ?? as being familiar to C#, PHP, Swift and Dart.
>>> That's two mature, well-known languages and two up-and-coming languages.
>>>
>>
>> What is the syntax of the ternary operator in these languages?
>>
>
> All four use:
>
>  condition ? first : second
>
> for the ternary if operator.
>

If all four use ?, it is natural that in operators which are shortcuts of
the ternary operator they use ?. But in Python the bar of introducing ? is
higher.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

US English is too broad.

I propose everybody talk like in the "Dallas" TV series
and wear  mandatory cowboy hats.

(Or was this mail just a dream?)

Stephan

2017-11-23 15:29 GMT+01:00 Carl Smith :

> Can't we just tell everyone to speak US English, and go back to ASCII? It
> would be a less painful migration.
>
> -- Carl Smith
> carl.in...@gmail.com
>
> On 23 November 2017 at 14:16, Chris Angelico  wrote:
>
>> On Fri, Nov 24, 2017 at 1:10 AM, Mikhail V  wrote:
>> > Well, then there is some bitter irony in this, so it allows pretty
>> > much everything,
>> > but does not allow me to beautify code with hyphens.
>> > I can fully understand the wish to use non-latin scripts in strings or
>> comments.
>> > As for identifiers, IMO, apart from latin letters and underscore, the
>> > first unicode candidate
>> > I would add is U+2010. And probably the LAST one I would add.
>> >
>>
>> Fortunately for the world, you're not the one who decided which
>> characters were permitted in Python identifiers. The ability to use
>> non-English words for function/variable names is of huge value; the
>> ability to use a hyphen is of some value, but not nearly as much.
>>
>> Can this thread move to python-list? Or, better, to
>> python-rants-about-unicode-list, to which I don't subscribe?
>>
>> ChrisA
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Adding a thin wrapper class around the functions in stdlib.heapq

[Stephan Houben]

I thought the purpose of heapq was to have a ready-made example
for instructors on how an API can be improved by applying object-oriented
techniques. ;-)

I think adding a HeapQueue class would be a great idea. Obviously the
existing functions
would need to continue existing for backward compatibility.

Stephan

2017-11-22 12:09 GMT+01:00 Antoine Pitrou :

> On Wed, 22 Nov 2017 00:22:00 -0600
> Nick Timkovich 
> wrote:
> >
> > Functions are great. I'm a big fan of functions. However,
> >
> > 1. Once there are several that all have the same thing as an argument:
> > thing_operation1(thing, arg), thing_operation2(thing, arg)...it's about
> > time to bind them together.
> > 2. And especially for the heap "soft-datatype": once it's heapified,
> > naively modifying it with other methods will ruin the heap invariant.
> **The
> > actual list you pass around can't be treated as a list.**
>
> A third reason: documentation and discoverability.  If I type
> help(some_heapified_list) at the prompt, I get the usual documentation
> for list methods, not the documentation of heapq functions...
>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

Hi all,

If anybody is still worried about this,
here is a 29-line proof-of-concept code checker which warns if your source
file
contains identifiers which are different but look the same.

https://gist.github.com/stephanh42/61eceadc2890cf1b53ada5e48ef98ad1

Stephan

2017-11-21 19:19 GMT+01:00 Mikhail V :

> Serhiy Storchaka wrote:
>
> > Yes, it causes less confusion that changing meaning of a minus.
>
> If those chars are not used at all, then yes :)
> And I don't recall I was exactly propsing changing meaning of minus
>
>
> > But the name моязмінна doesn't cause any confusion if used in an
> > appropriate context (for example in a lesson for young Ukrainian
> > children). I believe the above dot- and hyphen-like characters don't
> > cause confusion if used as letters in an appropriate language context.
>
> A single word written in local language should not. But its a perfect way
> to make whole code look like a mess.
> I think it is very interesting experience to use Cyrillic letters, since
> many are identical to Latin. So it would not be programming lessons
> in the first place, but rather constant changing of keyboard layout,
> and then trying to find unexplainable errors.
>
>
>
> Mikhail
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

2017-11-21 12:55 GMT+01:00 Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp>:

> Personally, I think that Python probably should ban non-ASCII
> non-letter characters in identifiers and whitespace, and maybe add
> them later in response to requests from native speakers of the
> relevant languages.


That would be quite a backward-incompatible change since
such identifiers have been legal since Python 3.0.


> I don't know how easy that would be to do,
> though, since I think the rule is already that identifiers must be
> composed only of letters, numbers, and ASCII "_".


See:
https://www.python.org/dev/peps/pep-313

The identifier syntax is  *.

ID_Start is defined as all characters having one of the general categories
uppercase letters (Lu), lowercase letters (Ll), titlecase letters (Lt),
modifier letters (Lm), other letters (Lo), letter numbers (Nl), the
underscore, and characters carrying the Other_ID_Start property. XID_Start
then closes this set under normalization, by removing all characters whose
NFKC normalization is not of the form ID_Start ID_Continue* anymore.

ID_Continue is defined as all characters in ID_Start, plus nonspacing marks
(Mn), spacing combining marks (Mc), decimal number (Nd), connector
punctuations (Pc), and characters carryig the Other_ID_Continue property.
Again, XID_Continue closes this set under NFKC-normalization; it also adds
U+00B7 to support Catalan.

Since Serhiy's
> examples are valid, we'd have to rule them out explicitly, rather than
> by reference to the Unicode database.  Yuck.
>

If we take this thinking to its logical extreme we should ban ASCII 1 and l
since they can be confused. Also 0 and O.

Realistically, this is extremely unlikely to be an issue in practice.
If you have people making such malignant code changes
with checkin permission, you have bigger problems...

Anyway, you can have your linter enforce ASCII or whatever
character subset you deem safe.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

You think that's bad?
https://github.com/reinderien/mimic/blob/master/README.md

Abandon all hope ye who use Unicode.

Op 19 nov. 2017 12:06 schreef "Antoine Pitrou" :

> On Sun, 19 Nov 2017 09:38:17 +0200
> Serhiy Storchaka 
> wrote:
> > 19.11.17 04:01, Mikhail V пише:
> > > Python allows underscore character as a separator in variables.
> > > This is better than nothing, still it does not make the look much
> better.
> > >
> > > **Proposal**: allow additional separator, namely hyphen character.
> >
> > You already can use "separators" different from the underscore.
> >
> > my·variable
> > myᝍvariable
> > myㅡvariable
> > myⵧvariable
> > myㄧvariable
> > myㆍvariable
>
> This is going to make for some fun code reviews.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

There is an unfortunate ambiguity in using a character that means "not" as
a word separator:

  nuke.do¬launch()

"But... I called the method which explicitly did *not* launch the nuke!"

Stephan

Op 19 nov. 2017 11:05 schreef "Steve Barnes" :



On 19/11/2017 05:01, Nick Timkovich wrote:
> Python does not use U+2010 HYPHEN for the minus operator, it uses the
> U+002D (-) HYPHEN-MINUS.
>
> In some monospace fonts, there is a subtle difference between U+002D,
> U+2013 EN DASH, and U+2014 EM DASH, but it's usually hard to tell them
> *all* apart.
>
> If you want to make a proposal, I'd suggest that you limit it to
> allowing the U+2010 HYPHEN to be used for names. U+002D simply cannot be
> changed because it would break billions of lines of code.
>
> On Sat, Nov 18, 2017 at 10:44 PM, Mikhail V  > wrote:
>
> On Sun, Nov 19, 2017 at 3:42 AM, Nick Coghlan  > wrote:
>
> > For anyone tempted to suggest "What about multiple underscores
> > indicating continuation of the variable name?", that's still a
> > compatibility problem due to the unary minus operator:
> >
> > >>> my--variable
> > 2
> > >>> my---variable
> > 0
>
> That seems to be another showcase of misfotune that Python
> uses hyphen for minus operator. I know it is not language designer's
> fault, because basic ASCII simply did not not include minus character.
> But do you realise that the **current** problem you are adressing is
> that
> font designers forgot to make the minus character (in monospaced font)
> distinctive from the hyphen character?
>
> Well, what can I say, I just think it should be a reason to make a
> collective complain to font providers, but not that you should
silently
> accept this and adopt the language design to someone's sloppy font
> design.
>
> As an aid for monospace die-hards, to minimise the confusion one could
> publish a style-guide that recommends to disclose the minus operator
> (currently
> hyphen char)  in spaces, like a - b, and probably disallow the new
> proposed
> hyphen character in the beginning of the identifiers.
> That would still leave potential for confusion because you cant'
> force everyone
> to follow style-guides, but one should struggle to break from this
> cycle anyway.
>
> >
> > Would hyphens in variable names improve readability sometimes?
>
> For reading code, indeed, always and very much. Of course not in case
> I would be forced
> to use monospaced font with a similar minus and hyphen. But
> in that case I am already accepting the level of readability of
> 12th century, so this would not make things much worse, and I
> would simply put spaces around the minus operator and try to highlight
> it with some strong color.
>
How about allowing ¬, (ASCII 172, U+00ac, NOT sign), in variable names
as in my¬variable - it has the advantages that:

  - it is visually distinguishable even in mono-spaced fonts,
(personally I use mono-spaced all of the time when programming but I
know that I am a dinosaur),
  - is actually on many keyboards as a single character, (I don't know
of any which actually produce different characters for minus on the
numeric keypad and hyphen elsewhere), so can be typed as a single key press,
  - Is generally unused AFAIK other than in papers about logic,
  - It is currently unused in the Python language.

This might upset some who would like use it to replace the unary not
operator but I suspect that it would be far fewer people than the
potential breakages discussed so far.

--
Steve (Gadget) Barnes
Any opinions in this message are my personal opinions and do not reflect
those of my employer.

---
This email has been checked for viruses by AVG.
http://www.avg.com

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Allow additional separator character in variables

[Stephan Houben]

Note that Python already allows Unicode characters from class "connector
punctuations (Pc)" in identifiers.

No dashes in that, but if you are into that kind of thing,
then this⁀is⁀a⁀valid⁀identifier .

Stephan

Op 19 nov. 2017 08:13 schreef "Bruce Leban" :

>
> On Sat, Nov 18, 2017 at 8:44 PM, Mikhail V  wrote:
>>
>>
>> That seems to be another showcase of misfotune that Python
>> uses hyphen for minus operator. I know it is not language designer's
>> fault, because basic ASCII simply did not not include minus character.
>> But do you realise that the **current** problem you are adressing is that
>> font designers forgot to make the minus character (in monospaced font)
>> distinctive from the hyphen character?
>
>
> It is not a misfortune or even true that Python uses hyphen for minus.
> The name of the character used in Python is HYPHEN-MINUS.
> http://unicode.org/cldr/utility/character.jsp?a=002D
> It is both a hyphen and a minus. And it served double-duty even in ASCII.
>
> A language that requires using characters not present on standard
> keyboards is unlikely to be successful.
> Or we would all be programming in APL.
>
> And it's not as if no one every thought of this before. Maybe you've heard
> of COBOL?
>
>
>>
>>
> > Would hyphens in variable names improve readability sometimes?
>>
>> For reading code, indeed, always and very much.
>
>
> No it wouldn't. You're personal preference is hardly authoritative. I am
> extremely skeptical that a legitimate usability study would find that
> record-count is better than record_count.
>
> There are studies that monospace fonts are harder to read than
> proportionally spaced, e.g., http://journals.sagepub.com/doi/pdf/10.1177/
> 001872088302500303. Yet many programmers use monospace fonts because the
> advantages -- in our opinions -- outweigh the disadvantages. And the
> reality is that only my opinion matters when I'm choosing the fonts to
> display my code in, not yours.
>
> You-know-what-really-would-increase-readability?
> Allowing-the-use-of-spaces-in-variable-names. As-you-can-see-from-this-
> example-hyphens-between-words-decreases-readability.
>
> And because spaces between words is mostly not valid syntax currently,
> this change would be easier to introduce than breaking every single program
> out there by re-purposing hyphen-minus. But I'm not seriously proposing
> this because I think the modest benefits are outweighed by the many
> problems it would introduce.
>
> --- Bruce
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Ignorable whitespaces in the re.VERBOSE mode

[Stephan Houben]

I put the actual space characters here so you can see them
in a non-proportional font (which I assume most Python programmer use).

https://gist.github.com/stephanh42/7c1c122154fd3f26d864233a40d8

The control characters aren't rendered at all (Vim renders them as ^\ ^] ^^
^_,
respectively). Most of the other spaces are rendered exactly like the
normal space.

The only ones which render differently are
U+1680 | | OGHAM SPACE MARK
U+3000 | | IDEOGRAPHIC SPACE

I understand Ogham has recently (since 6th century CE) seen a decline in
popularity.

However, I think Python should totally adopt U+3000 as a new whitespace
character
and start promoting it as the One True Way to indent code,
so as to finally end the age-old spaces vs tabs conflict.

[That was supposed to be a joke.]

Stephan



2017-11-17 16:38 GMT+01:00 Victor Stinner :

> I don't think that we need more than space (U+0020) and Unix newline
> (U+000A) ;-)
>
> Victor
>
> 2017-11-16 11:23 GMT+01:00 Serhiy Storchaka :
> > Currently the re module ignores only 6 ASCII whitespaces in the
> re.VERBOSE
> > mode:
> >
> >  U+0009 CHARACTER TABULATION
> >  U+000A LINE FEED
> >  U+000B LINE TABULATION
> >  U+000C FORM FEED
> >  U+000D CARRIAGE RETURN
> >  U+0020 SPACE
> >
> > Perl ignores characters that Unicode calls "Pattern White Space" in the
> /x
> > mode. It ignores additional 5 non-ASCII characters.
> >
> >  U+0085 NEXT LINE
> >  U+200E LEFT-TO-RIGHT MARK
> >  U+200F RIGHT-TO-LEFT MARK
> >  U+2028 LINE SEPARATOR
> >  U+2029 PARAGRAPH SEPARATOR
> >
> > The regex module just ignores characters for which str.isspace() returns
> > True. It ignores additional 20 non-ASCII whitespace characters, including
> > characters U+001C..001F whose classification as whitespaces is
> questionable,
> > but doesn't ignore LEFT-TO-RIGHT MARK and RIGHT-TO-LEFT MARK.
> >
> >  U+001C [FILE SEPARATOR]
> >  U+001D [GROUP SEPARATOR]
> >  U+001E [RECORD SEPARATOR]
> >  U+001F [UNIT SEPARATOR]
> >  U+00A0 NO-BREAK SPACE
> >  U+1680 OGHAM SPACE MARK
> >  U+2000 EN QUAD
> >  U+2001 EM QUAD
> >  U+2002 EN SPACE
> >  U+2003 EM SPACE
> >  U+2004 THREE-PER-EM SPACE
> >  U+2005 FOUR-PER-EM SPACE
> >  U+2006 SIX-PER-EM SPACE
> >  U+2007 FIGURE SPACE
> >  U+2008 PUNCTUATION SPACE
> >  U+2009 THIN SPACE
> >  U+200A HAIR SPACE
> >  U+202F NARROW NO-BREAK SPACE
> >  U+205F MEDIUM MATHEMATICAL SPACE
> >  U+3000 IDEOGRAPHIC SPACE
> >
> > Is it worth to extend the set of ignored whitespaces to "Pattern
> > Whitespaces"? Would it add any benefit? Or add confusion? Should this
> depend
> > on the re.ASCII mode? Should the byte b'\x85' be ignorable in verbose
> bytes
> > patterns?
> >
> > And there is a similar question about the Python parser. If Python uses
> > Unicode definition for identifier, shouldn't it accept non-ASCII "Pattern
> > Whitespaces" as whitespaces? There will be technical problems with
> > supporting this, but are there any benefits?
> >
> >
> > https://perldoc.perl.org/perlre.html
> > https://www.unicode.org/reports/tr31/tr31-4.html#Pattern_Syntax
> > https://unicode.org/L2/L2005/05012r-pattern.html
> >
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Nick,

2017-11-14 11:07 GMT+01:00 Nick Coghlan :

> On 14 November 2017 at 16:47, Michel Desmoulin 
> wrote:
>
>> Proposal A:
>> ---
>>
>> Suffix Python executable on Windows like on Unix, so that people will
>> type pythonX.X if they want a specify version.
>>
>> Pros: easy and discoverable.
>>
>> Cons: you need a lot of stuff in the system path.
>>
>
> Con: we hope to have the problem resolved on the Linux distro side such
> that "python" typically means "python" by the time community support for
> Python 2 ends in 2020. Since Windows has gone the better part of two
> decades without version Python commands, adding them because we're
> impatient with the pace of change at the Linux distro level doesn't really
> make sense (especially when Linux holds such a small fraction of the
> non-phone client device market).
>

Perhaps I could sell you on the idea of a Windows "python3" executable, not
as the New Official Way to do things,
but rather as a pragmatic measure to make code from those Linux weirdos
;-)  more likely to work on Windows.

I would like to compare it with the strings/bytes-for-pathnames issue: the
official recommendation is to use strings
everywhere, problems with round-tripping arbitrary not-valid-UTF8 filenames
on POSIX have been solved now.
Still, POSIX people continued to use bytes, so a pragmatic change was to
make so that bytes now also work
reliably as pathnames under Windows.

Similarly, even when all Linux distributions have switched to
python==python3, people will probably still read and write
tutorials with python3 in it, and perhaps we should accommodate that.

Otherwise I am +1 on your proposal C: if anything this thread has made it
clear that there is so much variety in
third-party Python installers, not just Linux distributions, but also
things like Anaconda, that it seems unreasonable
to require that the official Python documentation covers all those
situations.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] venv *is* provided in the standard Python install on Debian/Ubuntu

[Stephan Houben]

Hi Brett,

The current documentation *cannot* be fixed, since
fixing it would entail adding an initial two-page essay
on "how to start Python on various platforms/systems"
(it is really NOT as simple as Windows=python, Linux=python3)
and such a PR will certainly by rejected.

In my opinion, the only alternatives are

1. either harmonize the invocation of python across platforms
  (and *then* adapt the docs to follow).
  Which was pretty much the whole topic of this thread so far.

2. or just use "python" consistently across all docs
  (since that is the *only* command which is at least consistent among
   python.org installers), and add weasel-wording to "consult documentation
  of third-party installers"

3. or leave the docs broken for at least some people some of the time.

Stephan


2017-11-14 2:31 GMT+01:00 Brett Cannon :

>
>
> On Mon, Nov 13, 2017, 00:01 Stephan Houben,  wrote:
>
>> Hi all,
>>
>> Related to this text on https://docs.python.org/3/library/venv.html :
>>
>> 
>>
>>  Note
>>   The pyvenv script has been deprecated as of Python 3.6 in favor of
>>   using python3 -m venv to help prevent any potential confusion as to
>>   which Python interpreter a virtual environment will be based on.
>> 
>>
>> It's clearer than the text below to which I originally referred.
>>
>> However,  this text has also problems in that it is too unix-specific.
>> In particular:
>> * Most seriously, it refers to "python3" which doesn't work with the
>> python.org Windows installer.
>>
>
> It can, but it's opt-in. It's just one of those things that's easy to
> forget.
>
> * Less seriously, it refers to "pyenv" as a "script" which is unix jargon
>> and moreover technically
>>incorrect on Windows. (Also, needlessly specific, it should just be
>> "the pyenv command",
>>   how it is implemented is irrelevant for this section).
>>
>
> I disagree with this as Python refers to .Py files that you execute
> directly as "scripts", so I don't think this requires clarification.
>
>
> Anyway, a pull request with suggested wording to address your concerns
> would be the best way to try and rectify the issue.
>
> -brett
>
>
>
>> Stephan
>>
>> 2017-11-13 0:32 GMT+01:00 Chris Angelico :
>>
>>> On Mon, Nov 13, 2017 at 10:29 AM, Nick Coghlan 
>>> wrote:
>>> > On 13 November 2017 at 07:11, Chris Angelico  wrote:
>>> >> On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben 
>>> wrote:
>>> >>> Hi Antoine,
>>> >>>
>>> >>> The venv module is included,
>>> >>> however the pyvenv script is in a separate package
>>> >>> python3.5-venv .
>>> >>>
>>> >>> By the way, I was totally confused by the following text form the
>>> doc.
>>> >>>
>>> >>> https://docs.python.org/3/library/venv.html
>>> >>>
>>> >>> 
>>> >>> Deprecated since version 3.6: pyvenv was the recommended tool for
>>> creating
>>> >>> virtual environments for Python 3.3 and 3.4, and is deprecated in
>>> Python
>>> >>> 3.6.
>>> >>>
>>> >>> Changed in version 3.5: The use of venv is now recommended for
>>> creating
>>> >>> virtual environments.
>>> >>>
>>> >>> 
>>> >>
>>> >> Not sure where you're reading that. I'm seeing:
>>> >>
>>> >> """
>>> >> Note
>>> >> The pyvenv script has been deprecated as of Python 3.6 in favor of
>>> >> using python3 -m venv to help prevent any potential confusion as to
>>> >> which Python interpreter a virtual environment will be based on.
>>> >> """
>>> >>
>>> >> I think that's pretty clear. "python3 -m venv env" is the standard and
>>> >> recommended way to spin up a virtual environment.
>>> >
>>> > It's further down in the page, under
>>> > https://docs.python.org/3/library/venv.html#creating-
>>> virtual-environments
>>> >
>>> > I think the deprecation notice for pyvenv should just be deleted,
>>> > since it renders like the *module* is deprecated.
>>>
>>> Ah, I see it now, thanks.
>>>
>>> Agreed; or maybe downgrade it to a parenthetical comment. Focus on
>>> "this is how to do the obvious thing", and only as an afterthought
>>> mention "it used to be done differently" in case someone greps for
>>> pyvenv.
>>>
>>> ChrisA
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Chris,

+1 to all three of these proposals!

Stephan

2017-11-13 19:57 GMT+01:00 Chris Barker :

> This has gotten to be a big thread, and it's a pretty intractable problem,
> but I think there are a few fairly small things that could be done to at
> least make it a bit easier:
>
> 1) Add python2.exe and python3.exe files to the Windows installers -- am I
> insane or did Windows used to have that? I really think it did -- maybe got
> removed when py.exe was added.
>   1a) alternatively, we could add a "py" executable to the standard linux
> builds, so there would be THAT one way to do it. But I think that's a "BAD
> IDEA" -- the whole "py" thing is not widely know or used, it's not going to
> show up in package install instructions for a LONG time, (actualy we could
> do both anyway)
>
> Then "python2 -m pip install" would work everywhere (only with new
> installations, but at least with newbies, that's a bit more likely ...)
>
>
> 2) Make adding to the PATH in Windows the default. I think there are
> really three user groups:
>
>- newbies starting from scratch -- they want it on the PATH
>
>- newbies with whatever left over cruft from previous installations on
> their systems -- they want it at the FRONT of their PATH. They SHOULD
> uninstall all the cruft, but if they don't this will still work with as few
> surprises a possible.
>
>- not-newbies with a previous version of python they need to continue
> using. They can uncheck the box, or use py.exe
>
>
> 3) Make --user be be automatic for pip install. Not actually the default,
> but pip could do a user install if you don't have the permissions for a
> non-user install.
>
> This means folks might accidentally install in user mode because they
> forgot to type sudo -- but that would be a mostly-sysadmin/sophisticated
> user problem. And maybe have an environment variable of configuration key
> for  "prefer admin install". If tha was set, pip would only install in user
> mode if specifically asked to. I'm can't imagine a case where a user would
> have admin permissions, but want a user install (except people following
> bad practices!)
>
> Except for the pip change, these would be easy to implement and backward
> compatible. So why not?
>
>
> *NOTE:* even if nothing changes with any of this we need to get py.exe
> better documented and advertised -- it doesn't show up in:
>
> https://docs.python.org/3/faq/windows.html#id2
>
> for instance.
>
> In fact, I knew about py.exe (from this discussion), and was writing up
> notes about how to run a Python file (without access to a Windows box) ,
> and it took a LONG time to find ANY documentation of it (adding "py" to a
> google search about something python does not get far...).
>
> We can do that better, but frankly this may be a lesson on why we can't
> rely on anything "new" to help solve this problem, when maybe we could make
> the "old way" work better and more cross platform.
>
> -Chris
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R(206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115   (206) 526-6317   main reception
>
> chris.bar...@noaa.gov
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] venv *is* provided in the standard Python install on Debian/Ubuntu

[Stephan Houben]

Hi all,

Related to this text on https://docs.python.org/3/library/venv.html :


 Note
  The pyvenv script has been deprecated as of Python 3.6 in favor of
  using python3 -m venv to help prevent any potential confusion as to
  which Python interpreter a virtual environment will be based on.


It's clearer than the text below to which I originally referred.

However,  this text has also problems in that it is too unix-specific.
In particular:
* Most seriously, it refers to "python3" which doesn't work with the
python.org Windows installer.
* Less seriously, it refers to "pyenv" as a "script" which is unix jargon
and moreover technically
   incorrect on Windows. (Also, needlessly specific, it should just be "the
pyenv command",
  how it is implemented is irrelevant for this section).

Stephan

2017-11-13 0:32 GMT+01:00 Chris Angelico :

> On Mon, Nov 13, 2017 at 10:29 AM, Nick Coghlan  wrote:
> > On 13 November 2017 at 07:11, Chris Angelico  wrote:
> >> On Mon, Nov 13, 2017 at 6:24 AM, Stephan Houben 
> wrote:
> >>> Hi Antoine,
> >>>
> >>> The venv module is included,
> >>> however the pyvenv script is in a separate package
> >>> python3.5-venv .
> >>>
> >>> By the way, I was totally confused by the following text form the doc.
> >>>
> >>> https://docs.python.org/3/library/venv.html
> >>>
> >>> 
> >>> Deprecated since version 3.6: pyvenv was the recommended tool for
> creating
> >>> virtual environments for Python 3.3 and 3.4, and is deprecated in
> Python
> >>> 3.6.
> >>>
> >>> Changed in version 3.5: The use of venv is now recommended for creating
> >>> virtual environments.
> >>>
> >>> 
> >>
> >> Not sure where you're reading that. I'm seeing:
> >>
> >> """
> >> Note
> >> The pyvenv script has been deprecated as of Python 3.6 in favor of
> >> using python3 -m venv to help prevent any potential confusion as to
> >> which Python interpreter a virtual environment will be based on.
> >> """
> >>
> >> I think that's pretty clear. "python3 -m venv env" is the standard and
> >> recommended way to spin up a virtual environment.
> >
> > It's further down in the page, under
> > https://docs.python.org/3/library/venv.html#creating-
> virtual-environments
> >
> > I think the deprecation notice for pyvenv should just be deleted,
> > since it renders like the *module* is deprecated.
>
> Ah, I see it now, thanks.
>
> Agreed; or maybe downgrade it to a parenthetical comment. Focus on
> "this is how to do the obvious thing", and only as an afterthought
> mention "it used to be done differently" in case someone greps for
> pyvenv.
>
> ChrisA
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] venv *is* provided in the standard Python install on Debian/Ubuntu

[Stephan Houben]

2017-11-13 3:32 GMT+01:00 Nick Coghlan :

>
> So technically it's ensurepip that's broken by default, but that
> translates to venv also being broken by default.
>
> I haven't worked out what the actual steps needed to fix it are
>


On Debian, ensurepip  is in the python3.5-venv package.

https://packages.debian.org/stretch/amd64/python3.5-venv/filelist

Unfortunately there isn't (AFAIK) a meta-package which just says
"stop fooling around and just give me everything from the python.org
distribution, d*mmit."

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] venv *is* provided in the standard Python install on Debian/Ubuntu

[Stephan Houben]

Hi Antoine,

The venv module is included,
however the pyvenv script is in a separate package
python3.5-venv .

By the way, I was totally confused by the following text form the doc.

https://docs.python.org/3/library/venv.html


Deprecated since version 3.6: pyvenv was the recommended tool for creating
virtual environments for Python 3.3 and 3.4, and is deprecated in Python 3.6
.

Changed in version 3.5: The use of venv is now recommended for creating
virtual environments.


So many questions:
* What is the status of "pyenv" in 3.5? Apparently it is not deprecated
there.
* What is it replaced by? Apparently "venv", but it doesn't say so
explicitly.
* Is "venv" the same thing as "python -m venv" discussed earlier? Or is it a
  different thing? With so many things names so similarly, it is hard to
tell
* What does it mean for "venv" to be recommend in 3.5 if "pyvenv" is not
deprecated there?

Also, the link brings us to a long page of "What's New in Python 3.6" .
Just searching for "venv" only gives the apparently irrelevant:

"venv  accepts a
new parameter --prompt. This parameter provides an alternative prefix for
the virtual environment. (Proposed by Łukasz Balcerzak and ported to 3.6 by
Stéphane Wirtel in bpo-22829 .)
"

I suppose at that point the newbie gave up and downloaded node.js ;-)

Stephan

On Sun, 12 Nov 2017 12:20:45 +

> Paul Moore  wrote:
> >
> > > Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> > > pythonx -m venv ? Wait, it's not installed by default on debian.
> >
> > Seriously? Debian don't provide venv in the standard Python install?
> > That's just broken.
>
> Frankly, I don't know where the current discussion comes from, but on
> two recent Debian and Ubuntu setups, I get:
>
> $ dpkg -S /usr/lib/python3.5/venv/__init__.py
> libpython3.5-stdlib:amd64: /usr/lib/python3.5/venv/__init__.py
>
>
> Which, for the uninitiated, means "the venv module is provided by the
> Debian/Ubuntu package named libpython3.5-stdlib".  That package is, in
> turn, a dependency of the "python3.5" package.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

2017-11-12 13:20 GMT+01:00 Paul Moore :

> On 12 November 2017 at 06:19, Michel Desmoulin
>  wrote:
>
> > Well, not exactly. Do you do python -m venv, or py -x.x -m venv or
> > pythonx -m venv ? Wait, it's not installed by default on debian.
>
> Seriously? Debian don't provide venv in the standard Python install?
> That's just broken.
>

I think that it is wrong to think of Debian's "python3" package
as purporting to provide the "standard Python install".

This is really more properly though of as the Python *runtime*
environment, i.e. the minimum stuff you need to run programs
written in Python.

Developers are supposed to install additional packages.

Debian does the same for other languages,, i.e. their "node" package
doesn't contain npm.

I suppose it makes sense; a modern Linux desktop contains
applications written in Python/Perl/Node/Ruby/Tcl/OCaml/..., if
you start including all development tools in the base packages you
are probably ending doubling a typical install.

It's still annoying, I have personally decided that it is simpler
to just install the tarball from python.org than to chase all
the individual packages over which they split the Python install.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

One (smaller) suggestion on the PATH situation on Windows:

I noticed that Visual C++ Build Tools installs a number of  "Command
prompts"
under its Start menu item, each of which starts a cmd.exe with appropriate
PATH
set to the appropriate compiler (32/64 bits or ARM cross-compiler), and
assorted environment variables set to the appropriate include/library
directories.

Could we do something similar for Python?

I.e., Install under the "Python 3.6" start menu an additional
"Python command prompt", which will
start cmd.exe with an appropriate PATH so that python and pip
run without further prefix.

That way, the installer still doesn't need to mess with global PATH and you
can
easily have multiple versions of Python, each with their own
"Python command prompt" submenu.

At least for Windows users this would simplify the situation a bit.

Stephan



2017-11-06 23:53 GMT+01:00 Ivan Pozdeev via Python-ideas <
python-ideas@python.org>:

> On 07.11.2017 1:48, Chris Barker wrote:
>
> On Mon, Nov 6, 2017 at 9:52 AM, Michel Desmoulin <
> desmoulinmic...@gmail.com> wrote:
>
>> I know and you still:
>>
>> - have to use py -m on windows, python3 linux, python in virtualenv...
>>
>
> can't you use python3 -m pip install .
>
> everywhere?
>
> You can't. Windows versions don't create versioned executables. Got bitten
> with this myself.
>
>
> ...Maybe they should?
> (This is python-ideas, after all ;-) )
>
> --
> Regards,
> Ivan
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Stephan Houben]

2017-11-06 12:53 GMT+01:00 Brice Parent :

>
> I think the only problem we can reach here, not only in our lifetimes, but
> in the next years, is not Python3.10 vs Python31.0 (Python3.x will be long
> dead when we reach this point!), but the ordering of versions, like
> (python310 < python40). But it probably is a false problem, as after a
> two-digit minor version, we can fix the length of minor versions to two
> digits when needed (python310 < python400).
>

No probs with either of my proposals:

>>> "python39.dll" < "python3A.dll" < "python40.dll"
True
>>> "python39.dll" < "python3⑽.dll" < "python40.dll"
True

Stephan


>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Looking for input to help with the pip situation

[Stephan Houben]

Hi Michel,

That's exactly why I proposed a `pip` function available from the Python
prompt.
I suppose you could still tell your students to copy/paste the following
into their
Python interpreter.

def pip(args):
import sys
import subprocess
subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
print("Please restart Python now to use installed or upgraded
packages.")

I suppose an alternative is to set up jupyterhub

https://jupyterhub.readthedocs.io/en/latest/

and let all your students just access that from a webbrowser.

Stephan


2017-11-06 7:47 GMT+01:00 Michel Desmoulin :

> Hello,
>
> Today I'm going to give a training in Python again.
>
> And again it will go the same way.
>
> On Mac I will have to make people install python, then tell them to use
> pip3.
>
> On Windows, I will have to warn them about checking the "add python
> executable to system path" (that one of them will ALWAYS miss anyway).
> Then tell them to use py -3.x -m pip because some of them will have
> several versions of Python installed.
>
> Then on linux, I will tell them to install python-pip and python-venv
> and use python3 -m pip.
>
> I'll talk about --user, but commands won't be usable on some machine
> where the Scripts or bin dir is not in the system path.
>
> Then I will make them create a virtualenv so that they can avoid messing
> with their system python and finally can just use "pip install" like in
> most tutorials on the Web.
>
> And eventually I'll talk about pipenv and conda. The first one so they
> don't have to think about activating the virtualenv everytime, or pip
> freeze, or create the venv, or add it to gitignore, etc. The second
> because anaconda is very popular on windows.
>
> There is no way a beginner is going to get any that by themselves
> without a lot of time and pain. They will read some tutorial on the web
> and struggle to make sens of what pip is and why "pip install" doesn't
> work and why "python sucks".
>
> I think Python is offering an incredible experience for first timer.
> However, the whole "where is walpip" shenanigans is not one of them.
>
> I really want some people from this list to discuss here so we can find
> a way to either unify a bit the way we install and use pip, or find a
> way to express a tutorial that always works for people on the most
> popular platforms and spread the word so that any doc uses it.
>
> Michel
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Stephan Houben]

2017-11-05 21:02 GMT+01:00 Serhiy Storchaka :

> But len(os.fsencode("python3⑽.dll")) != len(os.fsencode("python39.dll")).
>
>
I think it is on Windows. os.fsencode should use UTF-16 there.
⑽ is in the BMP

Presumably, non-Windows platforms wouldn't be interested in .dll
files anyway...

Stephan


>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal to change Python version release cycle

[Stephan Houben]

After python39.dll I'd expect python3A.dll .

Or possibly python3⑽.dll

>>> len("python3⑽.dll") == len("python39.dll")
True

No worries, then.

Stephan

2017-11-05 20:28 GMT+01:00 Serhiy Storchaka :

> 04.11.17 17:29, Wolfgang пише:
>
>> A good point but two digits minor version numbers have the possibility
>> to break a lot code. There is a lot of stuff out where a single digit
>> major version is assumed. Even the official Python build for windows
>> with python27.dll, python36.dll can be problematic because the dot
>> is omitted between numbers.
>> Other do the same for compilation they concatenate only majorminor for a
>> name.
>> Then version 3.10 is the same as 31.0.
>>
>
> Actually this is yet one argument against your idea. With your proposal 27
> will be the same as 2.7.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

What about something like the following to simulate a "restart", portably.

def restart():
import sys
import os
import subprocess
if os.getenv("PYTHON_EXIT_ON_RESTART") == "1":
sys.exit(42)
else:
env = os.environ.copy()
env["PYTHON_EXIT_ON_RESTART"] = "1"
while True:
sp = subprocess.run([sys.executable], env=env)
if sp.returncode != 42:
sys.exit(sp.returncode)

Stephan

2017-10-30 17:33 GMT+01:00 Paul Moore :

> On 30 October 2017 at 16:22, Nick Coghlan  wrote:
> >> Also, on Windows, I believe that any emulation of execve either leaves
> >> the original process in memory, or has problems getting console
> >> inheritance right. It's been a long time since I worked at that level,
> >> and things may be better now, but getting a robust "restart this
> >> process" interface in Windows would need some care (that's one of the
> >> reasons the py launcher runs Python as a subprocess rather than doing
> >> any sort of exec equivalent).
> >
> > As long as the standard streams are passed along correctly, whatever the
> py
> > launcher does would presumably be adequate for a REPL restart as well,
> > assuming we decided to go down that path.
>
> The py launcher starts a subprocess for python.exe and waits on it. I
> wouldn't have thought that's going to work for installing mods in a
> REPL - imagine a long working session where I install 10 mods as I
> explore options for a particular problem (I don't know how likely that
> is in practice...) - there'd be a chain of 10+ Python processes, only
> the last of which is still useful. It's probably not a massive problem
> (I assume everything but the last process is paged out) but it's not
> exactly friendly.
>
> OTOH, if you lose the command history and the interpreter state after
> each install, you'll probably get fed up long before the number of
> processes is an issue...
>
> > It would also be reasonable to say that the regular REPL just issues a
> > warning that a restart might be needed, and it's only REPLs with a
> separate
> > client process that offer a way to restart the subprocess where code
> > actually executes.
>
> This feels awfully like the traditional Windows "your mouse has moved
> - please reboot to have your changes take effect" behaviour. I don't
> think we're going to impress many people emulating that :-(
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters :

> return “Please run pip from your system command prompt”
>
>
>

The target audience for my proposal are people who do not know
which part of the sheep the "system command prompt" is.

Stephan


>
>
>
>
> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@
> python.org] *On Behalf Of *Stephan Houben
> *Sent:* Sunday, October 29, 2017 3:19 PM
> *To:* Python-Ideas 
> *Subject:* [Python-ideas] install pip packages from Python prompt
>
>
>
> Hi all,
>
> Here is in somewhat more detail my earlier proposal for
>
> having in the interactive Python interpreter a `pip` function to
>
> install packages from Pypi.
>
> Motivation: it appears to me that there is a category of newbies
>
> for which "open a shell and do `pip whatever`" is a bit too much.
>
> It would, in my opinion, simplify things a bit if they could just
>
> copy-and-paste some text into the Python interpreter and have
>
> some packages from pip installed.
>
> That would simplify instructions on how to install package xyz,
>
> without going into the vagaries of how to open a shell on various
>
> platforms, and how to get to the right pip executable.
>
> I think this could be as simple as:
>
>   def pip(args):
>   import sys
>   import subprocess
>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>
>   print("Please re-start Python now to use installed or upgraded
> packages.")
>
> Note that I added the final message about restarting the interpreter
>
> as a low-tech solution to the problem of packages being already
>
> imported in the current Python session.
>
> I would imagine that the author of package xyz would then put on
>
> their webpage something like:
>
>   To use, enter in your Python interpreter:
>
>  pip("install xyz --user")
>
> As another example, consider prof. Baldwin from Woolamaloo university
>
> who teaches a course "Introductory Python programming for Sheep Shavers".
>
> In his course material, he instructs his students to execute the
>
> following line in their Python interpreter.
>
>pip("install woolamaloo-sheepshavers-goodies --user")
>
> which will install a package which will in turn, as dependencies,
>
> pull in a number of packages which are relevant for sheep shaving but
>
> which have nevertheless irresponsibly been left outside the stdlib.
>
> Stephan
>
>
>
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi Antoine,

2017-10-29 20:31 GMT+01:00 Antoine Rozo :

> Hi,
>
> What would be the difference with current pip module?
> pip.main(['install', 'some_package'])
>


My understanding is that direct use of the `pip` module is explicitly not
recommended.

Stephan




>
> 2017-10-29 20:26 GMT+01:00 Alex Walters :
>
>> I have a somewhat better, imo, implementation of a pip object to be
>> loaded into the repl.
>>
>>
>>
>> class pip:
>>
>> def __call__(self, *a, **kw):
>>
>> sys.stderr.write(str(self))
>>
>>
>>
>> def __repr__(self):
>>
>> return str(self)
>>
>>
>>
>> def __str__(self):
>>
>> return “Please run pip from your system command prompt”
>>
>>
>>
>>
>>
>>
>>
>> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=
>> sdamon@python.org] *On Behalf Of *Stephan Houben
>> *Sent:* Sunday, October 29, 2017 3:19 PM
>> *To:* Python-Ideas 
>> *Subject:* [Python-ideas] install pip packages from Python prompt
>>
>>
>>
>> Hi all,
>>
>> Here is in somewhat more detail my earlier proposal for
>>
>> having in the interactive Python interpreter a `pip` function to
>>
>> install packages from Pypi.
>>
>> Motivation: it appears to me that there is a category of newbies
>>
>> for which "open a shell and do `pip whatever`" is a bit too much.
>>
>> It would, in my opinion, simplify things a bit if they could just
>>
>> copy-and-paste some text into the Python interpreter and have
>>
>> some packages from pip installed.
>>
>> That would simplify instructions on how to install package xyz,
>>
>> without going into the vagaries of how to open a shell on various
>>
>> platforms, and how to get to the right pip executable.
>>
>> I think this could be as simple as:
>>
>>   def pip(args):
>>   import sys
>>   import subprocess
>>   subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
>>
>>   print("Please re-start Python now to use installed or upgraded
>> packages.")
>>
>> Note that I added the final message about restarting the interpreter
>>
>> as a low-tech solution to the problem of packages being already
>>
>> imported in the current Python session.
>>
>> I would imagine that the author of package xyz would then put on
>>
>> their webpage something like:
>>
>>   To use, enter in your Python interpreter:
>>
>>  pip("install xyz --user")
>>
>> As another example, consider prof. Baldwin from Woolamaloo university
>>
>> who teaches a course "Introductory Python programming for Sheep Shavers".
>>
>> In his course material, he instructs his students to execute the
>>
>> following line in their Python interpreter.
>>
>>pip("install woolamaloo-sheepshavers-goodies --user")
>>
>> which will install a package which will in turn, as dependencies,
>>
>> pull in a number of packages which are relevant for sheep shaving but
>>
>> which have nevertheless irresponsibly been left outside the stdlib.
>>
>> Stephan
>>
>>
>>
>>
>>
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>
>
> --
> Antoine Rozo
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] install pip packages from Python prompt

[Stephan Houben]

Hi all,

Here is in somewhat more detail my earlier proposal for
having in the interactive Python interpreter a `pip` function to
install packages from Pypi.

Motivation: it appears to me that there is a category of newbies
for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just
copy-and-paste some text into the Python interpreter and have
some packages from pip installed.
That would simplify instructions on how to install package xyz,
without going into the vagaries of how to open a shell on various
platforms, and how to get to the right pip executable.

I think this could be as simple as:

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())
  print("Please re-start Python now to use installed or upgraded
packages.")

Note that I added the final message about restarting the interpreter
as a low-tech solution to the problem of packages being already
imported in the current Python session.

I would imagine that the author of package xyz would then put on
their webpage something like:

  To use, enter in your Python interpreter:
 pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university
who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the
following line in their Python interpreter.

   pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,
pull in a number of packages which are relevant for sheep shaving but
which have nevertheless irresponsibly been left outside the stdlib.

Stephan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Stephan Houben]

Perhaps slightly off-topic, but I have sometimes wondered if
pip could not be made somewhat friendlier for the absolute newbie
and the classroom context.

Some concrete proposals.

1. Add a function `pip` to the interactive interpreter
  (similar to how `help` is available).

  def pip(args):
  import sys
  import subprocess
  subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

   This allows people to install something using pip as long as they have a
   Python prompt open, and avoids instructors to have to deal with
platform-specific
   instructions for various shells. Also avoids confusion when multiple
Python interpreters
   are available (it operates in the context of the current interpreter.)

2. Add to Pypi package webpages a line like:

To install, execute this line in your Python interpreter:
 pip("install my-package --user")

  Make this copyable with a button (like Github allows you to copy the repo
name).

3. Add the notion of a "channel" to which one can "subscribe".

   This is actually nothing new, just a Pypi package with no code and just
a bunch of
   dependencies. But it allows me to say: Just enter:

  pip("install stephans-awesome-stuff --user")

  in your Python and you get a bunch of useful stuff.

Stephan

2017-10-29 8:54 GMT+01:00 Nick Coghlan :

> On 29 October 2017 at 15:16, Guido van Rossum  wrote:
>
>> Why? What's wrong with pip install?
>>
>
> At a technical level, this would just be a really thin wrapper around 'pip
> install' (even thinner than ensurepip in general, since these libraries
> *wouldn't* be bundled for offline installation, only listed by name).
>
>
>> Why complicate things? Your motivation is really weak here. "beneficial"?
>> "difficult cases"?
>>
>
> The main recurring problems with "pip install" are a lack of
> discoverability and a potential lack of availability (depending on the
> environment).
>
> This then causes a couple of key undesirable outcomes:
>
> - folks using Python as a teaching language have to choose between
> teaching with just the standard library APIs, requiring that learners
> restrict themselves to a particular preconfigured learning environment, or
> make a detour into package management tools in order to ensure learners
> have access to the APIs they actually want to use (this isn't hypothetical
> - I was a technical reviewer for a book that justified teaching XML-RPC
> over HTTPS+JSON on the basis that xmlrpc was in the standard library, and
> requests wasn't)
> - folks using Python purely as a scripting language (i.e without app level
> dependency management) may end up having to restrict themselves to the
> standard library API, even when there's a well-established frequently
> preferred alternative for what they're doing (e.g. requests for API
> management, regex for enhanced regular expressions)
>
> The underlying problem is that our reasons for omitting these particular
> libraries from the standard library relate mainly to publisher side
> concerns like the logistics of ongoing bug fixing and support, *not* end
> user concerns like software reliability or API usability. This means that
> if educators aren't teaching them, or redistributors aren't providing them,
> then they're actively doing their users a disservice (as opposed to other
> cases like web frameworks and similar, where there are multiple competing
> options, you're only going to want one of them in any given application,
> and the relevant trade-offs between the available options depend greatly on
> exactly what you're doing)
>
> Now, the Python-for-data-science community have taken a particular
> direction around handling this, and there's an additional library set
> beyond the standard library that's pretty much taken for granted in a data
> science context. While conda has been the focal point for those efforts
> more recently, it started a long time ago with initiatives like Python(x,
> y) and the Enthought Python Distribution.
>
> Similarly, initiatives like Raspberry Pi are able to assume a particular
> learning environment (Raspbian in the Pi's case), rather than coping with
> arbitrary starting points.
>
> Curated lists like the "awesome-python" one that Stephan linked don't
> really help that much with the discoverability problem, since they become
> just another thing for people to learn: How do they find out such lists
> exist in the first place? Given such a list, how do they determine if the
> recommendations it offers are actually relevant to their needs? Since
> assessing a published package API against your needs as a user is a skill
> that has to be learned like any other, it can be a lot easier to get
> started in a more prescriptive environment that says "This is what you have
> to work with for now, we'll explain more about your options for branching
> out later".
>
> The proposal in this thread thus stems from asking the question "Who is
> going to be best positioned to offer authoritative advice on which

Re: [Python-ideas] Defining an easily installable "Recommended baseline package set"

[Stephan Houben]

There are already curated lists of Python packages, such as:

https://github.com/vinta/awesome-python

Even better, if you don't agree, you can just clone it and create your own
;-)

Stephan

2017-10-29 6:46 GMT+01:00 Alex Walters :

> At this point, I would punt to distutils-sig about curated packages, and
> pip tooling to support that, but they are bogged down as it stands with
> just getting warehouse up and running.  I don’t support putting specialized
> tooling in python itself to install curated packages, because that curation
> would be on the same release schedule as python itself.  Pip is an
> important special case, but it’s a special case to avoid further special
> cases.  If there was excess manpower on the packaging side, that’s where it
> should be developed.
>
>
>
> *From:* Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@
> python.org] *On Behalf Of *Guido van Rossum
> *Sent:* Sunday, October 29, 2017 1:17 AM
> *To:* Nick Coghlan 
> *Cc:* python-ideas@python.org
> *Subject:* Re: [Python-ideas] Defining an easily installable "Recommended
> baseline package set"
>
>
>
> Why? What's wrong with pip install? Why complicate things? Your motivation
> is really weak here. "beneficial"? "difficult cases"?
>
>
>
> On Sat, Oct 28, 2017 at 8:57 PM, Nick Coghlan  wrote:
>
> Over on python-dev, the question of recommending MRAB's "regex" module
> over the standard library's "re" module for more advanced regular
> expressions recently came up again.
>
> Because of various logistical issues and backwards compatibility risks,
> it's highly unlikely that we'll ever be able to swap out the current _sre
> based re module implementation in the standard library for an
> implementation based on the regex module.
>
> At the same time, it would be beneficial to have a way to offer an even
> stronger recommendation to redistributors that we think full-featured
> general purpose Python scripting environments should offer the regex module
> as an opt-in alternative to the baseline re feature set, since that would
> also help with other currently difficult cases like the requests module.
>
> What I'm thinking is that we could make some relatively simple additions
> to the `ensurepip` and `venv` modules to help with this:
>
>
>
> 1. Add a ensurepip.RECOMMENDED_PACKAGES mapping keyed by standard library
> module names containing dependency specifiers for recommended third party
> packages for particular tasks (e.g. "regex" as an enhanced alternative to
> "re", "requests" as an enhanced HTTPS-centric alternative to
> "urllib.request")
>
> 2. Add a new `install_recommended` boolean flag to ensurepip.bootstrap
>
> 3. Add a corresponding `--install-recommended flag to the `python -m
> ensurepip` CLI
>
> 4. Add a corresponding `--install-recommended flag to the `python -m venv`
> CLI (when combined with `--without-pip`, this would run pip directly from
> the bundled wheel file to do the installations)
>
> We'd also need either a new informational PEP or else a section in the
> developer guide to say that the contents of `ensurepip.RECOMMENDED_PACKAGES`
> are up to the individual module maintainers (hence keying the mapping by
> standard library module name, rather than having a single flat list for the
> entire standard library).
>
> For redistributors with weak dependency support, these reference
> interpreter level recommendations could become redistributor level
> recommendations. Redistributors without weak dependency support could still
> make a distinction between "default" installations (which would include
> them) and "minimal" installations (which would exclude them).
>
> Folks writing scripts and example code for independent distribution (i.e.
> no explicitly declared dependencies) could then choose between relying on
> just the standard library (as now), or on the standard library plus
> independently versioned recommended packages.
>
>
>
> Cheers,
>
> Nick.
>
>
> --
>
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
>
> --
>
> --Guido van Rossum (python.org/~guido )
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Dollar operator suggestion

[Stephan Houben]

Why not a functional syntax, i.e.

compose(f, g, h)

rather than f $ g $ h

Advantage: you can do it today.
Without need to convince Guido to add more line noise to the language.

https://gist.github.com/stephanh42/6c9158c2470832a675fad7658048be9d

Stephan


2017-10-26 13:06 GMT+02:00 Yan Pas :

> I've looked up this feature in haskell. Dollar sign operator is used to
> avoid parentheses.
>
> Rationalle:
> Python tends to use functions instead of methods ( e.g. len([1,2,3])
> instead of [1,2,3].len() ). Sometimes the expression inside parentheses
> may become big  and using a lot of parentheses may tend to bad readability.
> I suggest the following syntax:
>
> len $ [1,2,3]
>
> Functions map be also  chained:
>
> len $ list $ map(...)
>
> This operator may be used for function composition too:
>
> foo = len $ set $
> in the same as
> foo = lambda *as,**kas : len(set(*as, **kas))
> in current syntax
>
> Regards,
> Yan
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why not picoseconds?

[Stephan Houben]

Hi all,

Please excuse me for getting a bit off-topic, but I would like
to point out that except for bean-counters who need to be
bug-compatible with accounting standards, decimal
floating point is generally a bad idea.

That is because the worst-case bound on the rounding error
grows linear with the base size. So you really want to choose
a base size as small as possible, i.e., 2.
This is not really related to the fact that computers use base-2
arithmetic, that is just a happy coincidence.
If we used ternary logic for our computers, FP should still be
based on base-2 and computer architects would complain
about the costly multiplication and division with powers of two
(just as they have historically complained about the costly
implementation of denormals, but we still got that, mostly
thanks to prof. Kahan convincing Intel).

Worse, a base other than 2 also increases the spread in the
average rounding error. This phenomenon is called "wobble"
and adds additional noise into calculations.

The ultimate problem is that the real number line contains
quite a few more elements than our puny computers can handle.
There is no 100% solution for this, but of all the possible
compromises, floating-point forms a fairly optimum point
in the design space for a wide range of applications.

Stephan

Op 20 okt. 2017 3:13 p.m. schreef "Victor Stinner" :


Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Memory limits [was Re: Membership of infinite iterators]

[Stephan Houben]

Hi Steve,

2017-10-19 1:59 GMT+02:00 Steven D'Aprano :

> On Wed, Oct 18, 2017 at 02:51:37PM +0200, Stefan Krah wrote:
>
> > $ softlimit -m 10 python3
> [...]
> > MemoryError
> >
> >
> > People who are worried could make a python3 alias or use Ctrl-\.
>
> I just tried that on two different Linux computers I have, and neither
> have softlimit.
>
>
Yeah, not sure what "softlimit" is either.
I'd suggest sticking to POSIX-standard ulimit or just stick
something like this in the .pythonrc.py:

import resource
resource.setrlimit(resource.RLIMIT_DATA, (2 * 1024**3, 2 * 1024**3))

Nor (presumably) would this help Windows users.
>

I (quickly) tried to get something to work using the win32 package,
in particular the win32job functions.
However, it seems setting
"ProcessMemoryLimit" using win32job.SetInformationJobObject
had no effect
(i.e.  a subsequent win32job.QueryInformationJobObject
still showed the limit as 0)?

People with stronger Windows-fu may be aware what is going on here...

Stephan


>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Membership of infinite iterators

[Stephan Houben]

Hi all,

FWIW, I just tried the list(count()) experiment on my phone (Termux Python
interpreter under Android).

Python 3.6.2 (default, Sep 16 2017, 23:55:07)
[GCC 4.2.1 Compatible Android Clang 5.0.300080 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import itertools
>>> list(itertools.count())
Killed

Interestingly even the Termux app stays alive and otherwise the phone
remains responsive and doesn't get hot. I am now sending this mail from
that very phone.

So this issue is not an issue on the world's most popular OS 😁

Stephan







Op 18 okt. 2017 08:46 schreef "Brendan Barnwell" :

> On 2017-10-17 07:26, Serhiy Storchaka wrote:
>
>> 17.10.17 17:06, Nick Coghlan пише:
>>
>>> >Keep in mind we're not talking about a regular loop you can break out of
>>> >with Ctrl-C here - we're talking about a tight loop inside the
>>> >interpreter internals that leads to having to kill the whole host
>>> >process just to get out of it.
>>>
>> And this is the root of the issue. Just let more tight loops be
>> interruptible with Ctrl-C, and this will fix the more general issue.
>>
>
> I was just thinking the same thing.  I think in general it's
> always bad for code to be uninterruptible with Ctrl-C.  If these infinite
> iterators were fixed so they could be interrupted, this containment problem
> would be much less painful.
>
> --
> Brendan Barnwell
> "Do not follow where the path may lead.  Go, instead, where there is no
> path, and leave a trail."
>--author unknown
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why not picoseconds?

[Stephan Houben]

Hi all,

I realize this is a bit of a pet peeve of me, since
in my day job I sometimes get people complaining that
numerical data is "off" in the sixteenth significant digit
(i.e. it was stored as a double).

I have a bunch of favorite comparisons to make this clear
how accurate a "double" really is: you can measure the
distance to Neptune down to a millimeter with it, or
the time from the extinction of the dinosaurs until
now down to half-second resolution.

Unfortunately for my argument, measuring time is one
of the most accurate physical measurements we can make,
and the best of the best exceed double-precision accuracy.

For example, the best atomic clock
https://www.sciencealert.com/physicists-have-broken-the-record-for-the-most-accurate-clock-ever-built
achieves a measurement uncertainty of 3e-18, which is about two order of
magnitudes more accurate than what double-precision gives you;
the latter runs out of steam at 2.2e-16.

So I realize there is obvious a strong need for (the illusion of)
such precise time keeping in the Python API .

Interestingly, that 2.2e-16 pretty much aligns with the accuracy of the
cesium atomic clocks which are currently used to *define* the second.
So we move to this new API, we should provide our own definition
of the second, since those rough SI seconds are just too imprecise for that.

Stephan


2017-10-16 9:03 GMT+02:00 Stefan Krah :

> On Mon, Oct 16, 2017 at 08:40:33AM +0200, Stephan Houben wrote:
> > "The problem is that Python returns time as a floatting point number
> > > which is usually a 64-bit binary floatting number (in the IEEE 754
> > > format). This type starts to loose nanoseconds after 104 days."
> > >
> > >
> > Do we realize that at this level of accuracy, relativistic time
> dilatation
> > due
> > to continental drift starts to matter?
>
> tai64na has supported attoseconds for quite some time:
>
> https://cr.yp.to/libtai/tai64.html
>
> The relativity issue is declared to be out of scope for the document. :-)
>
>
> Stefan Krah
>
>
>
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why not picoseconds?

[Stephan Houben]

2017-10-15 22:08 GMT+02:00 Eric V. Smith :
>From Victor's original message, describing the current functions using
64-bit binary floating point numbers (aka double). They lose precision:

"The problem is that Python returns time as a floatting point number
> which is usually a 64-bit binary floatting number (in the IEEE 754
> format). This type starts to loose nanoseconds after 104 days."
>
>
Do we realize that at this level of accuracy, relativistic time dilatation
due
to continental drift starts to matter?

Stephan



> Eric.
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Why not picoseconds?

[Stephan Houben]

Hi all,

I propose multiples of the Planck time, 5.39 × 10 −44 s.
Unlikely computers can be more accurate than that anytime soon.

On a more serious note, I am not sure what problem was solved by moving from
double to a fixed-precision format.

I do know that it now introduced the issue of finding
the correct base unit of the fixed-precision format.

Stephan




2017-10-15 20:28 GMT+02:00 Victor Stinner :

> I proposed to use nanoseconds because UNIX has 1 ns resolution in
> timespec, the most recent API, and Windows has 100 ns.
>
> Using picoseconds would confuse users who may expect sub-nanosecond
> resolution, whereas no OS support them currently.
>
> Moreover, nanoseconds as int already landed in os.stat and os.utime.
>
> Last but not least, I already strugle in pytime.c to prevent integer
> overflow with 1 ns resolution. It can quickly become much more complex if
> there is no native C int type supporting a range large enough to more 1
> picosecond resolution usable. I really like using int64_t for _PyTime_t,
> it's well supported, very easy to use (ex: "t = t2 - t1"). 64-bit int
> supports year after 2200 for delta since 1970.
>
> Note: I only know Ruby which chose picoseconds.
>
> Victor
>
> Le 15 oct. 2017 19:18, "Antoine Pitrou"  a écrit :
>
>>
>> Since new APIs are expensive and we'd like to be future-proof, why not
>> move to picoseconds?  That would be safe until clocks reach the THz
>> barrier, which is quite far away from us.
>>
>> Regards
>>
>> Antoine.
>>
>>
>> On Fri, 13 Oct 2017 16:12:39 +0200
>> Victor Stinner 
>> wrote:
>> > Hi,
>> >
>> > I would like to add new functions to return time as a number of
>> > nanosecond (Python int), especially time.time_ns().
>> >
>> > It would enhance the time.time() clock resolution. In my experience,
>> > it decreases the minimum non-zero delta between two clock by 3 times,
>> > new "ns" clock versus current clock: 84 ns (2.8x better) vs 239 ns on
>> > Linux, and 318 us (2.8x better) vs 894 us on Windows, measured in
>> > Python.
>> >
>> > The question of this long email is if it's worth it to add more "_ns"
>> > time functions than just time.time_ns()?
>> >
>> > I would like to add:
>> >
>> > * time.time_ns()
>> > * time.monotonic_ns()
>> > * time.perf_counter_ns()
>> > * time.clock_gettime_ns()
>> > * time.clock_settime_ns()
>> >
>> > time(), monotonic() and perf_counter() clocks are the 3 most common
>> > clocks and users use them to get the best available clock resolution.
>> > clock_gettime/settime() are the generic UNIX API to access these
>> > clocks and so should also be enhanced to get nanosecond resolution.
>> >
>> >
>> > == Nanosecond resolution ==
>> >
>> > More and more clocks have a frequency in MHz, up to GHz for the "TSC"
>> > CPU clock, and so the clocks resolution is getting closer to 1
>> > nanosecond (or even better than 1 ns for the TSC clock!).
>> >
>> > The problem is that Python returns time as a floatting point number
>> > which is usually a 64-bit binary floatting number (in the IEEE 754
>> > format). This type starts to loose nanoseconds after 104 days.
>> > Conversion from nanoseconds (int) to seconds (float) and then back to
>> > nanoseconds (int) to check if conversions loose precision:
>> >
>> > # no precision loss
>> > >>> x=2**52+1; int(float(x * 1e-9) * 1e9) - x
>> > 0
>> > # precision loss! (1 nanosecond)
>> > >>> x=2**53+1; int(float(x * 1e-9) * 1e9) - x
>> > -1
>> > >>> print(datetime.timedelta(seconds=2**53 / 1e9))
>> > 104 days, 5:59:59.254741
>> >
>> > While a system administrator can be proud to have an uptime longer
>> > than 104 days, the problem also exists for the time.time() clock which
>> > returns the number of seconds since the UNIX epoch (1970-01-01). This
>> > clock started to loose nanoseconds since mid-May 1970 (47 years ago):
>> >
>> > >>> import datetime
>> > >>> print(datetime.datetime(1970, 1, 1) + datetime.timedelta(seconds=2**53
>> / 1e9))
>> > 1970-04-15 05:59:59.254741
>> >
>> >
>> > == PEP 410 ==
>> >
>> > Five years ago, I proposed a large and complex change in all Python
>> > functions returning time to support nanosecond resolution using the
>> > decimal.Decimal type:
>> >
>> >https://www.python.org/dev/peps/pep-0410/
>> >
>> > The PEP was rejected for different reasons:
>> >
>> > * it wasn't clear if hardware clocks really had a resolution of 1
>> > nanosecond, especially when the clock is read from Python, since
>> > reading a clock in Python also takes time...
>> >
>> > * Guido van Rossum rejected the idea of adding a new optional
>> > parameter to change the result type: it's an uncommon programming
>> > practice (bad design in Python)
>> >
>> > * decimal.Decimal is not widely used, it might be surprised to get such
>> type
>> >
>> >
>> > == CPython enhancements of the last 5 years ==
>> >
>> > Since this PEP was rejected:
>> >
>> > * the os.stat_result got 3 fields for timestamps as nanoseconds
>> > (Python int): st_atime_ns, st_ctime_ns, st_mtime_ns
>> >
>> > * Python 3.3 got 3 

Re: [Python-ideas] + operator on generators

[Stephan Houben]

Unfortunately this is existing syntax:

a++b
is parsed as
a+(+b)

Stephan


Op 27 jun. 2017 6:03 p.m. schreef "Joshua Morton" :

Just another syntactical suggestion: the binary ++ operator is used as
concat in various contexts in various languages, and is probably less
likely to confuse people as being either a logical or binary &.

On Tue, Jun 27, 2017 at 6:53 AM Stephan Houben  wrote:

> > Its the applications where it *is* important that
> > we should be looking at.
>
> Um, yes, but given our relative positions in this debate,
> the onus is not really on *me* to demonstrate such an application, right?
> That would just confuse everbody ;-)
>
> (FWIW, Sagemath is not mostly "numerical processing", it is mostly
> *symbolic* calculations and involves a lot of complex algorithms and
> datastructures, including sequences.)
>
> Stephan
>
> 2017-06-27 13:48 GMT+02:00 Steven D'Aprano :
> > On Tue, Jun 27, 2017 at 01:32:05PM +0200, Stephan Houben wrote:
> >> Hi Steven,
> >>
> >> To put this into perspective, I did some greps on Sagemath,
> >> being the largest Python project I have installed on this machine
> >> (1955 .py files).
> >
> > And one which is especially focused on numerical processing, not
> > really the sort of thing that does a much iterator chaining. That's
> > hardly a fair test -- we know there are applications where chaining is
> > not important at all. Its the applications where it *is* important that
> > we should be looking at.
> >
> >
> > --
> > Steve
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Stephan Houben]

> Its the applications where it *is* important that
> we should be looking at.

Um, yes, but given our relative positions in this debate,
the onus is not really on *me* to demonstrate such an application, right?
That would just confuse everbody ;-)

(FWIW, Sagemath is not mostly "numerical processing", it is mostly
*symbolic* calculations and involves a lot of complex algorithms and
datastructures, including sequences.)

Stephan

2017-06-27 13:48 GMT+02:00 Steven D'Aprano :
> On Tue, Jun 27, 2017 at 01:32:05PM +0200, Stephan Houben wrote:
>> Hi Steven,
>>
>> To put this into perspective, I did some greps on Sagemath,
>> being the largest Python project I have installed on this machine
>> (1955 .py files).
>
> And one which is especially focused on numerical processing, not
> really the sort of thing that does a much iterator chaining. That's
> hardly a fair test -- we know there are applications where chaining is
> not important at all. Its the applications where it *is* important that
> we should be looking at.
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Stephan Houben]

Hi Steven,

To put this into perspective, I did some greps on Sagemath,
being the largest Python project I have installed on this machine
(1955 .py files).

Occurrences:
enumerate: 922
zip: 585
itertools.product: 67
itertools.combinations: 18
itertools.islice: 17
chain: 14 (with or without itertools. prefix)

This seems to confirm my gut feeling that "chain" just isn't that
common an operation;
even among itertools functions, product, combinations and islice are
more common.

Based on this I would say there is little justification to even put
"chain" in builtins,
let alone to give it dedicated syntax.

I also note that * for repetition is only supported for a few
iterables (list, tuple),
incidentally the same ones which support + for sequence chaining.

Stephan




2017-06-27 12:38 GMT+02:00 Steven D'Aprano :
> On Tue, Jun 27, 2017 at 11:01:32AM +0200, Stephan Houben wrote:
>> Hi all,
>>
>> Is "itertools.chain" actually that common?
>> Sufficiently common to warrant its own syntax?
>
> I think it's much more common than (say) sequence repetition:
>
> a = [None]*5
>
> which has had an operator for a long, long time.
>
>> In my experience, "enumerate" is far more common
>> among the iterable operations.
>> And that doesn't have special syntax either.
>
> True. But enumerate is a built-in, and nearly always used in a single
> context:
>
> for i, x in enumerate(sequence):
> ...
>
> A stranger to Python could almost be forgiven for thinking that
> enumerate is part of the for-loop syntax.
>
> In contrast, chaining (while not as common as, say, numeric addition)
> happens in variable contexts: in expressions, as arguments to function
> calls, etc.
>
> It is absloutely true that this proposal brings nothing new to the
> language that cannot already be done. It's syntactic sugar. So I guess
> the value of it depends on whether or not you chain iterables enough
> that you would rather use an operator rather than a function.
>
>> A minimal  proposal would be to promote "chain" to builtins.
>
> Better than nothing, I suppose.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Stephan Houben]

Hi all,

Is "itertools.chain" actually that common?
Sufficiently common to warrant its own syntax?

In my experience, "enumerate" is far more common
among the iterable operations.
And that doesn't have special syntax either.

A minimal  proposal would be to promote "chain" to builtins.

Stephan

2017-06-27 10:40 GMT+02:00 Greg Ewing :
> David Mertz wrote:
>>
>> I just wish I could think of a good character that doesn't have some very
>> different meaning in other well-known contexts (even among iterables).
>
>
>(a;b)
>
> Should be unambiguous as long as the parens are required.
>
> --
> Greg
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] + operator on generators

[Stephan Houben]

I would like to add that for example numpy ndarrays are iterables, but
they have an __add__ with completely different semantics, namely
element-wise ( numerical) addition.

So this proposal would conflict with existing libraries with iterable
objects.

Stephan

Op 25 jun. 2017 2:51 p.m. schreef "Serhiy Storchaka" :

> 25.06.17 15:06, lucas via Python-ideas пише:
>
>> I often use generators, and itertools.chain on them.
>> What about providing something like the following:
>>
>>  a = (n for n in range(2))
>>  b = (n for n in range(2, 4))
>>  tuple(a + b)  # -> 0 1 2 3
>>
>> This, from user point of view, is just as how the
>> __add__ operator works on lists and tuples.
>> Making generators works the same way could be a great way to avoid calls
>> to itertools.chain everywhere, and to limits the differences between
>> generators and other "linear" collections.
>>
>> I do not know exactly how to implement that (i'm not that good at C, nor
>> CPython source itself), but by seeing the sources,
>> i imagine that i could do something like the list_concat function at
>> Objects/listobject.c:473, but in the Objects/genobject.c file,
>> where instead of copying elements i'm creating and initializing a new
>> chainobject as described at Modules/itertoolsmodule.c:1792.
>>
>> (In pure python, the implementation would be something like `def
>> __add__(self, othr): return itertools.chain(self, othr)`)
>>
>
> It would be weird if the addition is only supported for instances of the
> generator class, but not for other iterators. Why (n for n in range(2)) +
> (n for n in range(2, 4)) works, but iter(range(2)) + iter(range(2, 4)) and
> iter([0, 1]) + iter((2, 3)) don't? itertools.chain() supports arbitrary
> iterators. Therefore you will need to implement the __add__ method for
> *all* iterators in the world.
>
> However itertools.chain() accepts not just *iterators*. It works with
> *iterables*. Therefore you will need to implement the __add__ method also
> for all iterables in the world. But __add__ already is implemented for list
> and tuple, and many other sequences, and your definition conflicts with
> this.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Improving Catching Exceptions

[Stephan Houben]

2017-06-23 17:09 GMT+02:00 Andy Dirnberger :

> It's not really a proposal. It's existing syntax.

Wow! I have been using Python since 1.5.2 and I never knew this.

This is not Guido's famous time machine in action, by any chance?

Guess there's some code to refactor using this construct now...

Stephan


2017-06-23 17:09 GMT+02:00 Andy Dirnberger :
> Hi Stephan,
>
> On Fri, Jun 23, 2017 at 6:23 AM, Stephan Houben 
> wrote:
>>
>> Hi Andy,
>>
>> What you propose is essentially the "try .. catch .. in" construct as
>> described for Standard ML in:
>
>
> It's not really a proposal. It's existing syntax. I was suggesting a way to
> implement the example that would catch an IndexError raised by accessing
> elements in bah but not those raised by foo.
>
>
>>
>>
>>
>> https://pdfs.semanticscholar.org/b24a/60f84b296482769bb6752feeb3d93ba6aee8.pdf
>>
>> Something similar for Clojure is at:
>> https://github.com/rufoa/try-let
>>
>> So clearly this is something more people have struggled with.
>> The paper above goes into deep detail on the practical and
>> (proof-)theoretical
>> advantages of such a construct.
>>
>> Stephan
>
>
> Andy
>
>
>>
>>
>> 2017-06-23 1:47 GMT+02:00 Andy Dirnberger :
>> >
>> >
>> >> On Jun 22, 2017, at 7:29 PM, Cameron Simpson  wrote:
>> >>
>> >>> On 23Jun2017 06:55, Steven D'Aprano  wrote:
>> >>>> On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
>> >>>> We usually teach our newbies to catch exceptions as narrowly as
>> >>>> possible, i.e. MyModel.DoesNotExist instead of a plain Exception.
>> >>>> This
>> >>>> works out quite well for now but the number of examples continue to
>> >>>> grow
>> >>>> where it's not enough.
>> >>>
>> >>> (1) Under what circumstances is it not enough?
>> >>
>> >> I believe that he means that it isn't precise enough. In particular,
>> >> "nested exceptions" to me, from his use cases, means exceptions thrown 
>> >> from
>> >> within functions called at the top level. I want this control too 
>> >> sometimes.
>> >>
>> >> Consider:
>> >>
>> >>   try:
>> >>   foo(bah[5])
>> >>   except IndexError as e:
>> >>   ... infer that there is no bah[5] ...
>> >>
>> >> Of course, it is possible that bah[5] existed and that foo() raised an
>> >> IndexError of its own. One might intend some sane handling of a missing
>> >> bah[5] but instead silently conceal the IndexError from foo() by 
>> >> mishandling
>> >> it as a missing bah[5].
>> >>
>> >> Naturally one can rearrange this code to call foo() outside that
>> >> try/except, but that degree of control often leads to quite fiddly looking
>> >> code with the core flow obscured by many tiny try/excepts.
>> >>
>> >> One can easily want, instead, some kind of "shallow except", which
>> >> would catch exceptions only if they were directly raised from the surface
>> >> code; such a construct would catch the IndexError from a missing bah[5] in
>> >> the example above, but _not_ catch an IndexError raised from deeper code
>> >> such within the foo() function.
>> >>
>> >> Something equivalent to:
>> >>
>> >>   try:
>> >>   foo(bah[5])
>> >>   except IndexError as e:
>> >>   if e.__traceback__ not directly from the try..except lines:
>> >>   raise
>> >>   ... infer that there is no bah[5] ...
>> >>
>> >> There doesn't seem to be a concise way to write that. It might not even
>> >> be feasible at all, as one doesn't have a way to identify the line(s) 
>> >> within
>> >> the try/except in a form that one can recognise in a traceback.
>> >
>> > How about something like this?
>> >
>> > try:
>> > val = bah[5]
>> > except IndexError:
>> > # handle your expected exception here
>> > else:
>> > foo(val)
>> >>
>> >>
>> >> Cheers,
>> >> Cameron Simpson 
>> >> ___
>> >> Python-ideas mailing list
>> >> Python-ideas@python.org
>> >> https://mail.python.org/mailman/listinfo/python-ideas
>> >> Code of Conduct: http://python.org/psf/codeofconduct/
>> > ___
>> > Python-ideas mailing list
>> > Python-ideas@python.org
>> > https://mail.python.org/mailman/listinfo/python-ideas
>> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Improving Catching Exceptions

[Stephan Houben]

Hi Andy,

What you propose is essentially the "try .. catch .. in" construct as
described for Standard ML in:

https://pdfs.semanticscholar.org/b24a/60f84b296482769bb6752feeb3d93ba6aee8.pdf

Something similar for Clojure is at:
https://github.com/rufoa/try-let

So clearly this is something more people have struggled with.
The paper above goes into deep detail on the practical and (proof-)theoretical
advantages of such a construct.

Stephan

2017-06-23 1:47 GMT+02:00 Andy Dirnberger :
>
>
>> On Jun 22, 2017, at 7:29 PM, Cameron Simpson  wrote:
>>
>>> On 23Jun2017 06:55, Steven D'Aprano  wrote:
 On Thu, Jun 22, 2017 at 10:30:57PM +0200, Sven R. Kunze wrote:
 We usually teach our newbies to catch exceptions as narrowly as
 possible, i.e. MyModel.DoesNotExist instead of a plain Exception. This
 works out quite well for now but the number of examples continue to grow
 where it's not enough.
>>>
>>> (1) Under what circumstances is it not enough?
>>
>> I believe that he means that it isn't precise enough. In particular, "nested 
>> exceptions" to me, from his use cases, means exceptions thrown from within 
>> functions called at the top level. I want this control too sometimes.
>>
>> Consider:
>>
>>   try:
>>   foo(bah[5])
>>   except IndexError as e:
>>   ... infer that there is no bah[5] ...
>>
>> Of course, it is possible that bah[5] existed and that foo() raised an 
>> IndexError of its own. One might intend some sane handling of a missing 
>> bah[5] but instead silently conceal the IndexError from foo() by mishandling 
>> it as a missing bah[5].
>>
>> Naturally one can rearrange this code to call foo() outside that try/except, 
>> but that degree of control often leads to quite fiddly looking code with the 
>> core flow obscured by many tiny try/excepts.
>>
>> One can easily want, instead, some kind of "shallow except", which would 
>> catch exceptions only if they were directly raised from the surface code; 
>> such a construct would catch the IndexError from a missing bah[5] in the 
>> example above, but _not_ catch an IndexError raised from deeper code such 
>> within the foo() function.
>>
>> Something equivalent to:
>>
>>   try:
>>   foo(bah[5])
>>   except IndexError as e:
>>   if e.__traceback__ not directly from the try..except lines:
>>   raise
>>   ... infer that there is no bah[5] ...
>>
>> There doesn't seem to be a concise way to write that. It might not even be 
>> feasible at all, as one doesn't have a way to identify the line(s) within 
>> the try/except in a form that one can recognise in a traceback.
>
> How about something like this?
>
> try:
> val = bah[5]
> except IndexError:
> # handle your expected exception here
> else:
> foo(val)
>>
>>
>> Cheers,
>> Cameron Simpson 
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/