Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Paul Moore wrote:

Where in [fn(x) for x in lst if cond] is the *
allowed? fn(*x)? *fn(x)?


Obviously you're *allowed* to put fn(*x), because that's
already a valid function call, but the only *new* place
we're talking about, and proposing new semantics for, is
in front of the expression representing items to be added
to the list, i.e. [*fn(x) for ...]


I think it's probably time for someone to
describe the precise syntax (as BNF, like the syntax in the Python
docs at 
https://docs.python.org/3.6/reference/expressions.html#displays-for-lists-sets-and-dictionaries


Replace

   comprehension ::=  expression comp_for

with

   comprehension ::=  (expression | "*" expression) comp_for


and semantics (as an explanation of how to
rewrite any syntactically valid display as a loop).


The expansion of the "*" case is the same as currently except
that 'append' is replaced by 'extend' in a list comprehension,
'yield' is replaced by 'yield from' in a generator
comprehension.

If we decided to also allow ** in dict comprehensions, then
the expansion would use 'update'.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Jonathan Goble]

On Fri, Oct 14, 2016 at 1:54 AM, Steven D'Aprano  wrote:
>> and:
>>
>> o = ord (c)
>> if 100 <= o <= 150:
>
> Which is clearly not the same thing, and better written as:
>
> if "d" <= c <= "\x96":
> ...

Or, if you really want to use ord(), you can use hex literals:

o = ord(c)
if 0x64 <= o <= 0x96:
...
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Paul Moore wrote:

please can you explain how to modify that translation rule to
incorporate the suggested syntax?


It's quite simple: when there's a '*', replace 'append'
with 'extend':

[*fn(x) for x in lst if cond]

expands to

result = []
for x in lst:
   if cond:
  result.extend(fn(x))

The people thinking that you should just stick the '*x'
in as an argument to append() are misunderstanding the
nature of the expansion. You can't do that, because
the current expansion is based on the assumption that
the thing being substituted is an expression, and
'*x' is not a valid expression on its own. A new rule
is needed to handle that case.

And I'm the one who *invented* that expansion, so I
get to say what it means. :-)

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Sjoerd Job Postmus wrote:

I think the suggested spelling (`*`) is the confusing part. If it were
to be spelled `from ` instead, it would be less confusing.


Are you suggesting this spelling just for generator
comprehensions, or for list comprehensions as well?
What about dict comprehensions?

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Mikhail V]

On 13 October 2016 at 10:18, M.-A. Lemburg  wrote:

> I suppose you did not intend everyone to have to write
> \u010 just to get a newline code point to avoid the
> ambiguity.

Ok there are different usage cases.
So in short without going into detail,
for example if I need to type in a unicode
string literal in ASCII editor I would find such notation
replacement beneficial for me:

u'\u0430\u0431\u0432.txt'
-->
u"{1072}{1073}{1074}.txt"

Printing could be the same I suppose.
I use Python 2.7. And printing so
with numbers instead of non-ASCII would help me see
where I have non-ASCII chars. But I think the print
behavior must be easily configurable.

Any critics on it? Besides not following the unicode consortium.
Also I would not even mind fixed width 7-digit decimals actually.
Ugly but still for me better than hex.

Mikhail
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Neil Girdhar wrote:
At the end of this discussion it might be good to get a tally of how 
many people think the proposal is reasonable and logical.


I think it's reasonable and logical.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Random832 wrote:


[*map(math.exp, t) for t in [(1, 2), (3, 4)]]

[*(math.exp(x) for x in t) for t in [(1, 2), (3, 4)]]


Or more simply,

  [math.exp(x) for t in [(1, 2), (3, 4)] for x in t]

I think this brings out an important point. While it
would be nice to allow * unpacking in comprehensions
for consistency with displays, it's not strictly
necessary, since you can always get the same effect
with another level of looping.

So it comes down to whether you think added conistency,
plus maybe some efficiency gains in some cases, are worth
making the change.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Steven D'Aprano]

On Fri, Oct 14, 2016 at 07:21:48AM +0200, Mikhail V wrote:

> I'll explain what I mean with an example.
> This is an example which I would make to
> support my proposal. Compare:
> 
> if "\u1230" <= c <= "\u123f":

For an English-speaker writing that, I'd recommend:

if "\N{ETHIOPIC SYLLABLE SA}" <= c <= "\N{ETHIOPIC SYLLABLE SHWA}":
...

which is a bit verbose, but that's the price you pay for programming 
with text in a language you don't read. If you do read Ethiopian, then 
you can simply write:

if "ሰ" <= c <= "ሿ":
...

which to a literate reader of Ethiopean, is no harder to understand than 
the strange and mysterious rotated and reflected glyphs used by Europeans:

if "d" <= c <= "p":
...

(Why is "double-u" written as vv (w) instead of uu?)



> and:
> 
> o = ord (c)
> if 100 <= o <= 150:

Which is clearly not the same thing, and better written as:

if "d" <= c <= "\x96":
...


> So yours is a valid code but for me its freaky,
> and surely I stick to the second variant.
> You said, I can better see in which unicode page
> I am by looking at hex ordinal, but I hardly
> need it, I just need to know one integer, namely
> where some range begins, that's it.
> Furthermore this is the code which would an average
> programmer better read and maintain.

No, the average programmer is MUCH more skillful than that. Your 
standard for what you consider "average" seems to me to be more like 
"lowest 20%".

[...]
> I feel however like being misunderstood or so.

Trust me, we understand you perfectly. You personally aren't familiar or 
comfortable with hexadecimal, Unicode code points, or programming 
standards which have been in widespread use for at least 35 years, and 
probably more like 50, but rather than accepting that this means you 
have a lot to learn, you think you can convince the rest of the world to 
dumb-down and de-skill to a level that you are comfortable with. And 
that eventually the entire world will standardise on just 100 
characters, which you think is enough for all communication, maths and 
science.

Good luck with that last one. Even if you could convince the Chinese and 
Japanese to swap to ASCII, I'd like to see you pry the emoji out of the 
young folk's phones.



[...]
> It is not about speed, it is about brain load.
> Chinese can read their hieroglyphs fast, but
> the cognition load on the brain is 100 times higher
> than current latin set.

Citation required.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[David Mertz]

I've never used nor taught a * in a list display. I don't think they seem
so bad, but it's a step down a slippery slope towards forms that might as
well be Perl.

On Oct 13, 2016 10:33 PM, "Greg Ewing"  wrote:

> David Mertz wrote:
>
>> it would always be "Here's a Python wart to look out for if you see it in
>> other code... you should not ever use it yourself."
>>
>
> Do you currently tell them the same thing about the use
> of * in a list display?
>
> --
> Greg
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

David Mertz wrote:
it would always be "Here's a Python wart to look out 
for if you see it in other code... you should not ever use it yourself."


Do you currently tell them the same thing about the use
of * in a list display?

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Greg Ewing]

Steven D'Aprano wrote:


py> def gen():
... for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
... yield *t
  File "", line 3
yield *t
  ^
SyntaxError: invalid syntax

Even if it was allowed, what would it mean? It could only mean "unpack 
the sequence t, and collect the values into a tuple; then yield the 
tuple".


To maintain the identity

  list(*x for x in y) == [*x for x in y]

it would be necessary for the *x in (*x for x in y) to expand
to "yield from x".

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Mikhail V]

Greg Ewing wrote:

> #define O_RDONLY0x  /* open for reading only */
> #define O_WRONLY0x0001  /* open for writing only */
> #define O_RDWR  0x0002  /* open for reading and writing */
> #define O_ACCMODE   0x0003  /* mask for above modes */

Good example. But it is not an average high level
code of course. Example works again only if
we for some reason follow binary segmentation
which is bound to low level functionality, in
this case 8 bit grouping.

> If you have occasion to write a literal representing a
> character code, there's nothing to stop you writing it
> in hex to match the way it's shown in a repr(), or in
> published Unicode tables, etc.

>>> c = "\u1235"
>>> if "\u1230" <= c <= "\u123f":

> I don't see a need for any conversions back and forth.

I'll explain what I mean with an example.
This is an example which I would make to
support my proposal. Compare:

if "\u1230" <= c <= "\u123f":

and:

o = ord (c)
if 100 <= o <= 150:

So yours is a valid code but for me its freaky,
and surely I stick to the second variant.
You said, I can better see in which unicode page
I am by looking at hex ordinal, but I hardly
need it, I just need to know one integer, namely
where some range begins, that's it.
Furthermore this is the code which would an average
programmer better read and maintain.
So it is the question of maintainability (+1).
Secondly, for me it is the question of being able to
type in and correct these decimal values: look,
if I make a mistake, typo, or want to expand the range
by some value I need to make summ and substract
operation in my head to progress with my code effectively.
Obviously nobody operates good with two notations
in head simultanosly, so I will complete my code
without extra effort.
Is it clear now what I mean by
conversions back and forth?
This example alone actually explains
my whole point very well,
I feel however like being misunderstood or so.

>> I am not against base-16 itself in the first place,
>> but rather against the character set which is simply visually
>> inconsistent and not readable.
>Now you're talking about inventing new characters, or
>at least new glyphs for existing ones, and persuading
>everyone to use them. That's well beyond the scope of
>what Python can achieve!

Yes ideally one uses other glyphs for base-16
it does not however mean that one must
use new invented glyphs. In standard ASCII
there are enough glyphs that would work way better
together, but it is too late anyway, should be better
decided at the time of standard declaration.
Got to love it.

> The meaning of 0xC001 is much clearer to me than
> 1101, because I'd have to count the bits very
> carefully in the latter to distinguish it from, e.g.
> 6001 or 18001.
> The bits could be spaced out:
> 1100   0001
> but that just takes up even more room to no good effect.
> I don't find it any faster to read -- if anything, it's

Greg, I feel somehow that you are an open minded person
and I value this. You also can understand
quite good how you read.
What you refer to here is the brevity of the word
Indeed there is some degrade of readability
if the word is too big, or a font is set to
big size, so you brake it, one step towards better.

And now I'll explain you some further magic
regarding the binary representation.
If you find free time you can experiment a bit.
So what is so peculiar about bitstring actually?
Bitstring unlike higher bases
can be treated as an abscence/presence
of the signal, which is not possible
for higher bases, literally binary string
can be made almost "analphabetic"
if one could say so.
Consider such notation:

instead of 1100   0001 you write

ұұ-ұ   ---ұ

(NOTE: of course if you read this in non monospaced
font you will not see it correct, I should make screenshots
which I will do in a while)

Note that I choose this letter
not accidentally, this letter is similar
to one of glyphs with peak readability.
The unset value simply would be a stroke.
So I take only one letter.

ұұ-ұ   ---ұ
---ұ ---ұ --ұ- -ұ--
--ұ-   ---ұ
  --ұ- ---ұ
-ұұ- ұұ--  
   
--ұұ  ұ--- ---ұ
-ұ-- --ұұ  ---ұ

So the digits need not be equal-weighted as in
higher bases. What does it bring? Simple:
you can downscale the strings, so a 16-bit
value would be ~60 pixels wide (for 96dpi display)
without legibility loss, so it compensate
the "too wide to scan" issue.
And don't forget to make enough linespacing.

Other benefits of binary string obviously:
- nice editing feautures like bitshifting
- very interesting cognitive features,
(it becomes more noticable
however if you train to work with it)
...
So there is a whole bunch of good effects.
Understand me right, I don't have reason not to
believe you that you don't see any effect,
but you should always remember that this can be
simply caused by your habit. So if you are more
than 40 years old (sorry for some fa

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Thu, Oct 13, 2016, at 18:15, Steven D'Aprano wrote:
> Consider the analogy with f(*t), where t = (a, b, c). We *don't* have:
> 
> f(*t) is equivalent to f(a); f(b); f(c)

I don't know where this "analogy" is coming from.

f(*t) == f(a, b, c)
[*t] == [a, b, c]
{*t} == {a, b, c}

All of this is true *today*.

t, u, v = (a, b, c), (d, e, f), (g, h, i)
f(*t, *u, *v) == f(a, b, c, d, e, f, g, h, i)
[*t, *u, *v] == [a, b, c, d, e, f, g, h, i]


> > is very confusing, from what I understand: what does the `*` even mean
> > here.
> 
> Indeed. The reader may be forgiven for thinking that this is yet another 
> unrelated and arbitrary use of * to join the many other uses:

How is it arbitrary? 

> - mathematical operator;
> - glob and regex wild-card;
> - unpacking;

This is unpacking. It unpacks the results into the destination.

There's a straight line from [*t, *u, *v] to [*x for x in (t, u, v)].
What's surprising is that it doesn't work now.

I think last month we even had someone who didn't know about 'yield
from' propose 'yield *x' for exactly this feature. It is intuitive - it
is a straight-line extension of the unpacking syntax.

> - import all
> - and now yield from
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Null coalescing operator

[Mark E. Haase]

(Replying to multiple posts in this thread)

Guido van Rossum:

> Another problem is PEP 505 -- it
> is full of discussion but its specification is unreadable due to the
> author's idea to defer the actual choice of operators and use a
> strange sequence of unicode characters instead.


Hi, I wrote PEP-505. I'm sorry that it's unreadable. The choice of emoji as
operators was supposed to be a blatant joke. I'd be happy to submit a new
version that is ASCII. Or make any other changes that would facilitate
making a decision on the PEP.

As I recall, the thread concluded with Guido writing, "I'll have to think
about this," or something to that effect. I had hoped that the next step
could be a survey where we could gauge opinions on the various possible
spellings. I believe this was how PEP-308 was handled, and that was a very
similar proposal to this one.

Most of the discussion on list was really centered around the fact that
nobody like the proposed ?? or .? spellings, and nobody could see around
that fact to consider whether the feature itself was intrinsically
valuable. (This is why the PEP doesn't commit to a syntax.) Also, as
unfortunate side effect of a miscommunication, about 95% of the posts on
this PEP were written _before_ I submitted a complete draft and so most of
the conversation was arguing about a straw man.

David Mertz:
>
> The idea is that we can easily have both "regular" behavior and None
> coalescing just by wrapping any objects in a utility class... and WITHOUT
> adding ugly syntax.  I might have missed some corners where we would want
> behavior wrapped, but those shouldn't be that hard to add in principle.
>

The biggest problem with a wrapper in practice is that it has to be
unwrapped before it can be passed to any other code that doesn't know how
to handle it. E.g. if you want to JSON encode an object, you need to unwrap
all of the NullCoalesce objects because the json module wouldn't know what
to do with them. The process of wrapping and unwrapping makes the resulting
code more verbose than any existing syntax.

> How much of the time is a branch of the None check a single fallback value
> or attribute access versus how often a suite of statements within the
> not-None branch?
>
> I definitely check for None very often also. I'm curious what the
> breakdown is in code I work with.
>
There's a script in the PEP-505 repo that can you help you identify code
that could be written with the proposed syntax. (It doesn't identify blocks
that would not be affected, so this doesn't completely answer your
question.)

https://github.com/mehaase/pep-0505/blob/master/find-pep505.py

The PEP also includes the results of running this script over the standard
library.

On Sat, Sep 10, 2016 at 1:26 PM, Guido van Rossum  wrote:

> The way I recall it, we arrived at the perfect syntax (using ?) and
> semantics. The issue was purely strong hesitation about whether
> sprinkling ? all over your code is too ugly for Python, and in the end
> we couldn't get agreement on *that*. Another problem is PEP 505 -- it
> is full of discussion but its specification is unreadable due to the
> author's idea to defer the actual choice of operators and use a
> strange sequence of unicode characters instead.
>
> If someone wants to write a new, *short* PEP that defers to PEP 505
> for motivation etc. and just writes up the spec for the syntax and
> semantics we'll have a better starting point. IMO the key syntax is
> simply one for accessing attributes returning None instead of raising
> AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to
> `foo.bar.baz if (foo is not None and foo.bar is not None) else None`,
> except evaluating foo and foo.bar only once.
>
> On Sat, Sep 10, 2016 at 10:14 AM, Random832 
> wrote:
> > On Sat, Sep 10, 2016, at 12:48, Stephen J. Turnbull wrote:
> >> I forget if Guido was very sympathetic to null-coalescing operators,
> >> given somebody came up with a good syntax.
> >
> > As I remember the discussion, I thought he'd more or less conceded on
> > the use of ? but there was disagreement on how to implement it that
> > never got resolved. Concerns like, you can't have a?.b return None
> > because then a?.b() isn't callable, unless you want to use a?.b?() for
> > this case, or some people wanted to have "a?" [where a is None] return a
> > magic object whose attribute/call/getitem would give no error, but that
> > would have to keep returning itself and never actually return None for
> > chained operators.
> > ___
> > Python-ideas mailing list
> > Python-ideas@python.org
> > https://mail.python.org/mailman/listinfo/python-ideas
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/code

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[MRAB]

On 2016-10-14 02:04, Steven D'Aprano wrote:

On Thu, Oct 13, 2016 at 08:15:36PM +0200, Martti Kühne wrote:


Can I fix my name, though?


I don't understand what you mean. Your email address says your name is
Martti Kühne. Is that incorrect?


[snip]

You wrote "Marttii" and he corrected it when he quoted you in his reply.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 08:15:36PM +0200, Martti Kühne wrote:

> Can I fix my name, though?

I don't understand what you mean. Your email address says your name is 
Martti Kühne. Is that incorrect?



[...]
> I meant that statement in context of the examples which were brought up:
> the occurrence of a list comprehension inside an array have the
> following effect:
> 
> 1) [ ..., [expr for t in iterable] ]
> 
> is equivalent to:
> 
> def expr_long(iterable, result):
> result.append(iterable)
> return result
> 
> expr_long(iterable, [ ..., ])

The good thing about this example is that it is actual runnable code 
that we can run to see if they are equivalent. They are not equivalent.

py> def expr_long(iterable, result):
... result.append(iterable)
... return result
...
py> iterable = (100, 200, 300)
py> a = [..., [2*x for x in iterable]]
py> b = expr_long(iterable, [...])
py> a == b
False
py> print(a, b)
[Ellipsis, [200, 400, 600]] [Ellipsis, (100, 200, 300)]

For this to work, you have to evaluate the list comprehension first, 
then pass the resulting list to be appended to the result.

I don't think this is very insightful. All you have demonstrated is that 
a list display [a, b, c, ...] is equivalent to:

result = []
for x in [a, b, c, ...]:
result.append(x)


except that you have written it in a slightly functional form.


> so, if you make the case for pep448, you might arrive at the following:
> 
> 2) [ ..., *[expr for expr in iterable] ]

That syntax already works (in Python 3.5):

py> [1, 2, 3, *[x+1 for x in (100, 200, 300)], 4, 5]
[1, 2, 3, 101, 201, 301, 4, 5]


> which would be, if I'm typing it correctly, equivalent to, what
> resembles an external collection:
> 
> def expr_star(list_comp, result):
> result.extend(list(list_comp))
> return result
> 
> expr_star(iterable, [ ..., ])
> 
> Having this in mind, the step to making:
> 
> [ ..., [*expr for expr in iterable], ]
> 
> from:
> 
> def expr_insidestar(iterable, result):
> for expr in iterable:
> result.extend(expr)
> return result
> 
> does not appear particularly far-fetched, at least not to me and a few
> people on this list.

But you don't have [..., list_comp, ] you just have the list comp.

You are saying:

(1) List displays [a, b, c, d, ...] are like this; 
(2) we can sensibly extend that to the case [a, b, *c, d, ...]

I agree with (1) and (2). But then you have a leap:

(3) therefore [*t for t in iterable] should mean this.

There's a huge leap between the two.

To even begin to make sense of this, you have to unroll the list 
comprehension into a list display. But that's not very helpful:

[expr for t in iterable]

Would you rather see that explained as:

[expr, expr, expr, expr, ...] 

or as this?

result = []
for t in iterable:
result.append(expr)


The second form, the standard, documented explanation for 
comprehensions, also applies easily to more complex examples:

[expr for t in iter1 for u in iter2 for v in iter3 if condition]

result = []
for t in iter1:
for u in iter2:
for v in iter3:
if condition:
result.append(expr)



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Fwd: Add sorted (ordered) containers

[Grant Jenks]

On Thu, Oct 13, 2016 at 1:36 PM, Марк Коренберг  wrote:
>
> I mean mutable containers that are always sorted when iterating over them.
>
> See http://bugs.python.org/issue28433
>
> for example:
>
> * SortedSet (sorted unique elements, implemented using (rb?)tree instead of 
> hash)
> * SortedList (sorted elements, the same as SortedSet, but without uniquiness 
> constraint) - actually a (rb?)tree, not a list (i.e. not an array)
> * SortedDict (sorted by key when interating) - like C++'s ordered_map

Can you share more about your use cases for these containers? What are
you making?

Nick Coghlan gave an answer to this question on StackOverflow at
http://stackoverflow.com/a/5958960/232571 The answer kind of boils
down to "there should be one obvious way to do it" and existing Python
features like lists, sorted, bisect, and heapq cover many use cases.

I wrote the answer that is now the second highest rated for that
question. I've noticed that the upvotes have been accumulating at a
slightly higher rate than Nick's answer. I think that reflects an
increase in interest and maybe gradual tide change of opinion.

> There are many implementations in the net, like:
>
> http://www.grantjenks.com/docs/sortedcontainers

That's my project. I am the primary developer of the SortedContainers project.

You may also be interested in the
[SortedCollections](http://www.grantjenks.com/docs/sortedcollections/)
module which builds atop SortedContainers with data types like
ValueSortedDict and ItemSortedDict. Because it's pure-Python,
SortedContainers offers a lot of opportunity for
extension/customization. That's also made it easier for the API to
adapt/expand over time.

> I think it should be one standardized implementation of such containers in 
> CPython.
>
> Instead of trees, implementation may use SkipList structure, but this is just 
> implementation details.
>
> Such structres imply fast insertion and deletion, ability to iterate, and 
> also memory efficiency.

I gave a talk at PyCon 2016 about Python Sorted Collections[1] that's
worth watching. The first third discusses about six different
implementations with different strengths and weaknesses. The choice of
data type is more than implementation details. One of the biggest
issues is the various tradeoffs of data types like blists, rbtrees,
etc.

I have been meaning to discuss sorted collections further with Raymond
Hettinger (author of the Python collections module). We spoke after
the PyCon talk and wanted to continue the conversation. But I had a
busy summer and just a few weeks ago welcomed my first son into the
world. So realistically that conversation won't happen until 2017.

[1]: http://www.grantjenks.com/docs/sortedcontainers/pycon-2016-talk.html

> I recommend to read thorough review articles written by Andrew Barnert:
>
> http://stupidpythonideas.blogspot.com/2013/07/sorted-collections-in-stdlib.html
>
> http://stupidpythonideas.blogspot.com/2014/04/sortedcontainers.html

One of Andrew Barnert's conclusions is that SortedContainers could not
scale. I did a pretty rigorous performance analysis and benchmarking
at http://www.grantjenks.com/docs/sortedcontainers/performance-scale.html
Short answer: I scaled SortedContainers up through ten billion
elements, well past the memory limits of most machines.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Deprecate metaclasses that are not instances of type

[Neil Girdhar]

Bug: http://bugs.python.org/issue28437

On Thu, Oct 13, 2016 at 7:15 PM Neil Girdhar  wrote:

> That's fair.  However, the documentation should at least be repaired by
> replacing section 3.3.3.2 with:
>
> "The metaclass of a class definition is selected from the explicitly
> specified metaclass (if any) and the metaclasses (i.e. type(cls)) of all
> specified base classes. The most derived metaclass is one which is a
> subtype of all of these candidate metaclasses. If none of the candidate
> metaclasses meets that criterion, then the class definition will fail with
> TypeError. If provided, the explicit metaclass must be a callable accepting
> the positional arguments (name, bases, _dict)."
>
> This is because something happened along the way and Objects/typeobject.c:
> type_new no longer coincides with Lib/types.py:new_class.  The Python
> version conditionally calls _calculate_meta whereas the C version calls it
> unconditionally.  I consider the C implementation to be the "correct"
> version.
>
> Best,
>
> Neil
>
> On Thu, Oct 13, 2016 at 5:41 PM Steven D'Aprano 
> wrote:
>
> On Thu, Oct 13, 2016 at 01:46:34PM -0700, Neil Girdhar wrote:
>
> > If provided, the explicit metaclass must be an instance of
> > type()."
>
> -1 for pointless breakage.
>
> The metaclass has always been permitted to be any callable. You haven't
> given any good reason for gratuitously changing this.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/wrHDM0SOIqE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Deprecate metaclasses that are not instances of type

[Neil Girdhar]

That's fair.  However, the documentation should at least be repaired by
replacing section 3.3.3.2 with:

"The metaclass of a class definition is selected from the explicitly
specified metaclass (if any) and the metaclasses (i.e. type(cls)) of all
specified base classes. The most derived metaclass is one which is a
subtype of all of these candidate metaclasses. If none of the candidate
metaclasses meets that criterion, then the class definition will fail with
TypeError. If provided, the explicit metaclass must be a callable accepting
the positional arguments (name, bases, _dict)."

This is because something happened along the way and Objects/typeobject.c:
type_new no longer coincides with Lib/types.py:new_class.  The Python
version conditionally calls _calculate_meta whereas the C version calls it
unconditionally.  I consider the C implementation to be the "correct"
version.

Best,

Neil

On Thu, Oct 13, 2016 at 5:41 PM Steven D'Aprano  wrote:

> On Thu, Oct 13, 2016 at 01:46:34PM -0700, Neil Girdhar wrote:
>
> > If provided, the explicit metaclass must be an instance of
> > type()."
>
> -1 for pointless breakage.
>
> The metaclass has always been permitted to be any callable. You haven't
> given any good reason for gratuitously changing this.
>
>
>
> --
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
> --
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "python-ideas" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/python-ideas/wrHDM0SOIqE/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> python-ideas+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add sorted (ordered) containers

[Neil Girdhar]

Those are great articles.  One thing that Andrew does recommend would be to 
standardize the interface to the sorted containers, and add them to 
collections.abc as SortedDict, and SortedSet.

I recently switched from blist to sortedcontainers and it would be nice to 
have these standardized going forward.

Another reason to standardize them is for use with the new type checking.

Best,

Neil

On Thursday, October 13, 2016 at 5:54:09 PM UTC-4, Serhiy Storchaka wrote:
>
> On 13.10.16 23:36, Марк Коренберг wrote:
> > I think it should be one standardized implementation of such containers
> > in CPython.
> >
> > For example, C++ has both ordered_map and unorderd_map.
> >
> > Instead of trees, implementation may use SkipList structure, but this is
> > just implementation details.
> >
> > Such structres imply fast insertion and deletion, ability to iterate,
> > and also memory efficiency.
>
> I recommend to read thorough review articles written by Andrew Barnert:
>
>
> http://stupidpythonideas.blogspot.com/2013/07/sorted-collections-in-stdlib.html
>
> http://stupidpythonideas.blogspot.com/2014/04/sortedcontainers.html
>
>
> ___
> Python-ideas mailing list
> python...@python.org 
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 10:40:19PM +0200, Sjoerd Job Postmus wrote:

> Likewise,
> 
> l = [f(t) for t in iterable]
> 
> can be seen as sugar for
> 
> def gen():
> for t in iterable:
> yield f(t)
> l = list(gen())

But that is *not* how list comprehensions are treated today. Perhaps 
they should be?

https://docs.python.org/3.6/reference/expressions.html#displays-for-lists-sets-and-dictionaries


(Aside: earlier I contrasted "list display" from "list comprehension". 
In fact according to the docs, a comprehension is a kind of display, a 
special case of display. Nevertheless, my major point still holds: a 
list display like [1, 2, 3] is not the same as a list comprehension like 
[a+1 for a in (0, 1, 2)].)

There may be some conceptual benefits to switching to a model where 
list/set/dict displays are treated as list(gen_expr) etc. But that still 
leaves the question of what "yield *t" is supposed to mean?

Consider the analogy with f(*t), where t = (a, b, c). We *don't* have:

f(*t) is equivalent to f(a); f(b); f(c)

So why would yield *t give us this?

yield a; yield b; yield c

By analogy with the function call syntax, it should mean:

yield (a, b, c)

That is, t is unpacked, then repacked to a tuple, then yielded.


 
> Now the suggested spelling
> 
> l = [*f(t) for t in iterable]
> 
> is very confusing, from what I understand: what does the `*` even mean
> here.

Indeed. The reader may be forgiven for thinking that this is yet another 
unrelated and arbitrary use of * to join the many other uses:

- mathematical operator;
- glob and regex wild-card;
- unpacking;
- import all
- and now yield from


> However, consider the following spelling:
> 
> l = [from f(t) for t in iterable]
> 
> To me, it does not seem far-fetched that this would mean:
> 
> def gen():
> for t in iterable:
> yield from f(t)
> l = list(gen())


Now we're starting to move towards a reasonable proposal. It still 
requires a conceptual shift in how list comprehensions are documented, 
but at least now the syntax is no longer so arbitrary.


-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

Trying to restate the proposal, somewhat more formal following Random832
and Paul's suggestion.

I only speak about the single star.
---

*The suggested change of syntax:*

comprehension ::=  starred_expression comp_for

*Semantics:*

(In the following, f(x) must always evaluate to an iterable)

1. List comprehension:

result = [*f(x) for x in iterable if cond]

Translates to

result = []
for x in iterable:
if cond:
result.extend(f(x))

2. Set comprehension:

result = {*f(x) for x in iterable if cond}

Translates to

result = set()
for x in iterable:
if cond:
result.update(f(x))

3. Generator expression:

(*f(x) for x in iterable if cond)

Translates to

for x in iterable:
if cond:
yield from f(x)

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add sorted (ordered) containers

[Serhiy Storchaka]

On 13.10.16 23:36, Марк Коренберг wrote:

I think it should be one standardized implementation of such containers
in CPython.

For example, C++ has both ordered_map and unorderd_map.

Instead of trees, implementation may use SkipList structure, but this is
just implementation details.

Such structres imply fast insertion and deletion, ability to iterate,
and also memory efficiency.


I recommend to read thorough review articles written by Andrew Barnert:

http://stupidpythonideas.blogspot.com/2013/07/sorted-collections-in-stdlib.html

http://stupidpythonideas.blogspot.com/2014/04/sortedcontainers.html


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Suggestion: Deprecate metaclasses that are not instances of type

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 01:46:34PM -0700, Neil Girdhar wrote:

> If provided, the explicit metaclass must be an instance of 
> type()."

-1 for pointless breakage.

The metaclass has always been permitted to be any callable. You haven't 
given any good reason for gratuitously changing this.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Thu, Oct 13, 2016, at 16:59, Paul Moore wrote:
> I don't (for the reasons raised before). But thank you for your
> explanation, it clarifies what you were proposing. And it does so
> within the *current* uses of the * symbol, which is good. But:
> 
> 1. I'm not keen on extending append's meaning to overlap with extend's
> like this.

I think the "append(*x)" bit was just a flourish to try to explain it in
terms of the current use of * since you don't seem to understand it any
other way, rather than an actual proposal to actually change the append
method.

> 2. Your proposal does not generalise to generator expressions, set
> displays (without similarly modifying the set.add() method) or
> dictionary displays.

Basically it would make the following substitutions in the conventional
"equivalent loops"
generator yield => yield from
list append => extend
set add => update
dict __setitem__ => update

dict comprehensions would need to use **x - {*x for x in y} would be a
set comprehension.

> 3. *fn(x) isn't an expression, and yet it *looks* like it should be,
> and in the current syntax, an expression is required in that position.
> To me, that suggests it would be hard to teach. [1]

I can think of another position an expression used to be required in:

Python 3.5.2
>>> [1, *(2, 3), 4]
[1, 2, 3, 4]

Python 2.7.11
>>> [1, *(2, 3), 4]
  File "", line 1
[1, *(2, 3), 4]
^
SyntaxError: invalid syntax

Was that hard to teach? Maybe. But it's a bit late to object now, and
every single expression on the right hand side in my examples below
already has a meaning.


Frankly, I don't see why the pattern isn't obvious [and why people keep
assuming there will be a new meaning of f(*x) as if it doesn't already
have a meaning]

Lists, present:
   [x for x in [a, b, c]] == [a, b, c]
   [f(x) for x in [a, b, c]] == [f(a), f(b), f(c)]
   [f(*x) for x in [a, b, c]] == [f(*a), f(*b), f(*c)]
   [f(**x) for x in [a, b, c]] == [f(**a), f(**b), f(**c)]
Lists, future:
   [*x for x in [a, b, c]] == [*a, *b, *c]
   [*f(x) for x in [a, b, c]] == [*f(a), *f(b), *f(c)]
   [*f(*x) for x in [a, b, c]] == [*f(*a), *f(*b), *f(*c)]
   [*f(**x) for x in [a, b, c]] == [*f(**a), *f(**b), *f(**c)]

Sets, present:
   {x for x in [a, b, c]} == {a, b, c}
   {f(x) for x in [a, b, c]} == {f(a), f(b), f(c)}
   {f(*x) for x in [a, b, c]} == {f(*a), f(*b), f(*c)}
   {f(**x) for x in [a, b, c]} == {f(**a), f(**b), f(**c)}
Sets, future:
   {*x for x in [a, b, c]} == {*a, *b, *c}
   {*f(x) for x in [a, b, c]} == {*f(a), *f(b), *f(c)}
   {*f(*x) for x in [a, b, c]} == {*f(*a), *f(*b), *f(*c)}
   {*f(**x) for x in [a, b, c]} == {*f(**a), *f(**b), *f(**c)}

Dicts, future:
   {**x for x in [a, b, c]} == {**a, **b, **c}
   {**f(x) for x in [a, b, c]} == {**f(a), **f(b), **f(c)}
   {**f(*x) for x in [a, b, c]} == {**f(*a), **f(*b), **f(*c)}
   {**f(**x) for x in [a, b, c]} == {**f(**a), **f(**b), **f(**c)}
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Suggestion: Deprecate metaclasses that are not instances of type

[Neil Girdhar]

Background: I asked a stackoverflow question here 

.

The Python documentation 

 
is very confusing to me. It says that:

if an explicit metaclass is given and it is not an instance of type, then 
it is used directly as the metaclass

This seems to suggest that in this case, the "explicit metaclass" does not 
need to be "subtype of all of these candidate metaclasses" as it would in 
the third case.  (This is not true.)

Also, providing a callable as a metaclass doesn't seem to be any more 
flexible, readable, or powerful than providing an instance of type. 
 Therefore, I suggest that we deprecate the second case and replace the 
entire section (3.3.3.2) of the documentation to say:

"The metaclass of a class definition is selected from the explicitly 
specified metaclass (if any) and the metaclasses (i.e. type(cls)) of all 
specified base classes. The most derived metaclass is one which is a 
subtype of all of these candidate metaclasses. If none of the candidate 
metaclasses meets that criterion, then the class definition will fail with 
TypeError. If provided, the explicit metaclass must be an instance of 
type()."
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Add sorted (ordered) containers

[Марк Коренберг]

I mean mutable containers that are always sorted when iterating over them.

See http://bugs.python.org/issue28433

for example:

* SortedSet (sorted unique elements, implemented using (rb?)tree instead of 
hash)
* SortedList (sorted elements, the same as SortedSet, but without 
uniquiness constraint) - actually a (rb?)tree, not a list (i.e. not an 
array)
* SortedDict (sorted by key when interating) - like C++'s ordered_map

There are many implementations in the net, like:

https://bitbucket.org/bcsaller/rbtree
http://newcenturycomputers.net/projects/rbtree.html
https://sourceforge.net/projects/pyavl
http://www.grantjenks.com/docs/sortedcontainers
https://github.com/tailhook/sortedsets
https://pypi.python.org/pypi/skiplist

and also in pip:

pip3 search sorted | grep -Ei '[^a-z]sorted'

I think it should be one standardized implementation of such containers in 
CPython.

For example, C++ has both ordered_map and unorderd_map.

Instead of trees, implementation may use SkipList structure, but this is 
just implementation details.

Such structres imply fast insertion and deletion, ability to iterate, and 
also memory efficiency.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Fri, Oct 14, 2016 at 12:06 AM Neil Girdhar  wrote:


> Regarding Steven's example, like Sven, I also see it this way:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
> should mean:
>
>[*(1, 'a'), *(2, 'b'), *(3, 'c')]]
>
> Which coincides with what the OP is asking for.
>

>
>From a CPython implementation standpoint, we specifically blocked this code
> path, and it is only a matter of unblocking it if we want to support this.
>
>
This is *very, very* not surprising. And should be stressed.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 11:59 PM Paul Moore  wrote:

> On 13 October 2016 at 21:47, אלעזר  wrote:
> > if you allow result.append(1, 2, 3) to mean result.extend([1,2,3])  #
> which
> > was discussed before
>
> I don't (for the reasons raised before). But thank you for your
> explanation, it clarifies what you were proposing. And it does so
> within the *current* uses of the * symbol, which is good. But:
>
> 1. I'm not keen on extending append's meaning to overlap with extend's
> like this.
> 2. Your proposal does not generalise to generator expressions, set
> displays (without similarly modifying the set.add() method) or
> dictionary displays.
> 3. *fn(x) isn't an expression, and yet it *looks* like it should be,
> and in the current syntax, an expression is required in that position.
> To me, that suggests it would be hard to teach. [1]
>
> You can of course generalise Sjoerd's "from" proposal and then just
> replace "from" with "*" throughout. That avoids your requirement to
> change append, but at the cost of the translation no longer being a
> parallel to an existing use of "*".
>
>
I think it is an unfortunate accident of syntax, the use of "yield from
foo()" instead of "yield *foo()". These "mean" the same: a syntactic
context that directly handles iterable as repetition, (with some guarantees
regarding exceptions etc.). Alternatively, we could be writing [1, 2, from
[3, 4], 5, 6]. Whether it is "from x" or "*x" is just an accident. In my
mind.

As you said, the proposal should be written in a much more formal way, so
that it could be evaluated without confusion. I completely agree.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Neil Girdhar]

First of all:

+1 to Sven's very well-expressed support of the proposal, and
+1 to Nick's very well-explained reasons for rejecting it.

As one of the main implementers of PEP 448, I have always liked this, but I 
suggested that we leave this out when there was opposition since there's no 
rush for it.

Regarding Steven's example, like Sven, I also see it this way:

[*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]] 

should mean:

   [*(1, 'a'), *(2, 'b'), *(3, 'c')]] 

Which coincides with what the OP is asking for.

At the end of this discussion it might be good to get a tally of how many 
people think the proposal is reasonable and logical.  I imagine people will 
be asking this same question next year and the year after, and so it will 
be good to see if as familiarity with PEP 448 expands, more people will 
find this intuitive and logical.

>From a CPython implementation standpoint, we specifically blocked this code 
path, and it is only a matter of unblocking it if we want to support this.

Best,

Neil
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Thu, Oct 13, 2016, at 16:42, Paul Moore wrote:
> I remain puzzled.
> 
> Given the well-documented and understood transformation:
> 
> [fn(x) for x in lst if cond]
> 
> translates to
> 
> result = []
> for x in lst:
>if cond:
>   result.append(fn(x))
>
> please can you explain how to modify that translation rule to
> incorporate the suggested syntax?

In this case * would change this to result.extend (or +=)

just as result = [a, *b, c] is equivalent to:
result = []
result.append(a)
result.extend(b)
result.append(c)

result = [*x for x in lst if cond] would become:
result = []
for x in lst:
   if cond:
  result.extend(x)

I used yield from as my original example to include generator
expressions, which should also support this.

> Personally, I'm not even sure any more that I can *describe* the
> suggested syntax. Where in [fn(x) for x in lst if cond] is the *
> allowed? fn(*x)?

This already has a meaning, so it's obviously "allowed", but not in a
way relevant to this proposal. The elements of x are passed to fn as
arguments rather than being inserted into the list. Ultimately the
meaning is the same.

> *fn(x)? Only as *x with a bare variable, but no expression?

Both of these would be allowed. Any expression would be allowed, but at
runtime its value must be iterable, the same as other places that you
can use *x.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 13 October 2016 at 21:47, אלעזר  wrote:
> if you allow result.append(1, 2, 3) to mean result.extend([1,2,3])  # which
> was discussed before

I don't (for the reasons raised before). But thank you for your
explanation, it clarifies what you were proposing. And it does so
within the *current* uses of the * symbol, which is good. But:

1. I'm not keen on extending append's meaning to overlap with extend's
like this.
2. Your proposal does not generalise to generator expressions, set
displays (without similarly modifying the set.add() method) or
dictionary displays.
3. *fn(x) isn't an expression, and yet it *looks* like it should be,
and in the current syntax, an expression is required in that position.
To me, that suggests it would be hard to teach. [1]

You can of course generalise Sjoerd's "from" proposal and then just
replace "from" with "*" throughout. That avoids your requirement to
change append, but at the cost of the translation no longer being a
parallel to an existing use of "*".

Paul

[1] On a purely personal note, I'd say it's confusing, but I don't
want to go back to subjective arguments, so I only note that here as
an opinion, not an argument.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 13 October 2016 at 21:40, Sjoerd Job Postmus  wrote:
> However, consider the following spelling:
>
> l = [from f(t) for t in iterable]
>
> To me, it does not seem far-fetched that this would mean:
>
> def gen():
> for t in iterable:
> yield from f(t)
> l = list(gen())

Thank you. This is the type of precise definition I was asking for in
my previous post (your timing was superb!)

I'm not sure I *like* the proposal, but I need to come up with some
reasonable justification for my feeling, whereas for previous
proposals the "I don't understand what you're suggesting" was the
overwhelming feeling, and stifled any genuine discussion of merits or
downsides.

Paul

PS I can counter a suggestion of using *f(t) rather than from f(t) in
the above, by saying that it adds yet another meaning to the already
heavily overloaded * symbol. The suggestion of "from" avoids this as
"from" only has a few meanings already. (You can agree or disagree
with my view, but at least we're debating the point objectively at
that point!)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add sorted (ordered) containers

[Neil Girdhar]

Related:

Nick posted an excellent answer to this question here: 
 
http://stackoverflow.com/questions/5953205/why-are-there-no-sorted-containers-in-pythons-standard-libraries

On Thursday, October 13, 2016 at 4:36:39 PM UTC-4, Марк Коренберг wrote:
>
> I mean mutable containers that are always sorted when iterating over them.
>
> See http://bugs.python.org/issue28433
>
> for example:
>
> * SortedSet (sorted unique elements, implemented using (rb?)tree instead 
> of hash)
> * SortedList (sorted elements, the same as SortedSet, but without 
> uniquiness constraint) - actually a (rb?)tree, not a list (i.e. not an 
> array)
> * SortedDict (sorted by key when interating) - like C++'s ordered_map
>
> There are many implementations in the net, like:
>
> https://bitbucket.org/bcsaller/rbtree
> http://newcenturycomputers.net/projects/rbtree.html
> https://sourceforge.net/projects/pyavl
> http://www.grantjenks.com/docs/sortedcontainers
> https://github.com/tailhook/sortedsets
> https://pypi.python.org/pypi/skiplist
>
> and also in pip:
>
> pip3 search sorted | grep -Ei '[^a-z]sorted'
>
> I think it should be one standardized implementation of such containers in 
> CPython.
>
> For example, C++ has both ordered_map and unorderd_map.
>
> Instead of trees, implementation may use SkipList structure, but this is 
> just implementation details.
>
> Such structres imply fast insertion and deletion, ability to iterate, and 
> also memory efficiency.
>
>___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 11:42 PM Paul Moore  wrote:

> I remain puzzled.
>
> Given the well-documented and understood transformation:
>
> [fn(x) for x in lst if cond]
>
> translates to
>
> result = []
> for x in lst:
>if cond:
>   result.append(fn(x))
>
> please can you explain how to modify that translation rule to
> incorporate the suggested syntax?
>

if you allow result.append(1, 2, 3) to mean result.extend([1,2,3])  # which
was discussed before

result = []
for x in lst:
   if cond:
  result.append(*fn(x))

Or simply use result.extend([*fn(x)])

Personally, I'm not even sure any more that I can *describe* the
> suggested syntax. Where in [fn(x) for x in lst if cond] is the *
> allowed? fn(*x)? *fn(x)? Only as *x with a bare variable, but no
> expression? Only in certain restricted types of construct which aren't
> expressions but are some variation on an unpacking construct?
>
>
The star is always exactly at the place that should "handle" it. which
means [*(fn(x)) for x in lst if cond]. fn(x) must be iterable as always.


> We've had a lot of examples. I think it's probably time for someone to
> describe the precise syntax (as BNF, like the syntax in the Python
> docs at
> https://docs.python.org/3.6/reference/expressions.html#displays-for-lists-sets-and-dictionaries
> and following sections) and semantics (as an explanation of how to
> rewrite any syntactically valid display as a loop). It'll have to be
> done in the end, as part of any implementation, so why not now?
>
>
I will be happy to do so, and will be happy to work with anyone else
interested.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 13 October 2016 at 20:51, Random832  wrote:
> On Thu, Oct 13, 2016, at 15:46, Random832 wrote:
>> so,  under a similar 'transformation', "*foo for foo in bar" likewise
>> becomes "def f(): for foo in bar: yield from foo"
>>
>> bar = [(1, 2), (3, 4)]
>> (*(1, 2), *(3, 4)) == == tuple(f())
>> [*(1, 2), *(3, 4)] == == list(f())
>
>
> I accidentally hit ctrl-enter while copying and pasting, causing my
> message to go out while my example was less thorough than intended and
> containing syntax errors. It was intended to read as follows:
>
> ..."*foo for foo in bar" likewise becomes
>
> def f():
> for foo in bar:
> yield from foo
>
> a, b = (1, 2), (3, 4)
> bar = [a, b]
> (*a, *b) == (1, 2, 3, 4) == tuple(f()) # tuple(*foo for foo in bar)
> [*a, *b] == [1, 2, 3, 4] == list(f()) # [*foo for foo in bar]

I remain puzzled.

Given the well-documented and understood transformation:

[fn(x) for x in lst if cond]

translates to

result = []
for x in lst:
   if cond:
  result.append(fn(x))

please can you explain how to modify that translation rule to
incorporate the suggested syntax?

Personally, I'm not even sure any more that I can *describe* the
suggested syntax. Where in [fn(x) for x in lst if cond] is the *
allowed? fn(*x)? *fn(x)? Only as *x with a bare variable, but no
expression? Only in certain restricted types of construct which aren't
expressions but are some variation on an unpacking construct?

We've had a lot of examples. I think it's probably time for someone to
describe the precise syntax (as BNF, like the syntax in the Python
docs at 
https://docs.python.org/3.6/reference/expressions.html#displays-for-lists-sets-and-dictionaries
and following sections) and semantics (as an explanation of how to
rewrite any syntactically valid display as a loop). It'll have to be
done in the end, as part of any implementation, so why not now?

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Sjoerd Job Postmus]

After having followed this thread for a while, it occured to me that the
reason that the idea is confusing, is because the spelling is confusing.

I think the suggested spelling (`*`) is the confusing part. If it were
to be spelled `from ` instead, it would be less confusing.

Consider this:

g = (f(t) for t in iterable)

is "merely" sugar for

def gen():
for t in iterable:
yield f(t)
g = gen()

Likewise,

l = [f(t) for t in iterable]

can be seen as sugar for

def gen():
for t in iterable:
yield f(t)
l = list(gen())

Now the suggested spelling

l = [*f(t) for t in iterable]

is very confusing, from what I understand: what does the `*` even mean
here.

However, consider the following spelling:

l = [from f(t) for t in iterable]

To me, it does not seem far-fetched that this would mean:

def gen():
for t in iterable:
yield from f(t)
l = list(gen())

It follows the "rule" quite well: given a generator display, everything
before the first "for" gets placed after "yield ", and all the
`for`/`if`s are expanded to suites.

Now I'm not sure if I'm a fan of the idea, but I think that at least the
`from `-spelling is less confusing than the `*`-spelling.

(Unless I totally misunderstood what the `*`-spelling was about, given
how confusing it supposedly is. Maybe it confused me.)

On Fri, Oct 14, 2016 at 03:55:46AM +1100, Steven D'Aprano wrote:
> On Thu, Oct 13, 2016 at 04:34:49PM +0200, Martti Kühne wrote:
> 
> > > If I had seen a list comprehension with an unpacked loop variable:
> > >
> > > [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
> 
> Marttii, somehow you have lost the leading * when quoting me. What I 
> actually wrote was:
> 
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
> 
> 
> > As it happens, python does have an external consumption operation that
> > happens externally with an iteration implied:
> > 
> > for t in iterable:
> > yield t
> 
> If you replace the t with *t, you get a syntax error:
> 
> 
> py> def gen():
> ... for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
> ... yield *t
>   File "", line 3
> yield *t
>   ^
> SyntaxError: invalid syntax
> 
> Even if it was allowed, what would it mean? It could only mean "unpack 
> the sequence t, and collect the values into a tuple; then yield the 
> tuple".
> 
> 
>  
> > For your example [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]] that would 
> > mean:
> > 
> > for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
> > yield t
> > 
> > And accordingly, for the latter case [*t for t in [(1, 'a'), (2, 'b'),
> > (3, 'c')]] it would be:
> > 
> > for item in [(1, 'a'), (2, 'b'), (3, 'c')]:
> > for t in item:
> > yield t
> 
> No it wouldn't. Where does the second for loop come from? The list 
> comprehension shown only has one loop, not nested loops.
> 
> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Thu, Oct 13, 2016, at 15:46, Random832 wrote:
> so,  under a similar 'transformation', "*foo for foo in bar" likewise
> becomes "def f(): for foo in bar: yield from foo"
> 
> bar = [(1, 2), (3, 4)]
> (*(1, 2), *(3, 4)) == == tuple(f())
> [*(1, 2), *(3, 4)] == == list(f())


I accidentally hit ctrl-enter while copying and pasting, causing my
message to go out while my example was less thorough than intended and
containing syntax errors. It was intended to read as follows:

..."*foo for foo in bar" likewise becomes

def f():
for foo in bar:
yield from foo

a, b = (1, 2), (3, 4)
bar = [a, b]
(*a, *b) == (1, 2, 3, 4) == tuple(f()) # tuple(*foo for foo in bar)
[*a, *b] == [1, 2, 3, 4] == list(f()) # [*foo for foo in bar]
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Random832]

On Thu, Oct 13, 2016, at 14:50, David Mertz wrote:
> Neither of those follows conventional Python semantics for function
> calling
> or sequence unpacking.  So maybe that remains a type error or syntax
> error.  But then we exclude a very common pattern of using comprehensions
> to create collections of *transformed* data, not simply of filtered data.

[*map(math.exp, t) for t in [(1, 2), (3, 4)]]

[*(math.exp(x) for x in t) for t in [(1, 2), (3, 4)]]

I think "excluding" is a bit of a strong word - just because something
doesn't address a mostly unrelated need doesn't mean it doesn't have any
merit in its own right. Not every proposal is going to do everything. I
think the key is that the person originally asking this thought of *x as
a generalized "yield from x"-ish thing, for example:

"a, *b, c" becomes "def f(): yield a; yield from b; yield c;"

[a, *b, c] == list(f())
(a, *b, c) == tuple(f())

so,  under a similar 'transformation', "*foo for foo in bar" likewise
becomes "def f(): for foo in bar: yield from foo"

bar = [(1, 2), (3, 4)]
(*(1, 2), *(3, 4)) == == tuple(f())
[*(1, 2), *(3, 4)] == == list(f())


> In contrast, either of these are unambiguous and obvious:
> 
> [math.exp(t) for t in flatten([(1,2),(3,4)])]
> 
> Or:
> 
> [math.exp(n) for t in [(1,2),(3,4)] for n in t]
> 
> Obviously, picking math.exp() is arbitrary and any unary function would
> be
> the same issue.
> 
> 
> -- 
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[David Mertz]

>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>

Another problem with this is that it is very hard to generalize to the case
where the item included in a comprehension is a transformation on iterated
values.  E.g. what does this do?

[math.exp(*t) for t in [(1,2),(3,4)]]

Maybe that somehow magically gets us:

[2.7182, 7.38905, 20.0855, 54.5981]

Or maybe the syntax would be:

[*math.exp(t) for t in [(1,2),(3,4)]]

Neither of those follows conventional Python semantics for function calling
or sequence unpacking.  So maybe that remains a type error or syntax
error.  But then we exclude a very common pattern of using comprehensions
to create collections of *transformed* data, not simply of filtered data.

In contrast, either of these are unambiguous and obvious:

[math.exp(t) for t in flatten([(1,2),(3,4)])]

Or:

[math.exp(n) for t in [(1,2),(3,4)] for n in t]

Obviously, picking math.exp() is arbitrary and any unary function would be
the same issue.


-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[David Mertz]

Exactly with Paul!

As I mentioned, I teach software developers and scientists Python for a
living.  I get paid a lot of money to do that, and have a good sense of
what learners can easily understand and not (I've also written hundred of
articles and a few books about Python).  The people I write for and teach
are educated, smart, and generally have familiarity with multiple
programming languages.

In my opinion, this new construct—if added to the language—would be
difficult to teach, and most of my students would get it wrong most of the
time.

Yes, I understand the proposed semantics.  It is not *intuitive* to me, but
I could file the rule about the behavior if I had to.  But if I were forced
to teach it, it would always be "Here's a Python wart to look out for if
you see it in other code... you should not ever use it yourself."

On Thu, Oct 13, 2016 at 8:18 AM, Paul Moore  wrote:

> On 13 October 2016 at 15:32, Sven R. Kunze  wrote:
> > Steven, please. You seemed to struggle to understand the notion of the
> > [*] construct and many people (not just me) here tried their best to
> > explain their intuition to you.
>
> And yet, the fact that it's hard to explain your intuition to others
> (Steven is not the only one who's finding this hard to follow) surely
> implies that it's merely that - personal intuition - and not universal
> understanding.
>
> The *whole point* here is that not everyone understands the proposed
> notation the way the proposers do, and it's *hard to explain* to those
> people. Blaming the people who don't understand does not support the
> position that this notation should be added to the language. Rather,
> it reinforces the idea that the new proposal is hard to teach (and
> consequently, it may be a bad idea for Python).
>
> Paul
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>



-- 
Keeping medicines from the bloodstreams of the sick; food
from the bellies of the hungry; books from the hands of the
uneducated; technology from the underdeveloped; and putting
advocates of freedom in prisons.  Intellectual property is
to the 21st century what the slave trade was to the 16th.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Martti Kühne]

On Thu, Oct 13, 2016 at 6:55 PM, Steven D'Aprano  wrote:
> On Thu, Oct 13, 2016 at 04:34:49PM +0200, Martti Kühne wrote:
>
>> > If I had seen a list comprehension with an unpacked loop variable:
>> >
>> > [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
> Martti, somehow you have lost the leading * when quoting me. What I
> actually wrote was:
>
> [*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>

Sorry for misquoting you. Can I fix my name, though?
Also, this mail was too long in my outbox so the context was lost on it.
I reiterate it, risking that I would annoy some, but to be absolutely clear.

>
>> As it happens, python does have an external consumption operation that
>> happens externally with an iteration implied:
>>
>
> If you replace the t with *t, you get a syntax error:
>

I meant that statement in context of the examples which were brought up:
the occurrence of a list comprehension inside an array have the
following effect:

1) [ ..., [expr for t in iterable] ]

is equivalent to:

def expr_long(iterable, result):
result.append(iterable)
return result

expr_long(iterable, [ ..., ])

so, if you make the case for pep448, you might arrive at the following:

2) [ ..., *[expr for expr in iterable] ]

which would be, if I'm typing it correctly, equivalent to, what
resembles an external collection:

def expr_star(list_comp, result):
result.extend(list(list_comp))
return result

expr_star(iterable, [ ..., ])

Having this in mind, the step to making:

[ ..., [*expr for expr in iterable], ]

from:

def expr_insidestar(iterable, result):
for expr in iterable:
result.extend(expr)
return result

does not appear particularly far-fetched, at least not to me and a few
people on this list.

cheers!
mar77i
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Ned Batchelder]

On 10/13/16 2:42 AM, Mikhail V wrote:
> On 13 October 2016 at 08:02, Greg Ewing  wrote:
>> Mikhail V wrote:
>>> Consider unicode table as an array with glyphs.
>>
>> You mean like this one?
>>
>> http://unicode-table.com/en/
>>
>> Unless I've miscounted, that one has the characters
>> arranged in rows of 16, so it would be *harder* to
>> look up a decimal index in it.
>>
>> --
>> Greg
> Nice point finally, I admit, although quite minor. Where
> the data implies such pagings or alignment, the notation
> should be (probably) more binary-oriented.
> But: you claim to see bit patterns in hex numbers? Then I bet you will
> see them much better if you take binary notation (2 symbols) or quaternary
> notation (4 symbols), I guarantee. And if you take consistent glyph set for 
> them
> also you'll see them twice better, also guarantee 100%.
> So not that the decimal is cool,
> but hex sucks (too big alphabet) and _the character set_ used for hex
> optically sucks.
> That is the point.
> On the other hand why would unicode glyph table which is to the
> biggest part a museum of glyphs would be necesserily
> paged in a binary-friendly manner and not in a decimal friendly
> manner? But I am not saying it should or not, its quite irrelevant
> for this particular case I think.

You continue to overlook the fact that Unicode codepoints are
conventionally presented in hexadecimal, including in the page you
linked us to.  This is the convention.  It makes sense to stick to the
convention. 

When I see a numeric representation of a character, there are only two
things I can do with it: look it up in a reference someplace, or glean
some meaning from it directly.  For looking things up, please remember
that all Unicode references use hex numbering. Looking up a character by
decimal numbers is simply more difficult than looking them up by hex
numbers.

For gleaning meaning directly, please keep in mind that Unicode
fundamentally structured around pages of 256 code points, organized into
planes of 256 pages.  The very structure of how code points are
allocated and grouped is based on a hexadecimal-friendly system.  The
blocks of codepoints are aligned on hexadecimal boundaries:
http://www.fileformat.info/info/unicode/block/index.htm .  When I see
\u0414, I know it is a Cyrillic character because it is in block 04xx.

It simply doesn't make sense to present Unicode code points in anything
other than hex.

--Ned.


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 04:34:49PM +0200, Martti Kühne wrote:

> > If I had seen a list comprehension with an unpacked loop variable:
> >
> > [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]

Marttii, somehow you have lost the leading * when quoting me. What I 
actually wrote was:

[*t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]


> As it happens, python does have an external consumption operation that
> happens externally with an iteration implied:
> 
> for t in iterable:
> yield t

If you replace the t with *t, you get a syntax error:


py> def gen():
... for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
... yield *t
  File "", line 3
yield *t
  ^
SyntaxError: invalid syntax

Even if it was allowed, what would it mean? It could only mean "unpack 
the sequence t, and collect the values into a tuple; then yield the 
tuple".


 
> For your example [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]] that would mean:
> 
> for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
> yield t
> 
> And accordingly, for the latter case [*t for t in [(1, 'a'), (2, 'b'),
> (3, 'c')]] it would be:
> 
> for item in [(1, 'a'), (2, 'b'), (3, 'c')]:
> for t in item:
> yield t

No it wouldn't. Where does the second for loop come from? The list 
comprehension shown only has one loop, not nested loops.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 03:28:27PM +, אלעזר wrote:

> It may also suggest that there are currently two ways to understand the
> *[...] construct, 

This thread is about allowing sequence unpacking as the internal 
expression of list comprehensions:

[ *(expr) for x in iterable ]

It isn't about unpacking lists:

*[...]

so I don't see what relevance your comment has.

There may be two or three or ten or 100 ways to (mis)understand list 
comprehensions in Python, but only one of them is the correct way. List 
comprehensions are (roughly) syntactic sugar for:

result = []
for x in iterable:
result.append(expression)

Any other understanding of them is incorrect.

Now if people wish to make an argument for changing the meaning of 
comprehensions so that the suggested internal unpacking makes sense, 
then by all means try making that argument! That's absolutely fine.

In the past, I've tried a similar thing: I argued for a variant list 
comprehension that halts early:

[expr for x in iterable while condition]

(as found in at least one other language), but had that knocked back 
because it doesn't fit the existing list comprehension semantics. I 
wasn't able to convince people that the value of this new comprehension 
was worth breaking the existing semantics of comprehensions.

Maybe you will be able to do better than me.

But understand that:

[*(expr) for x in iterable] 

also fails to fit the existing list comprehension semantics. To make it 
work requires changing the meaning of Python list comps. It isn't enough 
to just deny the existing meaning and insist that your own personal 
meaning is correct.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 6:19 PM Paul Moore  wrote:

> On 13 October 2016 at 15:32, Sven R. Kunze  wrote:
> > Steven, please. You seemed to struggle to understand the notion of the
> > [*] construct and many people (not just me) here tried their best to
> > explain their intuition to you.
>
> And yet, the fact that it's hard to explain your intuition to others
> (Steven is not the only one who's finding this hard to follow) surely
> implies that it's merely that - personal intuition - and not universal
> understanding.
>
>
I fail to see this implication. Perhaps you mean that the universal
understanding is hard to get, intuitively. And trying to explain them is
the way to figure out howw hard can this difficulty be overcome.


> The *whole point* here is that not everyone understands the proposed
> notation the way the proposers do, and it's *hard to explain* to those
> people. Blaming the people who don't understand does not support the
> position that this notation should be added to the language. Rather,
> it reinforces the idea that the new proposal is hard to teach (and
> consequently, it may be a bad idea for Python).
>
>
It may also suggest that there are currently two ways to understand the
*[...] construct, and only one of them can be generalized to lead the new
proposal. So people that are *used* to the other way may have harder time
than people coming with a clean slate. So it might or might not be hard to
teach.

(I'm not saying that's necessarily the case)

I will be happy to understand that "other way" that is harder to
generalize; I think this discussion may be fruitful in making these
different understandings explicit.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Paul Moore]

On 13 October 2016 at 15:32, Sven R. Kunze  wrote:
> Steven, please. You seemed to struggle to understand the notion of the
> [*] construct and many people (not just me) here tried their best to
> explain their intuition to you.

And yet, the fact that it's hard to explain your intuition to others
(Steven is not the only one who's finding this hard to follow) surely
implies that it's merely that - personal intuition - and not universal
understanding.

The *whole point* here is that not everyone understands the proposed
notation the way the proposers do, and it's *hard to explain* to those
people. Blaming the people who don't understand does not support the
position that this notation should be added to the language. Rather,
it reinforces the idea that the new proposal is hard to teach (and
consequently, it may be a bad idea for Python).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Fwd: Fwd: unpacking generalisations for list comprehension

[Martti Kühne]

On Wed, Oct 12, 2016 at 5:41 PM, Nick Coghlan  wrote:
> However, set builder notation doesn't inherently include the notion of
> flattening lists-of-lists. Instead, that's a *consumption* operation
> that happens externally after the initial list-of-lists has been
> built, and that's exactly how it's currently spelled in Python:
> "itertools.chain.from_iterable(subiter for subiter in iterable)".


On Wed, Oct 12, 2016 at 5:42 PM, Steven D'Aprano  wrote:
> The fundamental design principle of list comps is that they are
> equivalent to a for-loop with a single append per loop:
>
> [expr for t in iterable]
>
> is equivalent to:
>
> result = []
> for t in iterable:
> result.append(expr)
>
>
> If I had seen a list comprehension with an unpacked loop variable:
>
> [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]]
>
>


As it happens, python does have an external consumption operation that
happens externally with an iteration implied:

for t in iterable:
yield t

For your example [t for t in [(1, 'a'), (2, 'b'), (3, 'c')]] that would mean:

for t in [(1, 'a'), (2, 'b'), (3, 'c')]:
yield t

And accordingly, for the latter case [*t for t in [(1, 'a'), (2, 'b'),
(3, 'c')]] it would be:

for item in [(1, 'a'), (2, 'b'), (3, 'c')]:
for t in item:
yield t

cheers!
mar77i
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Thomas Nyberg]

On 10/12/2016 07:13 PM, Mikhail V wrote:

On 12 October 2016 at 23:50, Thomas Nyberg  wrote:

Since when was decimal notation "standard"?

Depends on what planet do you live. I live on planet Earth. And you?


If you mean that decimal notation is the standard used for _counting_ by 
people, then yes of course that is standard. But decimal notation 
certainly is not standard in this domain.



opposite. For unicode representations, byte notation seems standard.

How does this make it a good idea?
Consider unicode table as an array with glyphs.
Now the index of the array is suddenly represented in some
obscure character set. How this index is other than index of any
array or natural number? Think about it...


Hexadecimal notation is hardly "obscure", but yes I understand that 
fewer people understand it than decimal notation. Regardless, byte 
notation seems standard for unicode and unless you can convince the 
unicode community at large to switch, I don't think it makes any sense 
for python to switch. Sometimes it's better to go with the flow even if 
you don't want to.



2. Mixing of two notations (hex and decimal) is a _very_ bad idea,
I hope no need to explain why.


Still not sure which "mixing" you refer to.


Still not sure? These two words in brackets. Mixing those two systems.



There is not mixing for unicode in python; it displays as hexadecimal. 
Decimal is used in other places though. So if by "mixing" you mean 
python should not use the standard notations of subdomains when working 
with those domains, then I would totally disagree. The language used in 
different disciplines is and has always been variable. Until that's no 
longer true it's better to stick with convention than add inconsistency 
which will be much more confusing in the long-term than learning the 
idiosyncrasies of a specific domain (in this case the use of hexadecimal 
in the unicode world).


Cheers,
Thomas
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Chris Angelico]

On Fri, Oct 14, 2016 at 1:25 AM, Steven D'Aprano  wrote:
> On Thu, Oct 13, 2016 at 03:56:59AM +0200, Mikhail V wrote:
>> and in long perspective when the world's alphabetical garbage will
>> dissapear, two digits would be ok.
> Talking about "alphabetical garbage" like that makes you seem to be an
> ASCII bigot: rude, ignorant, arrogant and rather foolish as well. Even
> 7-bit ASCII has more than 100 characters (128).

Solution: Abolish most of the control characters. Let's define a brand
new character encoding with no "alphabetical garbage". These
characters will be sufficient for everyone:

* [2] Formatting characters: space, newline. Everything else can go.
* [8] Digits: 01234567
* [26] Lower case Latin letters a-z
* [2] Vital social media characters: # (now officially called "HASHTAG"), @
* [2] Can't-type-URLs-without-them: colon, slash (now called both
"SLASH" and "BACKSLASH")

That's 40 characters that should cover all the important things anyone
does - namely, Twitter, Facebook, and email. We don't need punctuation
or capitalization, as they're dying arts and just make you look
pretentious. I might have missed a few critical characters, but it
should be possible to fit it all within 64, which you can then
represent using two digits from our newly-restricted set; octal is
better than decimal, as it needs less symbols. (Oh, sorry, so that's
actually "50" characters, of which "32" are the letters. And we can
use up to "100" and still fit within two digits.)

Is this the wrong approach, Mikhail? Perhaps we should go the other
way, then, and be *inclusive* of people who speak other languages.
Thanks to Unicode's rich collection of characters, we can represent
multiple languages in a single document; see, for instance, how this
uses four languages and three entirely distinct scripts:
http://youtu.be/iydlR_ptLmk Turkish and French both use the Latin
script, but have different characters. Alphabetical garbage, or
accurate representations of sounds and words in those languages?

Python 3 gives the world's languages equal footing. This is a feature,
not a bug. It has consequences, including that arbitrary character
entities could involve up to seven decimal digits or six hex (although
for most practical work, six decimal or five hex will suffice). Those
consequences are a trivial price to pay for uniting the whole
internet, as opposed to having pockets of different languages, like we
had up until the 90s.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 13.10.2016 16:10, Steven D'Aprano wrote:

On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
Multiplication with only a single argument? Come on. 

You didn't say anything about a single argument. Your exact words are
shown above: "where have I seen * so far?". I'm pretty sure you've seen
* used for multiplication. I also could have mentioned regexes, globs,
and exponentiation.

I cannot respond to your intended meaning, only to what you actually
write. Don't blame the reader if you failed to communicate clearly and
were misunderstood.


Steven, please. You seemed to struggle to understand the notion of the 
[*] construct and many people (not just me) here tried their best to 
explain their intuition to you. But now it seems you don't even try to 
come behind the idea and instead try hard not to understand the help 
offered. If you don't want help or don't really want to understand the 
proposal, that's fine but please, do us a favor and don't divert the 
thread with nitpicking nonsensical details (like multiplication) and 
waste everybody's time.



The context of the proposal is about lists/dicts and the */** unpacking 
syntax. So, I actually expect you to put every post here into this very 
context. Discussions without context don't make much sense. So, I won't 
reply further.



Best,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[אלעזר]

On Thu, Oct 13, 2016 at 5:10 PM Steven D'Aprano  wrote:

> On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
> > About the list constructor: we construct a list by writing [a,b,c] or by
> > writing [b for b in bs]. The end result is a list
>
> I construct lists using all sorts of ways:
>

I think there is a terminology problem here (again). "Constructor" in OOP
has a specific meaning, and "constructor" in functional terminology has a
slightly different meaning. I guess Sven uses the latter terminology
because pattern matching is the dual of the constructor - it is a
"destructor" - and it feels appropriate, although admittedly confusing. In
this terminology, map(), zip() etc. are definitely not constructors. there
is only one "constructor" (list()), and there are functions that may use it
as their implementation detail. In a way, [1, 2, 3] is just a syntactic
shorthand for list construction, so it is reasonable to a call it a
constructor.

This terminology is not a perfect fit into the object-oriented world of
Python, but it is very helpful in discussion of patterns how to apply them
uniformly, since they were pretty much invented in the functional world
(ML, I think, and mathematics). One only needs to be aware of the two
different meaning, and qualify if needed, so that we won't get lost in
terminology arguments again.

Elazar
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 03:56:59AM +0200, Mikhail V wrote:

> > How many decimal digits would you use to denote a single character?
> 
> for text, three decimal digits would be enough for me personally,

Well, if it's enough for you, why would anyone need more?


> and in long perspective when the world's alphabetical garbage will
> dissapear, two digits would be ok.

Are you serious? 

Talking about "alphabetical garbage" like that makes you seem to be an 
ASCII bigot: rude, ignorant, arrogant and rather foolish as well. Even 
7-bit ASCII has more than 100 characters (128).



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Steven D'Aprano]

On Thu, Oct 13, 2016 at 10:37:35AM +0200, Sven R. Kunze wrote:
> On 13.10.2016 01:29, Steven D'Aprano wrote:
> >On Wed, Oct 12, 2016 at 06:32:12PM +0200, Sven R. Kunze wrote:
> >>
> >>So, my reasoning would tell me: where have I seen * so far? *args and
> >>**kwargs!
> >And multiplication.
> 
> Multiplication with only a single argument? Come on.

You didn't say anything about a single argument. Your exact words are
shown above: "where have I seen * so far?". I'm pretty sure you've seen
* used for multiplication. I also could have mentioned regexes, globs,
and exponentiation.

I cannot respond to your intended meaning, only to what you actually 
write. Don't blame the reader if you failed to communicate clearly and 
were misunderstood.


[...]
> About the list constructor: we construct a list by writing [a,b,c] or by 
> writing [b for b in bs]. The end result is a list 

I construct lists using all sorts of ways:

list(argument)
map(func, sequence)
zip(a, b)
file.readlines()
dict.items()
os.listdir('.')
sorted(values)

and so on. Should I call them all "list constructors" just because they
return a list? No, I don't think so. Constructor has a specific meaning, 
and these are not all constructors -- and neither are list 
comprehensions.


> and that matters from 
> the end developer's point of view, no matter how fancy words you choose 
> for it.

These "fancy words" that you dismiss are necessary technical terms. 
Precision in language is something we should aim for, not dismiss as 
unimportant.

List comprehensions and list displays have different names, not to show 
off our knowledge of "fancy terms", but because they are different 
things which just happen to both return lists. Neither of them are what 
is commonly called a constructor. List displays are, in some senses, 
like a literal; list comprehensions are not, and are better understood 
as list builders, a process which builds a list. Emphasis should be on 
the *process* part: a comprehension is syntactic sugar for building a 
list using for-loop, not for a list display or list constructor.

The bottom line is that when you see a comprehension (list, set or dict) 
or a generator expression, you shouldn't think of list displays, but of 
a for-loop. That's one of the reasons why the analogy with argument 
unpacking fails: it doesn't match what comprehensions *actually* are.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Improve error message when missing 'self' in method definition

[Steven D'Aprano]

On Tue, Oct 11, 2016 at 02:31:25PM +1100, Chris Angelico wrote:
> On Tue, Oct 11, 2016 at 2:29 PM, Stephen J. Turnbull
>  wrote:
> > Chris Angelico writes:
> >
> >  > Given that it's not changing semantics at all, just adding info/hints
> >  > to an error message, it could well be added in a point release.
> >
> > But it does change semantics, specifically for doctests.
> 
> Blah, forgot about doctests. Guess that's off the cards for a point
> release, then, but still, shouldn't be a big deal for 3.7.

Error messages are not part of Python's public API. We should be able to 
change error messages at any time, including point releases.

Nevertheless, we shouldn't abuse that right. If it's only a change to 
the error message, and not a functional change, then maybe we can add it 
to the next 3.6 beta or rc. But its probably not worth backporting it to 
older versions.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Chris Angelico]

On Thu, Oct 13, 2016 at 9:05 PM, Cory Benfield  wrote:
> Binary notation seems like the solution, but note the above case: the only 
> way to work out how many bits are being masked out is to count them, and 
> there can be quite a lot. IIRC there’s some new syntax coming for binary 
> literals that would let us represent them as 0b___, which 
> would help the readability case, but it’s still substantially less dense and 
> loses clarity for many kinds of unusual bit patterns.
>

And if you were to write them like this, you would start to read them
in blocks of four - effectively, treating each underscore-separated
unit as a glyph, despite them being represented with four characters.
Fortunately, just like with Hangul characters, we have a
transformation that combines these multi-character glyphs into single
characters. We call it 'hexadecimal'.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Cory Benfield]

> On 13 Oct 2016, at 09:43, Greg Ewing  wrote:
> 
> Mikhail V wrote:
>> Did you see much code written with hex literals?
> 
> From /usr/include/sys/fcntl.h:
> 

Backing Greg up for a moment, hex literals are extremely common in any code 
that needs to work with binary data, such as network programming or fine data 
structure manipulation. For example, consider the frequent requirement to mask 
out certain bits of a given integer (e.g., keep the low 24 bits of a 32 bit 
integer). Here are a few ways to represent that:

integer & 0x00FF  # Hex
integer & 16777215  # Decimal
integer & 0o  # Octal
integer & 0b  # Binary

Of those four, hexadecimal has the advantage of being both extremely concise 
and clear. The octal representation is infuriating because one octal digit 
refers to *three* bits, which means that there is a non-whole number of octal 
digits in a byte (that is, one byte with all bits set is represented by 0o377). 
This causes problems both with reading comprehension and with most other common 
tasks. For example, moving from 0xFF to 0x (or 255 to 65535, also known as 
setting the next most significant byte to all 1) is represented in octal by 
moving from 0o377 to 0o17. This is not an obvious transition, and I doubt 
many programmers could do it from memory in any representation but hex or 
binary.

Decimal is no clearer. Programmers know how to represent certain bit patterns 
from memory in decimal simply because they see them a lot: usually they can do 
the all 1s case, and often the 0 followed by all 1s case (255 and 128 for one 
byte, 65535 and 32767 for two bytes, and then increasingly few programmers know 
the next set). But trying to work out what mask to use for setting only bits 15 
and 14 is tricky in decimal, while in hex it’s fairly easy (in hex it’s 0xC000, 
in decimal it’s 49152).

Binary notation seems like the solution, but note the above case: the only way 
to work out how many bits are being masked out is to count them, and there can 
be quite a lot. IIRC there’s some new syntax coming for binary literals that 
would let us represent them as 0b___, which would help the 
readability case, but it’s still substantially less dense and loses clarity for 
many kinds of unusual bit patterns. Additionally, as the number of bits 
increases life gets really hard: masking out certain bits of a 64-bit number 
requires a literal that’s at least 66 characters long, not including the 
underscores that would add another 15 underscores for a literal that is 81 
characters long (more than the PEP8 line width recommendation). That starts 
getting unwieldy fast, while the hex representation is still down at 18 
characters.

Hexadecimal has the clear advantage that each character wholly represents 4 
bits, and the next 4 bits are independent of the previous bits. That’s not true 
of decimal or octal, and while it’s true of binary it costs a fourfold increase 
in the length of the representation. It’s definitely not as intuitive to the 
average human being, but that’s ok: it’s a specialised use case, and we aren’t 
requiring that all human beings learn this skill.

This is a very long argument to suggest that your argument against hexadecimal 
literals (namely, that they use 16 glyphs as opposed to the 10 glyphs used in 
decimal) is an argument that is too simple to be correct. Different collections 
of glyphs are clearer in different contexts. For example, decimal numerals can 
be represented using 10 glyphs, while the english language requires 26 glyphs 
plus punctuation. But I don’t think you’re seriously proposing we should swap 
from writing English using the larger glyph set to writing it in decimal 
representation of ASCII bytes.

Given this, I think the argument that says that the Unicode consortium said 
“write the number in hex” is good enough for me.

Cory

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Greg Ewing]

Mikhail V wrote:

But: you claim to see bit patterns in hex numbers? Then I bet you will
see them much better if you take binary notation (2 symbols) or quaternary
notation (4 symbols), I guarantee.


Nope. The meaning of 0xC001 is much clearer to me than
1101, because I'd have to count the bits very
carefully in the latter to distinguish it from, e.g.
6001 or 18001.

The bits could be spaced out:

1100   0001

but that just takes up even more room to no good effect.
I don't find it any faster to read -- if anything, it's
slower, because my eyes have to travel further to see
the whole thing.

Another point -- a string of hex digits is much easier
for me to *remember* if I'm transliterating it from one
place to another. Not only because it's shorter, but
because I can pronounce it. "Cee zero zero one" is a
lot easier to keep in my head than "one one zero zero
zero zero zero zero zero zero zero zero zero zero zero
one"... by the time I get to the end, I've already
forgotten how it started!


And if you take consistent glyph set for them
also you'll see them twice better, also guarantee 100%.


When I say "instantly", I really do mean *instantly*.
I fail to see how a different glyph set could reduce
the recognition time to less than zero.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Greg Ewing]

Mikhail V wrote:

Eee how would I find if the character lies in certain range?


>>> c = "\u1235"
>>> if "\u1230" <= c <= "\u123f":
...  print("Boo!")
...
Boo!

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Greg Ewing]

Mikhail V wrote:

Ok, but if I write a string filtering in Python for example then
obviously I use decimal everywhere to compare index ranges, etc.
so what is the use for me of that label? Just redundant
conversions back and forth. 


I'm not sure what you mean by that. If by "index ranges"
you're talking about the numbers you use to index into
the string, they have nothing to do with character codes,
so you can write them in whatever base is most convenient
for you.

If you have occasion to write a literal representing a
character code, there's nothing to stop you writing it
in hex to match the way it's shown in a repr(), or in
published Unicode tables, etc.

I don't see a need for any conversions back and forth.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Greg Ewing]

Mikhail V wrote:

I am not against base-16 itself in the first place,
but rather against the character set which is simply visually
inconsistent and not readable.


Now you're talking about inventing new characters, or
at least new glyphs for existing ones, and persuading
everyone to use them. That's well beyond the scope of
what Python can achieve!

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[Greg Ewing]

Mikhail V wrote:

Did you see much code written with hex literals?


From /usr/include/sys/fcntl.h:

/*
 * File status flags: these are used by open(2), fcntl(2).
 * They are also used (indirectly) in the kernel file structure f_flags,
 * which is a superset of the open/fcntl flags.  Open flags and f_flags
 * are inter-convertible using OFLAGS(fflags) and FFLAGS(oflags).
 * Open/fcntl flags begin with O_; kernel-internal flags begin with F.
 */
/* open-only flags */
#define O_RDONLY0x  /* open for reading only */
#define O_WRONLY0x0001  /* open for writing only */
#define O_RDWR  0x0002  /* open for reading and writing */
#define O_ACCMODE   0x0003  /* mask for above modes */

/*
 * Kernel encoding of open mode; separate read and write bits that are
 * independently testable: 1 greater than the above.
 *
 * XXX
 * FREAD and FWRITE are excluded from the #ifdef KERNEL so that TIOCFLUSH,
 * which was documented to use FREAD/FWRITE, continues to work.
 */
#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define FREAD   0x0001
#define FWRITE  0x0002
#endif
#define O_NONBLOCK  0x0004  /* no delay */
#define O_APPEND0x0008  /* set append mode */
#ifndef O_SYNC  /* allow simultaneous inclusion of  */
#define O_SYNC  0x0080  /* synch I/O file integrity */
#endif
#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define O_SHLOCK0x0010  /* open with shared file lock */
#define O_EXLOCK0x0020  /* open with exclusive file lock */
#define O_ASYNC 0x0040  /* signal pgrp when data ready */
#define O_FSYNC O_SYNC  /* source compatibility: do not use */
#define O_NOFOLLOW  0x0100  /* don't follow symlinks */
#endif /* (_POSIX_C_SOURCE && !_DARWIN_C_SOURCE) */
#define O_CREAT 0x0200  /* create if nonexistant */
#define O_TRUNC 0x0400  /* truncate to zero length */
#define O_EXCL  0x0800  /* error if already exists */
#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define O_EVTONLY   0x8000  /* descriptor requested for event 
notifications only */
#endif


#define O_NOCTTY0x2 /* don't assign controlling terminal */


#if !defined(_POSIX_C_SOURCE) || defined(_DARWIN_C_SOURCE)
#define O_DIRECTORY 0x10
#define O_SYMLINK   0x20/* allow open of a symlink */
#endif

#ifndef O_DSYNC /* allow simultaneous inclusion of  */
#define O_DSYNC 0x40/* synch I/O data integrity */
#endif

--
Greg

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Fwd: unpacking generalisations for list comprehension

[Sven R. Kunze]

On 13.10.2016 01:29, Steven D'Aprano wrote:

On Wed, Oct 12, 2016 at 06:32:12PM +0200, Sven R. Kunze wrote:


So, my reasoning would tell me: where have I seen * so far? *args and
**kwargs!

And multiplication.


Multiplication with only a single argument? Come on.



And sequence unpacking.


We are on the right side of the = if any and not no the left side.




[...] is just the list constructor.

Also indexing: dict[key] or sequence[item or slice].


There's no name in front of [. So, I cannot be an index either.


Nothing else matches (in my head) and I also don't see any ambiguities. 
YMMV.


I remember a new co-worker, I taught how to use *args and **kwargs. It 
was unintuitive to him on the first time as well.


About the list constructor: we construct a list by writing [a,b,c] or by 
writing [b for b in bs]. The end result is a list and that matters from 
the end developer's point of view, no matter how fancy words you choose 
for it.


Cheers,
Sven
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Proposal for default character representation

[M.-A. Lemburg]

On 13.10.2016 01:06, Mikhail V wrote:
> On 12 October 2016 at 23:48, M.-A. Lemburg  wrote:
>> The hex notation for \u is a standard also used in many other
>> programming languages, it's also easier to parse, so I don't
>> think we should change this default.
> 
> In programming literature it is used often, but let me point out that
> decimal is THE standard and is much much better standard
> in sence of readability. And there is no solid reason to use 2 standards
> at the same time.

I guess it's a matter of choosing the right standard for the
right purpose. For \u and \U the intention was to be able
to represent a Unicode code point using its standard Unicode ordinal
representation and since the standard uses hex for this, it's
quite natural to use the same here.

>> Take e.g.
>>
> s = "\u123456"
> s
>> 'ሴ56'
>>
>> With decimal notation, it's not clear where to end parsing
>> the digit notation.
> 
> How it is not clear if the digit amount is fixed? Not very clear what
> did you mean.

Unicode code points have ordinals from the range [0, 1114111],
so it's not clear where to stop parsing the decimal representation
and continue to interpret the literal as regular string, since
I suppose you did not intend everyone to have to write
\u010 just to get a newline code point to avoid the
ambiguity.

PS: I'm not even talking about the breakage such a change would
cause. This discussion is merely about the pointing out how
things got to be how they are now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Oct 13 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/