[issue471928] global made w/nested list comprehensions

[admin]

Change by admin :


--
github: None -> 35341

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue406867] nested list comprehensions crash

[admin]

Change by admin :


--
github: None -> 34118

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue523995] PDB single steps list comprehensions

[admin]

Change by admin :


--
github: None -> 36182

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue510384] Separate Scope for List Comprehensions

[admin]

Change by admin :


--
github: None -> 35996

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue406076] fix for nested list comprehensions (#406049)

[admin]

Change by admin :


--
github: None -> 34069

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401211] list comprehensions: require initial for clause

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400759] range-lists: [1:10] (relative to list comprehensions patch)

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400654] list comprehensions in Python (for 2.0)

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401211] list comprehensions: require initial for clause

[admin]

Change by admin :


--
github: None -> 32950

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400759] range-lists: [1:10] (relative to list comprehensions patch)

[admin]

Change by admin :


--
github: None -> 32529

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400654] list comprehensions in Python (for 2.0)

[admin]

Change by admin :


--
github: None -> 32471

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41918] exec fails to take locals into account when running list comprehensions or functions

[patterns]

patterns  added the comment:

Also occurs in Python 3.6.9

--
nosy: +patterns

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41918] exec fails to take locals into account when running list comprehensions or functions

[Nikolas Havrikov]

Nikolas Havrikov  added the comment:

This issue also occurs in Python 3.8.6

--
nosy: +havrikov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41918] exec fails to take locals into account when running list comprehensions or functions

[undefined blinded]

undefined blinded  added the comment:

This seems to happen only when both arguments to exec are used:

```
Python 3.7.7 (default, Mar 10 2020, 15:43:27)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exec('text = ["hallo"]\ntext.append("Welt")\nprint(" ".join([text[i] for i 
>>> in range(0, 2)]))')
hallo Welt
>>> exec('text = ["hallo"]\ntext.append("Welt")\nprint(" ".join([text[i] for i 
>>> in range(0, 2)]))', {})
hallo Welt
>>> exec('text = ["hallo"]\ntext.append("Welt")\nprint(" ".join(text))', {}, {})
hallo Welt
>>> exec('text = ["hallo"]\ntext.append("Welt")\nprint(" ".join([text[i] for i 
>>> in range(0, 2)]))', {}, {})
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 3, in 
  File "", line 3, in 
NameError: name 'text' is not defined
```

--
nosy: +alex0kamp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41918] exec fails to take locals into account when running list comprehensions or functions

[Quentin Peter]

Quentin Peter  added the comment:

Fails for functions as well:
```
In [4]: exec(compile('print(my_var)\ndef a():\n print(my_var)\na()', '', 
'exec'), globals(), {"my_var": 0})
0
Traceback (most recent call last):

File "", line 1, in 
exec(compile('print(my_var)\ndef a():\n print(my_var)\na()', '', 
'exec'), globals(), {"my_var": 0})

File "", line 4, in 

 File "", line 3, in a

NameError: name 'my_var' is not defined
```

--
title: exec fails to take locals into account when running list comprehensions 
-> exec fails to take locals into account when running list comprehensions or 
functions

___
Python tracker 
<https://bugs.python.org/issue41918>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41918] exec fails to take locals into account when running list comprehensions

[Quentin Peter]

New submission from Quentin Peter :

The exec function fails to take locals into account when executing a list 
comprehension:
```
Python 3.7.7 (default, Mar 10 2020, 15:43:33) 
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exec(compile('[my_var for i in range(1)]\n', '', 'single'), 
>>> {**globals(), "my_var": 0}, None)
[0]
>>> exec(compile('[my_var for i in range(1)]\n', '', 'single'), 
>>> globals(), {"my_var": 0})
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 1, in 
  File "", line 1, in 
NameError: name 'my_var' is not defined
>>> 
```
This is the cause of https://bugs.python.org/issue21161

--
messages: 377862
nosy: qpeter
priority: normal
severity: normal
status: open
title: exec fails to take locals into account when running list comprehensions
type: enhancement
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue41918>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[Anselm Kiefner]

Anselm Kiefner  added the comment:

I just stumbled over this same restriction and when I googled for "SyntaxError: 
cannot assign to named expression", 0 actual results showed - an absolute 
unicorn for a Python error.

> "Due to design constraints in the reference implementation (the symbol table 
> analyser cannot easily detect when names are re-used between the leftmost 
> comprehension iterable expression and the rest of the comprehension), named 
> expressions are disallowed entirely as part of comprehension iterable 
> expressions (the part after each "in", and before any subsequent "if" or 
> "for" keyword):"

Might the new PEG parser maybe help alleviate this restriction, so we could 
declare this a bug instead?

--
nosy: +Anselm Kiefner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41216] eval don't load local variable in dict and list comprehensions.

[Eric V. Smith]

Eric V. Smith  added the comment:

Agreed that this is a duplicate, so I'm closing it.

--
nosy: +eric.smith
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> eval() function in List Comprehension doesn't work

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41216] eval don't load local variable in dict and list comprehensions.

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This seems to be a duplicate of issue5242, issue36300

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41216] eval don't load local variable in dict and list comprehensions.

[Andrej Klychin]

New submission from Andrej Klychin :

I'm not sure is it a bug or a fecature of comprehensions or eval, but 
intuitively it seems like it should work.

def foo(baz):
return eval("[baz for _ in range(10)]")

foo(3)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2, in foo
  File "", line 1, in 
  File "", line 1, in 
NameError: name 'baz' is not defined

def bar(baz):
return eval("{i: baz for i in range(10)}")

bar(3)
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 2, in bar
  File "", line 1, in 
  File "", line 1, in 
NameError: name 'baz' is not defined

--
components: Interpreter Core
messages: 373054
nosy: Andy_kl
priority: normal
severity: normal
status: open
title: eval don't load local variable in dict and list comprehensions.
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue41216>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[STINNER Victor]

STINNER Victor  added the comment:

> Isn't this the same as https://bugs.python.org/issue36551 ?

Yes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Isn't this the same as https://bugs.python.org/issue36551 ?

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[Ammar Askar]

Ammar Askar  added the comment:

Aah, thanks for the catcher Victor. Missed that this was about list 
/comprehensions/, not the list constructor.

--

___
Python tracker 
<https://bugs.python.org/issue14126>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[STINNER Victor]

STINNER Victor  added the comment:

> I believe this was implemented in issue33234

I don't think so. The bytecode in Python 3.9 still uses "BUILD_LIST 0":

Python 3.9.0a4+ (heads/daemon_thread_runtime2-dirty:48652767d5, Mar  7 2020, 
00:56:07) 
>>> def f():
... for i in range(1):
... [j for j in range(1)]
... 
>>> import dis; dis.dis(f)
(...)

Disassembly of  at 0x7ffab2c9fd40, file "", line 
3>:
  3   0 BUILD_LIST   0
  2 LOAD_FAST0 (.0)
>>4 FOR_ITER 8 (to 14)
  6 STORE_FAST   1 (j)
  8 LOAD_FAST1 (j)
 10 LIST_APPEND  2
 12 JUMP_ABSOLUTE4
>>   14 RETURN_VALUE

--
resolution: fixed -> 
status: closed -> open
superseder: Improve list() pre-sizing for inputs with known lengths -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[Ammar Askar]

Ammar Askar  added the comment:

I believe this was implemented in issue33234

--
nosy: +ammar2
resolution:  -> fixed
stage: needs patch -> resolved
status: open -> closed
superseder:  -> Improve list() pre-sizing for inputs with known lengths

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14126] Speed up list comprehensions by preallocating the list where possible

[Brett Cannon]

Change by Brett Cannon :


--
nosy:  -brett.cannon
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21161] list comprehensions don't see local variables in pdb in python3

[Andreas Kloeckner]

Change by Andreas Kloeckner :


--
nosy: +inducer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[Eric V. Smith]

Eric V. Smith  added the comment:

This is explicitly mentioned in PEP 572 as being disallowed:

"Due to design constraints in the reference implementation (the symbol table 
analyser cannot easily detect when names are re-used between the leftmost 
comprehension iterable expression and the rest of the comprehension), named 
expressions are disallowed entirely as part of comprehension iterable 
expressions (the part after each "in", and before any subsequent "if" or "for" 
keyword):"

The error message also makes it clear.

--
nosy: +eric.smith
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[EGN]

EGN  added the comment:

Even the simple code like this doesn't work:

[print(p) for n in (p := ['a', 'b', 'c'])]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[EGN]

EGN  added the comment:

But if I'm taking main for loop out of the square brackets everything works 
fine.

from os import getcwd, listdir, rename
import re

for n in listdir(p := f"{getcwd()}\\{input('Folder: ')}\\"):
[rename(f'{p}{n}', f"{p}{''.join([w[:3] if len(w) > 3 else w for w in 
re.split('[-_. ]', n)[:-1]])}.{n.split('.')[-1]}")]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[EGN]

EGN  added the comment:

from os import getcwd, listdir, rename
import re

[rename(f'{p}{n}', f"{p}{''.join([w[:3] if len(w) > 3 else w for w in 
re.split('[-_. ]', n)[:-1]])}.{n.split('.')[-1]}")
 for n in listdir(p := f"{getcwd()}\\{input('Folder: ')}\\")]

When I run this code, I'm getting:
  File "C:\Users\1\Desktop\sn.py", line 4
[rename(f'{p}{n}',f"{p}{''.join([w[:3] if len(w)>3 else w for w in 
re.split('[-_. ]',n)[:-1]])}.{n.split('.')[-1]}") for n in 
listdir(p:=f"{getcwd()}\\{input('Folder: ')}\\")]

 ^
SyntaxError: assignment expression cannot be used in a comprehension iterable 
expression

Process finished with exit code 1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[Steven D'Aprano]

Change by Steven D'Aprano :


--
components: +Interpreter Core
type: enhancement -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[Steven D'Aprano]

Steven D'Aprano  added the comment:

We're not mind-readers, how do you expect us to know what you tried if you 
don't tell us?

The walrus operator works for me:

>>> [spam for c in "hello world" if (spam:=c.upper()) in 'AEIOU']
['E', 'O', 'O']

>>> [(spam:=x**2, spam+1) for x in range(5)]
[(0, 1), (1, 2), (4, 5), (9, 10), (16, 17)]


What did you try, and what happened?

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Can you please post a short snippet of what you are trying to do and the issue 
you are facing like traceback if any?

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in Python 3.8.0 in list comprehensions

[EGN]

New submission from EGN :

Just tried to use Walrus operator in list comprehensions (for loop) and it is 
not possible.

Is it a bug?

--
messages: 355124
nosy: EGN
priority: normal
severity: normal
status: open
title: Walrus operator in Python 3.8.0 in list comprehensions
type: enhancement
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38556>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38556] Walrus operator in list comprehensions [Python 3.8.0]

[EGN]

Change by EGN :


--
title: Walrus operator in Python 3.8.0 in list comprehensions -> Walrus 
operator in list comprehensions [Python 3.8.0]

___
Python tracker 
<https://bugs.python.org/issue38556>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21161] list comprehensions don't see local variables in pdb in python3

[daniel hahler]

Change by daniel hahler :


--
pull_requests: +14926
pull_request: https://github.com/python/cpython/pull/15194

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21161] list comprehensions don't see local variables in pdb in python3

[daniel hahler]

daniel hahler  added the comment:

Georg, please consider re-opening.
This patch here appears to fix this after all.

--
nosy: +blueyed
versions: +Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

We can return to this issue if make the invocation of __length_hint__ much much 
faster. For example by adding the tp_length_hint slot. But currently it is too 
large change and his has negative effects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

Change by anthony shaw :


--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

Change by anthony shaw :


--
assignee:  -> anthonypjshaw
nosy: +anthonypjshaw

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

>I am -1 for this optimization because it affects only one particular case 
>(neither other kinds of comprehensions, nor generator expressions, nor list 
>comprehensions with 
 conditions) and even in this case it is small. It is possible to add a lot of 
other optimizations for other cases which will sped up them to 50% or 100%, but 
we do not do 
 this, because every such optimization has a cost. It increases the amount of 
code which should be maintained and covered by tests, it adds small overhead in 
common cases to 
 speed up an uncommon case, and increasing the code base can negatively affect 
surrounding code (just because the CPU cache and registers are used 
inappropriate and the 
 compiler optimizes less important paths).

Understood, I had hoped this change would have a broader impact. The additional 
opcode is not ideal either. 

> In addition, while this change speed up list comprehensions for long list, it 
> slows down them for short lists. Short lists are more common.

I've been profiling this today, basically, this implementation receives the 
`list_iter`, `range_iter`, etc.

There is no standard object model for an iterator's length, _PyObject_HasLen 
would return false because it neither implements tp_as_sequence nor, 
tp_as_mapping (rightly so).

What this has uncovered (so hopefully there's some value from this whole 
experience!) is that __length_hint__ for iterators is _really_ inefficient. 
Take a list_iterator for example:

PyObject_LengthHint will call, _PyObject_HasLen, which returns false, which 
then goes to call
_PyObject_LookupSpecial, then 
_PyObject_CallNoArg, which calls 
listiter_len, which calls 
PyList_GET_SIZE which returns a Py_ssize_t, which is then converted to a PyLong 
via
PyLong_FromSsize_t, which is then returned back to PyObject_LengthHint, which 
then
PyLong_AsSsize_t is run to convert the PyLong back into a Py_ssize_t

The Py_ssize_t is then finally returned to the caller!

My conclusion was that the list comprehension should be initialized to the 
length of the target, before GET_ITER is run. This would remove the overhead 
for range objects, because you could simply call _PyObject_HasLen, which would 
return true for dict, list, tuple and set, but false for range objects (which 
is what you want).

The issue is that GET_ITER is called outside the code object for the 
comprehension, so you'd have to pass an additional argument to the 
comprehension generator.

This is way outside of my expertise, but the only way I can see to find a 
sizeable benefit, with minimal code and no edge cases which are slower.

Thanks for your time

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> I was going to note that the algorithm Anthony has pursued here is the same 
> one we already use for the list constructor and list.extend(), but Inada-san 
> already pointed that out :)

And that optimization looks questionable to me. I tried to reduce an overhead 
for small lists, but this requires much more complex code and gives mixed 
results.

I am -1 for this optimization because it affects only one particular case 
(neither other kinds of comprehensions, nor generator expressions, nor list 
comprehensions with conditions) and even in this case it is small. It is 
possible to add a lot of other optimizations for other cases which will sped up 
them to 50% or 100%, but we do not do this, because every such optimization has 
a cost. It increases the amount of code which should be maintained and covered 
by tests, it adds small overhead in common cases to speed up an uncommon case, 
and increasing the code base can negatively affect surrounding code (just 
because the CPU cache and registers are used inappropriate and the compiler 
optimizes less important paths).

In addition, while this change speed up list comprehensions for long list, it 
slows down them for short lists. Short lists are more common.

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

; richards; scimark_fft; scimark_lu; 
scimark_sor; sympy_expand; sympy_integrate; sympy_sum; xml_etree_parse

I'd like to look at the way range object LengthHint works, it looks like the 
path for those is not ideal and could use some optimization. Also, 
BUILD_LIST_PREALLOC uses the Iterator, not the actual object, so you can't use 
the much faster _HasLen and PyObject_Length().

I'm going to look at how __length_hint__ could be optimized for iterators that 
would make the smaller range cases more efficient.

meteor_contest uses a lot of list comprehensions, so should show the impact for 
the patch.

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Here are the updated results for the benchmark suite. The previous results 
(unlinked from the issue to reduce noise)
were against an old version of the master branch.

2019-04-08_13-08-master-58721a903074.json.gz


Performance version: 0.7.0
Report on Linux-5.0.5-1-ARCH-x86_64-with-glibc2.28
Number of logical CPUs: 12
Start date: 2019-04-09 00:32:40.656576
End date: 2019-04-09 00:54:03.721798

12718-5f06333a4e49.json.gz
==

Performance version: 0.7.0
Report on Linux-5.0.5-1-ARCH-x86_64-with-glibc2.28
Number of logical CPUs: 12
Start date: 2019-04-09 00:07:53.433511
End date: 2019-04-09 00:29:32.603022

  
-- CURRENT MASTER --
-- PR12718 --
+-+--++--++
| Benchmark   | 2019-04-08_13-08-master-58721a903074.json.gz | 
12718-5f06333a4e49.json.gz | Change   | Significance   |
+=+==++==++
| 2to3| 455 ms   | 465 
ms | 1.02x slower | Significant (t=-2.27)  |
+-+--++--++
| chaos   | 171 ms   | 180 
ms | 1.05x slower | Significant (t=-5.92)  |
+-+--++--++
| crypto_pyaes| 169 ms   | 177 
ms | 1.05x slower | Significant (t=-6.06)  |
+-+--++--++
| deltablue   | 11.5 ms  | 11.9 
ms| 1.03x slower | Significant (t=-3.28)  |
+-+--++--++
| django_template | 175 ms   | 191 
ms | 1.09x slower | Significant (t=-13.00) |
+-+--++--++
| dulwich_log | 127 ms   | 131 
ms | 1.03x slower | Significant (t=-8.97)  |
+-+--++--++
| fannkuch| 646 ms   | 665 
ms | 1.03x slower | Significant (t=-12.37) |
+-+--++--++
| float   | 165 ms   | 165 
ms | 1.00x faster | Not significant|
+-+--++--++
| go  | 386 ms   | 381 
ms | 1.01x faster | Not significant|
+-+--++--++
| hexiom  | 15.4 ms  | 15.5 
ms| 1.00x slower | Not significant|
+-+--++--++
| html5lib| 130 ms   | 132 
ms | 1.01x slower | Not significant|
+-+--++--++
| json_dumps  | 18.4 ms  | 18.3 
ms| 1.00x faster | Not significant|
+-+--++--++
| json_loads  | 38.6 us  | 38.6 
us| 1.00x faster | Not significant|

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg339665

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

12718-5f06333a4e49.json.gz
==

Performance version: 0.7.0
Report on macOS-10.14.4-x86_64-i386-64bit
Number of logical CPUs: 8
Start date: 2019-04-08 19:40:57.104845
End date: 2019-04-08 20:02:37.392894

+-+--++--++
| Benchmark   | 2019-01-18_22-05-master-2cf1ddaff4c8.json.gz |  
PR12718   | Change   | Significance   |
+=+==++==++
| 2to3| 522 ms   | 512 
ms | 1.02x faster | Not significant|
+-+--++--++
| chaos   | 184 ms   | 195 
ms | 1.06x slower | Significant (t=-7.09)  |
+-+--++--++
| crypto_pyaes| 169 ms   | 177 
ms | 1.05x slower | Significant (t=-8.85)  |
+-+--++--++
| deltablue   | 11.9 ms  | 12.8 
ms| 1.07x slower | Significant (t=-11.53) |
+-+--++--++
| django_template | 186 ms   | 194 
ms | 1.04x slower | Significant (t=-7.94)  |
+-+--++--++
| dulwich_log | 137 ms   | 151 
ms | 1.11x slower | Significant (t=-11.70) |
+-+--++--++
| fannkuch| 642 ms   | 676 
ms | 1.05x slower | Significant (t=-7.55)  |
+-+--++--++
| float   | 165 ms   | 180 
ms | 1.09x slower | Significant (t=-13.23) |
+-+--++--++
| go  | 394 ms   | 418 
ms | 1.06x slower | Significant (t=-9.24)  |
+-+--++--++
| hexiom  | 15.6 ms  | 15.8 
ms| 1.01x slower | Not significant|
+-+--++--++
| html5lib| 136 ms   | 137 
ms | 1.01x slower | Not significant|
+-+--++--++
| json_dumps  | 19.6 ms  | 20.7 
ms| 1.06x slower | Significant (t=-7.41)  |
+-+--++--++
| json_loads  | 39.8 us  | 42.7 
us| 1.07x slower | Significant (t=-9.45)  |
+-+--++--++
| logging_format  | 16.0 us  | 16.7 
us| 1.05x slower | Significant (t=-5.70)  |
+-+--++--++
| logging_silent  | 318 ns   | 315 
ns | 1.01x faster | Not significant|

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Nick Coghlan]

Nick Coghlan  added the comment:

I was going to note that the algorithm Anthony has pursued here is the same one 
we already use for the list constructor and list.extend(), but Inada-san 
already pointed that out :)

While length_hint is allowed to be somewhat inaccurate, we do expect it to be 
at least *vaguely* accurate (otherwise it isn't very useful, and if it can be 
inaccurate enough to trigger OverflowError or MemoryError in cases that would 
otherwise work reasonably well, it would be better for a type not to implement 
it at all).

While it would be nice to be able to avoid adding a new opcode, the problem is 
that the existing candidate opcodes (BUILD_LIST, BUILD_LIST_UNPACK) are both 
inflexible in what they do:

- BUILD_LIST requires that the final list length be known at compile time
- BUILD_LIST_UNPACK infers the final length from an object reference, but calls 
_PyList_Extend directly, so it combines the preallocation and the iterator 
consumption into a single step, and hence can't be used outside of iterable 
unpacking into a list

At the same time, attempting to generalise either of them isn't desirable, 
since it would slow them down for their existing use cases, and be slower than 
a new opcode for this use case.

The proposed BUILD_LIST_PREALLOC opcode splits the difference: it lets the 
compiler provide the interpreter with a *hint* as to how big the resulting list 
is expected to be.

That said, you'd want to run the result through the benchmark suite rather than 
relying solely on microbenchmarks, as even though unfiltered 
"[some_operation_on_x for x in y]" comprehensions without nested loops or 
filter clauses are pretty common (more common than the relatively new "[*itr]" 
syntax), it's far less clear what the typical distribution in input lengths 
actually is, and how many memory allocations need to be avoided in order to 
offset the cost of the initial _PyObject_LengthHint call (as Pablo's small 
scale results show).

(Note that in the _PyList_Extend code, there are preceding special cases for 
builtin lists and tuples that take those down a much faster path that avoids 
the _PyObject_LengthHint call entirely)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> That is a one-off cost for the __length_hint__ of the range object 
> specifically.
Objects with a known length (lists, sets, tuples) would not have that overhead.

That seems incorrect. This is not unique of range objects as it affects also 
objects with known lengths (like a list):

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x*2 for x in k]",
   setup="k=list(range(10))")


Current master:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 3.82 us +- 0.13 us

PR 12718:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 4.38 us +- 0.16 us


Check also my other benchmark with a list iterator ( iter(list(range(10))) ) or 
this one with a generator comp:

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x*2 for x in it]",
   setup="k=list(range(10));it=(x for x in k)")


Current master:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 945 ns +- 27 ns


PR 12718:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 1.33 us +- 0.05 us

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
Removed message: https://bugs.python.org/msg339634

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> That is a one-off cost for the __length_hint__ of the range object 
> specifically.
Objects with a known length (lists, sets, tuples) would not have that overhead.

That seems incorrect. This is not unique of range objects as it affects also 
objects with known lengths (like a list):

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x*2 for x in k]",
   setup="k=list(range(10))")


Current master:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 3.82 us +- 0.13 us

PR 12718:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 4.38 us +- 0.16 us


Check also my other benchmark with a list iterator ( iter(list(range(10))) ) or 
this one with a generator comp:

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x*2 for x in it]",
   setup="k=list(range(10));it=(x for x in k)")


Current master:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 967 ns +- 27 ns


PR 12718:
❯ ./python.exe ../check.py -n 10
.
list_comp: Mean +- std dev: 1.22 us +- 0.04 us

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

I'm sorry.  list_extend raises OverflowError too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

"useful" doesn't mean "use it as-is".
It is just a hint.  It will be wrong.

See here for list example:
https://github.com/python/cpython/blob/7a0630c530121725136526a88c49589b54da6492/Objects/listobject.c#L929-L940

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> If your patch uses __length_hint__, it is bug.

I’m not sure I understand this comment,

PEP424 says “This is useful for presizing containers when building from an 
iterable.“

This patch uses __length_hint__ to presize the list container for a list 
comprehension.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> If your patch uses __length_hint__, it is bug.
iterator will return 2**1000 for __length_hint__, but produce no item
on iteration.

It raises an OverflowError because of the goto
https://github.com/python/cpython/pull/12718/files#diff-7f17c8d8448b7b6f90549035d2147a9fR2493
 this could just as easily set size to 0.

I put `goto error` given the opportunity to handle an expected fault gracefully.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

> Try
>
> [x for x in range(2**1000)]
>
> in a REPL. It doesn’t raise anything, it tries to create a list that will 
> eventually exceed PY_SIZE_MAX, but it only crashes once it reaches that 
> iteration.

It is expected behavior.

> This raises an OverflowError instead, the same way:
> len(range(2**1000))
> raises an OverflowError

If your patch uses __length_hint__, it is bug.
iterator will return 2**1000 for __length_hint__, but produce no item
on iteration.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> In such case, current behavior works.  And your patch will raise 
> OverflowError.

Try

[x for x in range(2**1000)] 

in a REPL. It doesn’t raise anything, it tries to create a list that will 
eventually exceed PY_SIZE_MAX, but it only crashes once it reaches that 
iteration.

This raises an OverflowError instead, the same way:
len(range(2**1000))
raises an OverflowError

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> This patch makes it slow for small iterators

That is a one-off cost for the __length_hint__ of the range object specifically.
Objects with a known length (lists, sets, tuples) would not have that overhead.

I can run a more useful set of benchmarks against this.

So the +0.6us would be the same for ranges 8-16. Then less for 16-25, then 
again for 25-35 as the removal of the reallocation process has a more 
significant factor for larger ranges.

--
nosy:  -pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

> > This might cause a MemoryError when the __length_hint__ of the source 
> > returns a too large value, even when the actual size of the comprehension 
> > is smaller, e.g.:
>
> The current implementation of list comprehensions raise neither a memoryerror 
> or overflow error. They will consume all available memory and crash the 
> interpreter.
>
> This patch raises an OverflowError before execution instead of just looping 
> until memory heap exhaustion
>

Note PEP 424.

"""
__length_hint__ must return an integer (else a TypeError is raised) or
NotImplemented, and is not required to be accurate. It may return a
value that is either larger or smaller than the actual size of the
container.
"""

it.__length_hint__ can return 2**1000 even if len(list(it))==0.

In such case, current behavior works.  And your patch will raise OverflowError.

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

More benchmarks for slow iterators:

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x**2 for x in k]",
   setup="k=iter(list(range(10)))")


Current master:
❯ ./python.exe ../check.py
.
list_comp: Mean +- std dev: 924 ns +- 35 ns

PR 12718:
❯ ./python.exe ../check.py
.
list_comp: Mean +- std dev: 1.17 us +- 0.06 us

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

> I should have been more explicit, this patch improves the performance of all 
> list comprehensions that don’t have an if clause.

But in these cases, overhead of reallocation will be smaller than simple case.

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> This might cause a MemoryError when the __length_hint__ of the source returns 
> a too large value, even when the actual size of the comprehension is smaller, 
> e.g.:

The current implementation of list comprehensions raise neither a memoryerror 
or overflow error. They will consume all available memory and crash the 
interpreter.

This patch raises an OverflowError before execution instead of just looping 
until memory heap exhaustion

--

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

> How about converting `[x for x in it]` to `[*it]` in AST?

I should have been more explicit, this patch improves the performance of all 
list comprehensions that don’t have an if clause.

Not just
[x for x in y]

but:

d = {} # some sort of dictionary
[f”{k} — {v}” for k, v in d.items()]

a = iterable
[val**2 for val in a]

Would all use BUILD_LIST_PREALLOC and use a LengthHint.

I can do another speed test for those other scenarios. 

Most of the stdlib packages have these sorts of list comps, including those in 
the default site.py.

--
nosy:  -pablogsal

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This patch makes it slow for small iterators:

Perf program:

import perf

runner = perf.Runner()
runner.timeit("list_comp",
   stmt="[x for x in range(10)]",
   setup="")


Current master:
❯ ./python.exe ../check.py
.
list_comp: Mean +- std dev: 3.97 us +- 0.15 us

PR 12718:
❯ ./python.exe ../check.py
.
list_comp: Mean +- std dev: 4.57 us +- 0.17 us

The overhead is very likely due to calling __length_hint__

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

$ python3 -m timeit -s 'r=range(1000)' -- '[x for x in r]'
5000 loops, best of 5: 40 usec per loop

$ python3 -m timeit -s 'r=range(1000)' -- '[*r]'
2 loops, best of 5: 17.3 usec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Inada Naoki]

Inada Naoki  added the comment:

I agree with Serhiy.  Benefit seems too small to add new opcode.

> I could seek another way of implementing the same behaviour without an 
> additional opcode?

How about converting `[x for x in it]` to `[*it]` in AST?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This might cause a MemoryError when the __length_hint__ of the source returns a 
too large value, even when the actual size of the comprehension is smaller, 
e.g.:

 [x**2 for x in range(LARGE_VALUE) if is_prime(x)]

See also issue28940

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

anthony shaw  added the comment:

The opcode would not solely apply to this specific use case.

I could seek another way of implementing the same behaviour without an 
additional opcode?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The benefit is too small to add a new opcode.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

Change by anthony shaw :


--
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[Aaron Hall]

Change by Aaron Hall :


--
nosy: +Aaron Hall

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

Change by anthony shaw :


--
keywords: +patch
pull_requests: +12644
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36551] Optimize list comprehensions with preallocate size and protect against overflow

[anthony shaw]

New submission from anthony shaw :

List comprehensions currently create a series of opcodes inside a code object, 
the first of which is BUILD_LIST with an oparg of 0, effectively creating a 
zero-length list with a preallocated size of 0.

If you're doing a simple list comprehension on an iterator, e.g.

def foo():
  a = iterable
  return [x for x in a]

Disassembly of  at 0x109db2c40, file "", line 3>:
  3   0 BUILD_LIST   0
  2 LOAD_FAST0 (.0)
>>4 FOR_ITER 8 (to 14)
  6 STORE_FAST   1 (x)
  8 LOAD_FAST1 (x)
 10 LIST_APPEND  2
 12 JUMP_ABSOLUTE4
>>   14 RETURN_VALUE

The list comprehension will do a list_resize on the 4, 8, 16, 25, 35, 46, 58, 
72, 88th iterations, etc.

This PR preallocates the list created in a list comprehension to the length of 
the iterator using PyObject_LengthHint().

It uses a new BUILD_LIST_PREALLOC opcode which builds a list with the allocated 
size of PyObject_LengthHint(co_varnames[oparg]).

[x for x in iterable] compiles to:

Disassembly of  at 0x109db2c40, file "", line 3>:
  3   0 BUILD_LIST_PREALLOC  0
  2 LOAD_FAST0 (.0)
>>4 FOR_ITER 8 (to 14)
  6 STORE_FAST   1 (x)
  8 LOAD_FAST1 (x)
 10 LIST_APPEND  2
 12 JUMP_ABSOLUTE4
>>   14 RETURN_VALUE

If the comprehension has ifs, then it will use the existing BUILD_LIST opcode

Testing using a range length of 1

./python.exe -m timeit "x=list(range(1)); [y for y in x]"

Gives 392us on the current 3.8 branch
and 372us with this change (about 8-10% faster)

the longer the iterable, the bigger the impact.

This change also catches the issue that a very large iterator, like a range 
object :
[a for a in range(2**256)]

Would cause the 3.8< interpreter to consume all memory and crash because there 
is no check against PY_SSIZE_MAX currently.

With this change (assuming there is no if inside the comprehension) is now 
caught and thrown as an OverflowError:

>>> [a for a in range(2**256)]
Traceback (most recent call last):
  File "", line 1, in 
  File "", line 1, in 
OverflowError: Python int too large to convert to C ssize_t

--
components: Interpreter Core
messages: 339586
nosy: anthony shaw, ncoghlan
priority: normal
severity: normal
status: open
title: Optimize list comprehensions with preallocate size and protect against 
overflow
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36551>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Lukas Geiger]

Lukas Geiger  added the comment:

Thank you very much for your thorough explanation, totally makes sense.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I concur with all the other -1 comments and will mark this a closed.

FWIW, we generally discourage sweeping across the library with minor rewrites.  
Guido articulated a principle of "holistic refactoring" where we make code 
improvements while working on the module as a whole.  This helps makes sure 
that code changes are made in the context of a thorough understanding of what 
the module is trying to do.  This also helps us reduce maintenance-induced-bugs 
where minor code adjustments create new bugs that weren't there before.

Thank you for taking a look at the source and please continue to do so. If you 
find a specific case that is problematic, feel free to post that one particular 
case.  That said, a better use of time is to take one of the many open tracker 
issues, research it, and propose a fix.

--
nosy: +rhettinger
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

In some places, using a lot is also a deliberate choice to improve readability.

I think the boy scout rule is more appropriate for such changes.

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Agreed with @p-ganssle . Similar issue33152 in the past for the refactoring in 
timeit module to use list comprehension. Large refactorings like this could 
also make git blame little more difficult in places where this has less benefit 
like smaller lists for common cases. I think it's much better to identify hot 
code paths with potentially large lists that could benefit from this along with 
a benchmark that could be accepted by the core dev/maintainer of the module.

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Paul Ganssle]

Paul Ganssle  added the comment:

I think I'm -1 on a general project or policy to replace all for-append loops 
with list comprehensions, but I'm generally positive about individual 
improvements to the code base, on a case-by-case basis.

I can't speak for anyone but myself, but I imagine that if you can identify a 
few places where this sort of change would make a significant performance 
improvement, it would be well-received.

--
nosy: +p-ganssle

___
Python tracker 
<https://bugs.python.org/issue36039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36039] Replace append loops with list comprehensions

[Lukas Geiger]

New submission from Lukas Geiger :

Lib uses loops to append to a new list in quite a few places. I think it would 
be better to replace those with list comprehensions.

Benefits of this change:
  - List comprehensions are generally more readable than appending to a newly 
created list
  - List comprehensions are also a lot faster.
Toy example:
In [1]: %%timeit
   ...: l = []
   ...: for i in range(5000):
   ...: l.append(i)
375 µs ± 1.73 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)

In [2]: %%timeit
   ...: l = [i for i in range(5000)]
168 µs ± 1.08 µs per loop (mean ± std. dev. of 7 runs, 1 loops each)

Possible drawbacks:
  - Refactoring can always introduce bugs and makes it harder to get meaningful 
output from git blame. In this case I think the diff is very manageable, making 
the changes easy to review.

Personally, I think the codebase would benefit from this change both in terms 
of some small performance gains and maintainability. I'd be happy to make a PR 
to fix this.

--
components: Library (Lib)
messages: 335961
nosy: lgeiger
priority: normal
severity: normal
status: open
title: Replace append loops with list comprehensions
type: performance
versions: Python 3.8

___
Python tracker 
<https://bugs.python.org/issue36039>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset b619b097923155a7034c05c4018bf06af9f994d0 by Serhiy Storchaka in 
branch 'master':
bpo-31241: Fix AST node position for list and generator comprehensions. 
(GH-10633)
https://github.com/python/cpython/commit/b619b097923155a7034c05c4018bf06af9f994d0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Brett Cannon]

Brett Cannon  added the comment:

Yeah, it's a tough call because it's one of those things others have probably 
worked around already, so backporting would break the work-arounds.

If you don't want to bother backporting, Serhiy, I think it would be fine to 
not do it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I am not sure what parts of this PR should be backported if either.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

For list comprehensions and generator expressions this is definitely a bug. But 
tuples syntax technically does not include surrounded parentheses.

There is also a problem with generator expression passes as a single argument. 
Generator expression parentheses can be collapsed with function call 
parentheses: f(a for a in b).

PR 10633 makes the following changes:

* Fixes position for list comprehensions and generator expressions.

* If generator expression parentheses are be collapsed with function call 
parentheses, the position of the AST node for the generator expression points 
to the left parenthesis.

* For tuples surrounded with parentheses, the position of the AST node points 
to the left brace. For tuples without parentheses, it points to the position of 
the first tuple item.

I am not sure whether these changes should be backported to maintained versions.

--
nosy: +benjamin.peterson, brett.cannon, yselivanov

___
Python tracker 
<https://bugs.python.org/issue31241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +9880
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31241] ast col_offset wrong for list comprehensions, generators and tuples

[Samuel Colvin]

New submission from Samuel Colvin:

With Python 3.5 and 3.6 list comprehensions, generators and tuples have the 
col_offset for their ast nodes off by 1:

```
import ast
ast.parse('{a for a in range(3)}').body[0].value.col_offset
>> 0  # set comprehension correct

ast.parse('{a: 1 for a in range(3)}').body[0].value.col_offset
>> 0  # dict comprehension correct

ast.parse('[a for a in range(3)]').body[0].value.col_offset
>> 1  # list comprehension wrong!

ast.parse('(a for a in range(3))').body[0].value.col_offset
>> 1  # generator comprehension wrong!

ast.parse('[1, 2, 3]').body[0].value.col_offset
>> 0  # list correct

ast.parse('{1, 2, 3}').body[0].value.col_offset
>> 0  # set correct

ast.parse('{1: 1, 2: 2, 3: 3}').body[0].value.col_offset
>> 0  # dict correct

ast.parse('(1, 2, 3)').body[0].value.col_offset
>> 1  # tuple wrong!
```

I haven't tried 3.4, the issue could be there too.

There are some other related issues #16806 and #21295 but they don't seem quite 
the same.

--
components: Interpreter Core
messages: 300606
nosy: samuelcolvin
priority: normal
severity: normal
status: open
title: ast col_offset wrong for list comprehensions, generators and tuples
versions: Python 3.5, Python 3.6

___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue31241>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Temporary variables in list comprehensions

[Piet van Oostrum]

Vincent Vande Vyvre  writes:

> final = [(x, y+1) for x, y in zip(e, e)]
final = [(x, x+1) for x in e]
-- 
Piet van Oostrum 
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Jussi Piitulainen]

Roel Schroeven writes:

> Lele Gaifax schreef op 6/04/2017 20:07:
>> Piet van Oostrum  writes:
>>
>>> It is a poor man's 'let'. It would be nice if python had a real 'let'
>>> construction. Or for example:
>>>
>>> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>>>
>>> Alas!
>>
>> It would be nice indeed!
>>
>> Or even
>>
>>   [(tmp, tmp + 1) for x in data
>>with expensive_calculation(x) as tmp
>>if tmp is not None]
>>
>
> Perhaps this:
>
> [(tmp, tmp + 1) for tmp in
>  (expensive_calculation(x) for x in data)
>  if tmp is not None]
>
> A bit less elegant, but works right now.

The "poor man's let" works right now.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Roel Schroeven]

Lele Gaifax schreef op 6/04/2017 20:07:

Piet van Oostrum  writes:


It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!


It would be nice indeed!

Or even

  [(tmp, tmp + 1) for x in data
   with expensive_calculation(x) as tmp
   if tmp is not None]



Perhaps this:

[(tmp, tmp + 1) for tmp in
 (expensive_calculation(x) for x in data)
 if tmp is not None]

A bit less elegant, but works right now.


--
Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Tim Chase]

On 2017-04-06 14:56, Vincent Vande Vyvre wrote:
> With two passes
> 
> e = [expensive_calculation(x) for x in data]
> final = [(x, y+1) for x, y in zip(e, e)]

Using a generator it can be done in one pass:

 final = [
   (value, tmp, tmp+1)
   for value, tmp
   in (
 (x, expensive_calculation(x))
 for x in data
 )
   ]

The above makes use of the original value as well at top level
(whether you need it for "if" filtering, or in your final tuple
result). If you don't care, you can discard it

 final = [
   (tmp, tmp+1)
   for tmp
   in (
 expensive_calculation(x)
 for x in data
 )
   ]

-tkc


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Lele Gaifax]

Piet van Oostrum  writes:

> It is a poor man's 'let'. It would be nice if python had a real 'let'
> construction. Or for example:
>
> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>
> Alas!

It would be nice indeed!

Or even

  [(tmp, tmp + 1) for x in data
   with expensive_calculation(x) as tmp
   if tmp is not None]

that fits the usual "with" syntax.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Jussi Piitulainen]

Vincent Vande Vyvre writes:

> Le 06/04/17 à 14:25, Piet van Oostrum a écrit :
>> Steven D'Aprano  writes:
>>
>>> Suppose you have an expensive calculation that gets used two or more
>>> times in a loop. The obvious way to avoid calculating it twice in an
>>> ordinary loop is with a temporary variable:
>>>
>>> result = []
>>> for x in data:
>>>  tmp = expensive_calculation(x)
>>>  result.append((tmp, tmp+1))
>>>
>>>
>>> But what if you are using a list comprehension? Alas, list comps
>>> don't let you have temporary variables, so you have to write this:
>>>
>>>
>>> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>>>
>>>
>>> Or do you? ... no, you don't!
>>>
>>>
>>> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>>>
>>>
>>> I can't decide whether that's an awesome trick or a horrible hack...
>> It is a poor man's 'let'. It would be nice if python had a real 'let'
>> construction. Or for example:
>>
>> [(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]
>>
>> Alas!
>
> With two passes
>
> e = [expensive_calculation(x) for x in data]
> final = [(x, y+1) for x, y in zip(e, e)]
>
> Vincent

Imagine some crazy combinatory question - how many ways can one choose
two subsets of the ten decimal digits so that the size of the first is
the minimum of the second and the size of the second is the maximum of
the first _or_ the minima and maxima of the two are the same?

Comprehensions lend themselves readily to such explorations. It happens
that some expensively computed value is needed twice, like the minima
and maxima of the two combinations in this exercise (because this
exercise was carefully crafted to be just so, but anyway), and then it
saves time to do the computations once: let the values have names.

from itertools import combinations as choose

print(sum(1 for m in range(1,10) for n in range(1,10)
  for a in choose(range(1,10), m)
  for b in choose(range(1,10), n)
  if ((len(a) == min(b) and len(b) == max(a)) or
  (min(a) == min(b) and max(a) == max(b)

print(sum(1 for m in range(1,10) for n in range(1,10)
  for a in choose(range(1,10), m)
  for b in choose(range(1,10), n)
  for lena, mina, maxa in [[len(a), min(a), max(a)]]
  for lenb, minb, maxb in [[len(b), min(b), max(b)]]
  if ((lena == minb and lenb == maxa) or
  (mina == minb and maxa == maxb

I realized afterwards that the sizes, len(a) and len(b), already had
names, m and n, and were only used once in the condition anyway, but let
that illustrate the point: this kind of expression lends itself to
analysis and modification, which is what one wants in explorative code.

(But the "for x in [foo(u,w)]" works, so, shrug, I guess? I'd welcome a
proper let construction, but then I find that I can live without.)
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Vincent Vande Vyvre]

Le 06/04/17 à 14:25, Piet van Oostrum a écrit :

Steven D'Aprano  writes:


Suppose you have an expensive calculation that gets used two or more times in a
loop. The obvious way to avoid calculating it twice in an ordinary loop is with
a temporary variable:

result = []
for x in data:
 tmp = expensive_calculation(x)
 result.append((tmp, tmp+1))


But what if you are using a list comprehension? Alas, list comps don't let you
have temporary variables, so you have to write this:


[(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]


Or do you? ... no, you don't!


[(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]


I can't decide whether that's an awesome trick or a horrible hack...

It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!


With two passes

e = [expensive_calculation(x) for x in data]
final = [(x, y+1) for x, y in zip(e, e)]

Vincent


--
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Piet van Oostrum]

Steven D'Aprano  writes:

> Suppose you have an expensive calculation that gets used two or more times in 
> a 
> loop. The obvious way to avoid calculating it twice in an ordinary loop is 
> with 
> a temporary variable:
>
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
>
>
> But what if you are using a list comprehension? Alas, list comps don't let 
> you 
> have temporary variables, so you have to write this:
>
>
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
>
>
> Or do you? ... no, you don't!
>
>
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]
>
>
> I can't decide whether that's an awesome trick or a horrible hack...

It is a poor man's 'let'. It would be nice if python had a real 'let'
construction. Or for example:

[(tmp, tmp + 1) for x in data with tmp = expensive_calculation(x)]

Alas!
-- 
Piet van Oostrum 
WWW: http://pietvanoostrum.com/
PGP key: [8DAE142BE17999C4]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[breamoreboy]

On Sunday, April 2, 2017 at 1:08:17 AM UTC+1, Robert L. wrote:

> I don't believe in western morality, i.e. don't kill civilians or children
> The only way to fight a moral war is the Jewish way: Destroy their holy sites.
> Kill men, women, and children (and cattle). --- Rabbi Manis Friedman
> web.archive.org/web/20090605154706/http://www.momentmag.com/Exclusive/2009/2009-06/200906-Ask_Rabbis.html
> archive.org/download/DavidDukeVideo/TheZionistMatrixOfPowerddhd.ogv

Completely agree with Steven D'Aprano so would the moderators please ban Robert 
L with immediate effect.

Kindest regards.

Mark Lawrence.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Robert L.]

On 1/8/2017, Steven D'Aprano wrote:

> Suppose you have an expensive calculation that gets used two or
> more times in a loop. The obvious way to avoid calculating it
> twice in an ordinary loop is with a temporary variable:
> 
> result = []
> for x in data:
> tmp = expensive_calculation(x)
> result.append((tmp, tmp+1))
> 
> 
> But what if you are using a list comprehension? Alas, list comps
> don't let you have temporary variables, so you have to write
> this:
> 
> [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data]
> 
> Or do you? ... no, you don't!
> 
> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]]

[2,3,5].map{|n| tmp = Math.sqrt n; [tmp, tmp+1]}

 ===> 
[[1.4142135623730951, 2.414213562373095],
 [1.7320508075688772, 2.732050807568877],
 [2.23606797749979, 3.23606797749979]]


-- 
I don't believe in western morality, i.e. don't kill civilians or children
The only way to fight a moral war is the Jewish way: Destroy their holy sites.
Kill men, women, and children (and cattle). --- Rabbi Manis Friedman
web.archive.org/web/20090605154706/http://www.momentmag.com/Exclusive/2009/2009-06/200906-Ask_Rabbis.html
archive.org/download/DavidDukeVideo/TheZionistMatrixOfPowerddhd.ogv
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Chris Angelico]

On Sun, Apr 2, 2017 at 11:53 AM, Steve D'Aprano
 wrote:
> Robert, I've asked you once to stop posting anti-Semitic signatures in your
> posts. You've now posted four times, and each one has included racist
> material in the signature.
>
> You are welcome to participate here if you discuss Python, or even to
> discuss general programming techniques, but if you continue to post
> anti-Semitic or other racist links and so-called "quotes" (usually invented
> or made up), you will be reported for a CoC violation and banned from the
> mailing list and kill-filed on the newsgroup.

If you were responding to something in this thread, I didn't even see
it, which quite possibly means he's already banned from the mailing
list.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Temporary variables in list comprehensions

[Steve D'Aprano]

Robert, I've asked you once to stop posting anti-Semitic signatures in your
posts. You've now posted four times, and each one has included racist
material in the signature.

You are welcome to participate here if you discuss Python, or even to
discuss general programming techniques, but if you continue to post
anti-Semitic or other racist links and so-called "quotes" (usually invented
or made up), you will be reported for a CoC violation and banned from the
mailing list and kill-filed on the newsgroup.




On Sun, 2 Apr 2017 10:08 am, Robert L. wrote:

> I don't believe in western morality[...]
> archive.org/download/DavidDukeVideo/[...]


-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list