[issue23316] Incorrect evaluation order of function arguments with *args

[Irit Katriel]

Irit Katriel  added the comment:

See also issue33492.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I was pointing out what to me is a second related contradiction in the doc, 
between one sentence and the next.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

I'm currently not very interested in philosophizing about why we allow one
syntax but not the other.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

(The nosy list change was an accident of my local copy not being complete 
refreshed before posting.)

If (b=b, *c) were evaluated in order, then the byte code for b=b and any 
subsequent keyword arguments would have to be put aside, such as in a separate 
buffer, until it was known that there would be no following *exp.  Without 
lookahead, this is known when either **kw or closing ) is reached.  At that 
point, the keyword buffer would be copied to the main buffer.

It might actually be easier to remove all order restrictions and compile all 
keyword values to a side buffer, to be copied to the main buffer when the 
closing ) is reached. 

One version of the question I am raising is this: given that f(a, b=b) and 
f(*(a,), b=b) have the same effect (are alternate spellings of the same 
instruction(s)), why should f(b=b, a) and f(b=b, *(a,)) *not* have the same 
effect, with one spelling being prohibited and the other not?

The meaning of '*expression' is defined as having the same effect as an 
equivalent sequence of positional argument expression in the same place as the 
expression. 

"If the syntax *expression appears in the function call, expression must 
evaluate to an iterable. Elements from these iterables are treated as if they 
were additional positional arguments. For the call f(x1, x2, *y, x3, x4), if y 
evaluates to a sequence y1, …, yM, this is equivalent to a call with M+4 
positional arguments x1, x2, y1, …, yM, x3, x4."

The first sentence of the next paragrapsh, allowing only *exp to follow b=exp, 
contradicts the last line above.  I am sorry I did not read these paregraphs 
carefully until now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

Looks like Terry accidentally removed Serhiy. Adding him back.

--
nosy: +Guido.van.Rossum, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Irit Katriel]

Irit Katriel  added the comment:

It should be possible to make the compiler emit code that evaluates the arg 
values left to right in all cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
versions: +Python 3.11 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Would not be be better in long term to get rid of irregularities? It would make 
the grammar simpler and the documentation clearer.

The only use case of such syntax in wrappers, but they always can be rewritten 
in other style, with natural order of arguments evaluation.

def wrapper(*args, **kw):
return wrapped_fn(*args, some_arg=1, **kw)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar  added the comment:

FYI: https://github.com/PyCQA/pylint/issues/4586

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

Well, it seems we're stuck -- we can't make the grammar more restrictive (at 
least, I don't think we should, since it is a backwards incompatibility), so 
we'll have to live with the exception. Since it is now clear what the rule is, 
we can update the docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also thread "Order of positional and keyword arguments" on the Python-Dev 
mailing list 
(https://mail.python.org/archives/list/python-...@python.org/thread/I6AUYV6DVEMP7XVYVDWC62N6PK6FHX3H/).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The following does not consider keyword-only args with or without defaults.  My 
understanding is the the compiler does not know or considered the signature of 
the function when evaluating the args.

from dis import dis
from itertools import permutations as pm  # My first use of this.

for a1, a2, a3, a4 in pm(('a', 'b=b', '*g', '**k')):
s = f"e({a1}, {a2}, {a3}, {a4})"
print(s)
try:
cd = compile(s, '', 'eval')
except SyntaxError as e:
print(e)
continue
dis(cd)

Since positional arg cannot follow keyword arg or keyword arg unpacking and 
positional arg unpacking cannot follow keyword arg unpacking, only 5 orders of 
'a', 'b=b', '*g', and '**k' are legal.

e(a, b=b, *g, **k)  g before b
e(a, *g, b=b, **k)  normal
e(a, *g, **k, b=b)  normal
e(*g, a, b=b, **k)  normal
e(*g, a, **k, b=b)  normal

For whatever reason, the byte code is more complicated without function 
arguments.

e(a, b=b, *g, **k)
  1   0 LOAD_NAME0 (e)
  2 LOAD_NAME1 (a)
  4 BUILD_LIST   1
  6 LOAD_NAME2 (g)
  8 LIST_EXTEND  1
 10 LIST_TO_TUPLE
 12 LOAD_CONST   0 ('b')
 14 LOAD_NAME3 (b)
 16 BUILD_MAP1
 18 LOAD_NAME4 (k)
 20 DICT_MERGE   1
 22 CALL_FUNCTION_EX 1
 24 RETURN_VALUE

The exceptional case essentially says that positional arg unpacking cannot 
follow keyword args.  This is consistent with the other 3 prohibitions.  
Arguments passed by position form one group, those passed by name another.  The 
nice, simple, and consistent rule is that positional args (and unpacking 
thereof) cannot follow keyword args (and unpacking there).

But in this one case, instead of raising SyntaxError like the other 3 
prohibited orders, the compiler in effect rewrites* the code in a legal order 
-- and then compiles as if written that way.  How often does it rewrite code to 
make it correct?

I think it would have been better to have stuck with 'positional before 
keyword'.  Can we consider deprecating something rare and easy to get wrong?

Thinking more, I might prefer leaving the Evaluation Order section alone, as it 
is correct with respect to the internal rewrite, and revise the explanation of 
the quirk in the function doc. We could deprecate the mis-ordering at least in 
the doc.  

* After writing this, I notice Neil's suggestion that PEP 8 'prohibit' named 
params before iterable (positional) unpacking, so that a fixer program could 
literally rewrite in the correct order.  I am suggesting that it is useful to 
take the viewpoint that the compiler is doing this.

--

___
Python tracker 
<https://bugs.python.org/issue23316>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Guido van Rossum]

Guido van Rossum  added the comment:

> "The one exception is in function calls with *expression after a keyword 
> argument: f(x=expr2, *expr1)."

So the question before us is whether to add this phrase to the docs, or to 
tweak the compiler to fix it. It is certainly convenient to update the docs 
rather than expend the resources to change the compiler for what looks like a 
rare corner case.

How certain are we that this is truly the only exception? Can someone read the 
compiler source code related to argument order, or experiment with all the 
different permutations of positional args, keyword args, *args, and **kwargs? 
(Fortunately the evaluation order is independent from the function definition, 
so it's really just all permutations of those four things.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The particular example of left-to-right function evaluation in
https://docs.python.org/3/reference/expressions.html#evaluation-order
is "expr1(expr2, expr3, *expr4, **expr5)".

Joshua's claim, without evidence, that "'f(*a(), b=b())' will evaluate b() 
before a()" was false, as shown by Neil with dis and is now, as shown by Irit 
with a code example.

But Neil also showed, again with dis, that a revised discrepancy claim, 
""'f(b=b(), *a())' will evaluate a() before b()", is true, because reversing 
the order of these particular arguments does not reverse the evaluation order.  
The dis today is equivalent to the one 7 years ago, with CALL_FUNCTION_VAR 
expanded to two operations.

dis.dis('f(b=b(), *a())')
  1   0 LOAD_NAME0 (f)
  2 LOAD_NAME1 (a)
  4 CALL_FUNCTION0
  6 LOAD_CONST   0 ('b')
  8 LOAD_NAME2 (b)
 10 CALL_FUNCTION0
 12 BUILD_MAP1
 14 CALL_FUNCTION_EX 1
 16 RETURN_VALUE

Irit's example, carried one step further confirms this.

>>> f(b=b(), *a())
a
b
f

Although I considered SilentGhost's reading of
https://docs.python.org/3/reference/expressions.html#calls
as too 'loose' (evaluation order of function argument *is* defined in general), 
that section address this exact case.

"A consequence of this is that although the *expression syntax may appear after 
explicit keyword arguments, it is processed before the keyword arguments ..."

I read this as an acknowledgement that this order violates the general rule.

I do wonder whether exception is specific to using a stack machine and 
implementation convenience or whether *all* implementations must follow this 
order.  If C-Python specific, it should be labelled as such.

The doc continues with an example call, for a different 'f', where, unlike 
Irit's 'f', the result is "TypeError: f() got multiple values for keyword 
argument 'a'".  It concludes "It is unusual for both keyword arguments and the 
*expression syntax to be used in the same call, so in practice this confusion 
does not arise."

The problem with mixing unpacking and named values might be even clearer if 
"f(b=1, *(2,1))" were added to the example box.  Perhaps *exp after x=exp 
should be prohibited.

If the exception is not removed from the implementation, then perhaps it should 
be added to the evaluation order section.  Something like

"The one exception is in function calls with *expression after a keyword 
argument: f(x=expr2, *expr1)."

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Irit Katriel]

Irit Katriel  added the comment:

I don't think this is true anymore:

>>> def a(): 
...   print('a')
...   return (1,2)
... 
>>> def b():
...   print('b')
...   return (3,4)
... 
>>> def f(*args, b=None):
...   print('f')
... 
>>> f(*a(), b=b())
a
b
f
>>>

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43227] Backslashes in function arguments in f-string expressions

[Eric V. Smith]

Change by Eric V. Smith :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43227] Backslashes in function arguments in f-string expressions

[Eric V. Smith]

Eric V. Smith  added the comment:

Yes, they're prohibited anywhere inside of the braces {}.

This might be relaxed in the future, but through 3.9 it's still enforced.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43227] Backslashes in function arguments in f-string expressions

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43227] Backslashes in function arguments in f-string expressions

[Thomas Nabelek]

New submission from Thomas Nabelek :

>From https://www.python.org/dev/peps/pep-0498/#escape-sequences:
"Backslashes may not appear inside the expression portions of f-strings"

Is this supposed to be true even for arguments to functions inside of the 
expression?

my_str = "test\ test\ test"
path = f"{my_str.replace(r'\ ', ' ')}"

gives

my_str = "test\ test\ test"
path = f"{my_str.replace(r'\ ', ' ')}"

SyntaxError: f-string expression part cannot include a backslash

--
messages: 386978
nosy: nabelekt
priority: normal
severity: normal
status: open
title: Backslashes in function arguments in f-string expressions
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue43227>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42921] Inferred Optional type of wrapper function arguments

[Joseph Perez]

New submission from Joseph Perez :

`typing.get_type_hints` gives a different result for a wrapper created with 
`functools.wraps` in case of inferred `Optional` arguments (when the default 
value of the argument is None)

```python
from functools import wraps
from typing import get_type_hints

def foo(bar: int = None): ...

@wraps(foo)
def foo2(*args, **kwargs): ...

print(get_type_hints(foo))  # {'bar': typing.Optional[int]}
print(get_type_hints(foo2))  # {'bar': }
```

This is because `get_type_hints` use the defauts of the wrapper (`foo2`) and 
not those of the wrapped function (`foo`).
This is not consistent with some other tools like `inspect.signature` which 
gives the same signature (and thus same default argument) for the wrapped 
function and its wrapper.

I think this case has simply been forgotten in the resolution of 
https://bugs.python.org/issue37838 (fixing `get_type_hints` not taking `wraps` 
in account at all), because inferred `Optional` is a kind deprecated feature 
(https://github.com/python/typing/issues/275).

--
messages: 385005
nosy: joperez
priority: normal
severity: normal
status: open
title: Inferred Optional type of wrapper function arguments
type: behavior

___
Python tracker 
<https://bugs.python.org/issue42921>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Is there a Python profiler that preserves function arguments?

[Go Luhng]

I'm profiling a Python function `foo()` that takes a single argument,
but that argument makes a huge difference in what the function
actually does.

Currently I'm using `cProfile`, which records every call to `foo()` as
if it was the same, preventing me from figuring out what's going on.

Is there a way to get `cProfile`, or any other Python profiler, to
preserve function call arguments, so when I look at the call stack
later (especially using a visualizer like SnakeViz) I can distinguish
between `foo('bar')` and `foo('qux')`?

P.S. arguably this is a code design issue: since `foo('bar')` and
`foo('qux')` do very different things, they should be distinctly-named
separate functions, like `foo_bar()` and `foo_qux()`. However, the
question is whether I can profile them as-is, without refactoring.
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue34792] Tutorial doesn''t discuss / and * function arguments

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I don't think we should backport the changes or modify the file for the 3.7 
branch, so I close the issue. Feel free to reopen if you still think it makes 
sense :)

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34792] Tutorial doesn''t discuss / and * function arguments

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

With PEP 570 implementation docs were expanded : 
https://github.com/python/cpython/commit/b76302ddd0896cb39ce69909349b53db6e7776e2#diff-d764089fca21fdc70d55804714b1cba5

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Re: Get Count of function arguments passed in

[Roel Schroeven]

Sayth Renshaw schreef op 11/09/2019 om 12:11:

I want to allow as many lists as needed to be passed into a function.
But how can I determine how many lists have been passed in?

I expected this to return 3 but it only returned 1.

matrix1 = [[1, -2], [-3, 4],]
matrix2 = [[2, -1], [0, -1]]
matrix3 = [[2, -1], [0, -1]]
# print(add(matrix1, matrix2))

def add(*matrix):
 print(len(locals()))


May I suggest renaming matrix to matrices or matrix_list or something? I 
find that much clearer to read and understand: singular for one object, 
plural for a collection of objects.



def add(*matrices):
print(len(matrices))

--
"Honest criticism is hard to take, particularly from a relative, a
friend, an acquaintance, or a stranger."
-- Franklin P. Jones

Roel Schroeven

--
https://mail.python.org/mailman/listinfo/python-list

Re: Get Count of function arguments passed in

[David Lowry-Duda]

> I expected this to return 3 but it only returned 1.
> 
> matrix1 = [[1, -2], [-3, 4],]
> matrix2 = [[2, -1], [0, -1]]
> matrix3 = [[2, -1], [0, -1]]
> 
> def add(*matrix):
> print(len(locals()))
>
> print(add(matrix1, matrix2))

In this case, locals will be a dictionary with exactly one key. The key 
will be "matrix", and its value will be a tuple of the two matrices 
matrix1 and matrix2. One solution (the wrong solution) to your problem 
would be to instead have

# Don't do this --- this is to illustrate what locals is doing
def add(*matrix):
print(len(locals()['matrix']))

Now let's get to the right way to do this. Python stores the arguments 
specified by *matrix as a tuple of the name `matrix`. Of course, this is 
what locals what telling us, but there is no need to go through locals.

Instead, you can do

def add(*matrix):
print(len(matrix))

and this will tell you how many arguments were passed in.

Good luck!

David Lowry-Duda
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Get Count of function arguments passed in

[ast]

Le 11/09/2019 à 12:11, Sayth Renshaw a écrit :

Hi

I want to allow as many lists as needed to be passed into a function.
But how can I determine how many lists have been passed in?

I expected this to return 3 but it only returned 1.

matrix1 = [[1, -2], [-3, 4],]
matrix2 = [[2, -1], [0, -1]]
matrix3 = [[2, -1], [0, -1]]
# print(add(matrix1, matrix2))

def add(*matrix):
 print(len(locals()))

add(matrix1,matrix2,matrix3)

Cheers

Sayth



It returns 1 because there is only 1 local variable
inside function add. It's a list matrix which contains
the 3 matrix

If you want the number of arguments passed, then
just call: len(matrix)

def add(*matrix):
print(len(matrix))



--
https://mail.python.org/mailman/listinfo/python-list

Re: Get Count of function arguments passed in

[Sayth Renshaw]

On Wednesday, 11 September 2019 20:25:32 UTC+10, Sayth Renshaw  wrote:
> On Wednesday, 11 September 2019 20:11:21 UTC+10, Sayth Renshaw  wrote:
> > Hi
> > 
> > I want to allow as many lists as needed to be passed into a function.
> > But how can I determine how many lists have been passed in?
> > 
> > I expected this to return 3 but it only returned 1.
> > 
> > matrix1 = [[1, -2], [-3, 4],]
> > matrix2 = [[2, -1], [0, -1]]
> > matrix3 = [[2, -1], [0, -1]]
> > # print(add(matrix1, matrix2))
> > 
> > def add(*matrix):
> > print(len(locals()))
> > 
> > add(matrix1,matrix2,matrix3)
> > 
> > Cheers
> > 
> > Sayth
> 
> Tried creating a list of the arguments, however I am creating too many 
> positional arguments.
> 
> matrix1 = [[1, -2], [-3, 4],]
> matrix2 = [[2, -1], [0, -1]]
> matrix3 = [[2, -1], [0, -1]]
> 
> matrix = []
> def add_many(a = list(*matrix)):
> for num in a:
> for i in range(len(matrix[num])):
> for j in range(len(matrix[num])):
> print(matrix[num][i][j] + matrix[num][i][j])
> 
> add_many(matrix1,matrix2)

Last failure for the moment

matrix1 = [[1, -2], [-3, 4],]
matrix2 = [[2, -1], [0, -1]]
matrix3 = [[2, -1], [0, -1]]

matrix = []
def add_many(*matrix):
for num in range(add_many().func_code.co_argcount):
for i in range(len(matrix[num])):
for j in range(len(matrix[num])):
print(matrix[i][j] + matrix[i][j])

add_many(matrix1,matrix2)

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Get Count of function arguments passed in

[Sayth Renshaw]

On Wednesday, 11 September 2019 20:11:21 UTC+10, Sayth Renshaw  wrote:
> Hi
> 
> I want to allow as many lists as needed to be passed into a function.
> But how can I determine how many lists have been passed in?
> 
> I expected this to return 3 but it only returned 1.
> 
> matrix1 = [[1, -2], [-3, 4],]
> matrix2 = [[2, -1], [0, -1]]
> matrix3 = [[2, -1], [0, -1]]
> # print(add(matrix1, matrix2))
> 
> def add(*matrix):
> print(len(locals()))
> 
> add(matrix1,matrix2,matrix3)
> 
> Cheers
> 
> Sayth

Tried creating a list of the arguments, however I am creating too many 
positional arguments.

matrix1 = [[1, -2], [-3, 4],]
matrix2 = [[2, -1], [0, -1]]
matrix3 = [[2, -1], [0, -1]]

matrix = []
def add_many(a = list(*matrix)):
for num in a:
for i in range(len(matrix[num])):
for j in range(len(matrix[num])):
print(matrix[num][i][j] + matrix[num][i][j])

add_many(matrix1,matrix2)
-- 
https://mail.python.org/mailman/listinfo/python-list

Get Count of function arguments passed in

[Sayth Renshaw]

Hi

I want to allow as many lists as needed to be passed into a function.
But how can I determine how many lists have been passed in?

I expected this to return 3 but it only returned 1.

matrix1 = [[1, -2], [-3, 4],]
matrix2 = [[2, -1], [0, -1]]
matrix3 = [[2, -1], [0, -1]]
# print(add(matrix1, matrix2))

def add(*matrix):
print(len(locals()))

add(matrix1,matrix2,matrix3)

Cheers

Sayth
-- 
https://mail.python.org/mailman/listinfo/python-list

[issue35138] timeit documentation should have example with function arguments

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Anders Hovmöller! Example added to Python 3.7 and 3.8 documentation :-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[miss-islington]

miss-islington  added the comment:


New changeset 7f485ea4fc17c5afb38cd0478ff15326fb5a47fc by Miss Islington (bot) 
in branch '3.7':
bpo-35138: Added an example for timeit.timeit with callable arguments (GH-9787)
https://github.com/python/cpython/commit/7f485ea4fc17c5afb38cd0478ff15326fb5a47fc


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8da5ebe11e0cb6599af682b22f7c2b2b7b9debd8 by Victor Stinner 
(Anders Hovmöller) in branch 'master':
bpo-35138: Added an example for timeit.timeit with callable arguments (GH-9787)
https://github.com/python/cpython/commit/8da5ebe11e0cb6599af682b22f7c2b2b7b9debd8


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[miss-islington]

Change by miss-islington :


--
pull_requests: +13205

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[davidak]

Change by davidak :


--
keywords: +patch
pull_requests: +9598
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

There is an open PR that adds a similar example. Is it the same as the one you 
are proposing and I think you can add a review comment to add it with the same 
PR if it's different?

PR : https://github.com/python/cpython/pull/9787

Thanks

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +easy
stage:  -> needs patch
type:  -> enhancement
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35138] timeit documentation should have example with function arguments

[davidak]

New submission from davidak :

It is described here: https://www.pythoncentral.io/time-a-python-function/

I have just tested it. Works as expected.

Best would be if timeit supports this natively.

--
assignee: docs@python
components: Documentation
messages: 329079
nosy: davidak, docs@python
priority: normal
severity: normal
status: open
title: timeit documentation should have example with function arguments
versions: Python 3.6, Python 3.7

___
Python tracker 
<https://bugs.python.org/issue35138>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34792] Tutorial doesn''t discuss / and * function arguments

[Ezio Melotti]

Change by Ezio Melotti :


--
nosy: +ezio.melotti

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34792] Tutorial doesn''t discuss / and * function arguments

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

There is a related issue to document / in signatures though : 
https://bugs.python.org/issue21314

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34792] Tutorial doesn''t discuss / and * function arguments

[Mark Diekhans]

New submission from Mark Diekhans :

https://docs.python.org/3/tutorial/controlflow.html doest not discuss / and * 
arguments (end of positional, keyword required)

--
assignee: docs@python
components: Documentation
messages: 326277
nosy: diekhans, docs@python
priority: normal
severity: normal
status: open
title: Tutorial doesn''t discuss / and * function arguments
type: enhancement
versions: Python 3.7

___
Python tracker 
<https://bugs.python.org/issue34792>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Raymond Hettinger]

Change by Raymond Hettinger :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[miss-islington]

Change by miss-islington :


--
pull_requests: +8728

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +patch
pull_requests: +8727
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Srinivas Reddy T]

Srinivas  Reddy T  added the comment:

Hi Raymond,
   I find your statement hard to understand.I agree with Solstag, it is 
always helpful to have an example.

+1 for solstag wording.

--
nosy: +thatiparthy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Al-Scandar Solstag]

Al-Scandar Solstag  added the comment:

Speaking frankly, I might not have grasped what might happen by reading your 
line. I think having at least a minimal example is crucial. Perhaps:

"Distinct argument patterns, such as `f(1)` and `f(first_arg=1)`, may not cache 
for each other even if the underlying function sees them as equivalent calls."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I propose this, "Distinct argument patterns may be considered to be distinct 
calls with distinct results even if the underlying function sees them as 
equivalent calls."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Al-Scandar Solstag]

Al-Scandar Solstag  added the comment:

Hi Raymond, I think I understand what you mean, and would suggest something 
along the lines of:

"""
Note that lru_cache only guarantees cache matches on the exact way function 
arguments are specified, so the following ways of calling 'def f(a, b=7)' are 
not guaranteed to cache each other: f(1), f(a=1), f(1, 7), f(1, b=7), f(a=1, 
b=7).
"""

--

___
Python tracker 
<https://bugs.python.org/issue33774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Sure, I can add a line mentioning that distinct argument patterns may be 
considered as distinct cache entries even though they otherwise seem to be 
equivalent calls.  That will just be a general statement though. 
The specific details are implementation dependent, have changed over time, and 
may change again in the future.

--
assignee: docs@python -> rhettinger
priority: normal -> low

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33774] Document that @lru_cache caches based on exactly how the function arguments are specified

[R. David Murray]

R. David Murray  added the comment:

Agreed that this should be documented.

--
nosy: +r.david.murray, rhettinger
stage:  -> needs patch
title: Improve doc of @lru_cache to avoid misuse and confusion -> Document that 
@lru_cache caches based on exactly how the function arguments are specified
versions: +Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue33774>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33588] Unicode function arguments aren't preserved

[Steven D'Aprano]

Steven D'Aprano  added the comment:

This does seem to be a normalisation issue:

py> unicodedata.normalize('NFKC', 'ϵαγϕ') == ''.join(func.__code__.co_varnames)
True

so I'm closing this as not a bug.

Mike, thank you for copying and pasting the relevant text into the bug report, 
but for future reference, there is no need to duplicate that information with a 
redundant screen shots of text for bug report. (Unless you are reporting a 
specifically visual bug e.g. a display problem with the turtle module.) The 
content of screen shots cannot be searched for, or copied and pasted, and they 
make it difficult for the blind and visually impaired who use screen readers.

--
nosy: +steven.daprano
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33588] Unicode function arguments aren't preserved

[Mat Leonard]

Mat Leonard  added the comment:

Ah great, then I just need to do something
like unicodedata.normalize('NFKC', 'Ω'). Thanks!

On Sun, May 20, 2018 at 2:59 PM Eric V. Smith 
wrote:

>
> Eric V. Smith  added the comment:
>
> I think you're seeing identifier normalization. See this SO question for a
> description of the issue:
> https://stackoverflow.com/questions/34097193/identifier-normalization-why-is-the-micro-sign-converted-into-the-greek-letter
>
> --
> nosy: +eric.smith
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33588] Unicode function arguments aren't preserved

[Eric V. Smith]

Eric V. Smith  added the comment:

I think you're seeing identifier normalization. See this SO question for a 
description of the issue: 
https://stackoverflow.com/questions/34097193/identifier-normalization-why-is-the-micro-sign-converted-into-the-greek-letter

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33588] Unicode function arguments aren't preserved

[Mat Leonard]

New submission from Mat Leonard <leonard@gmail.com>:

Unicode symbols used as function arguments aren't preserved in the variable 
names available from .__code__.co_varnames. Example shown below.

def func(ϵ, α, γ, ϕ):
pass

varnames = func.__code__.co_varnames
print(varnames)
print('ϵ' == varnames[0])
print('α' == varnames[1])
print('γ' == varnames[2])
print('ϕ' == varnames[3])

>> ('ε', 'α', 'γ', 'φ')
>> False
>> True
>> True
>> False

I wrote some code dependent on using function arguments obtained from 
.__code__.co_varnames in a dictionary. Since the unicode arguments aren't 
preserved from defining the function and .__code__.co_varnames, the lookup in 
the dictionary fails.

Looks like same thing happens with the inspect module (maybe 
.__code__.co_varnames comes from inspect)

inspect.signature(func)
>> 

--
components: Unicode
files: Screen Shot 2018-05-20 at 2.32.05 PM.png
messages: 317201
nosy: ezio.melotti, mcleonard, vstinner
priority: normal
severity: normal
status: open
title: Unicode function arguments aren't preserved
type: behavior
versions: Python 3.6
Added file: https://bugs.python.org/file47608/Screen Shot 2018-05-20 at 2.32.05 
PM.png

___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33588>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Terry J. Reedy]

Terry J. Reedy added the comment:

I expect left to right as documented (and designed by Guido).  His OK or 
clarification would be to intentionally do differently.

--
nosy: +terry.reedy

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

After thinking about this a bit more, my suggestion is not to fix it.  Instead, 
I suggest that PEP 8 be modified to suggest that all positional arguments and 
iterable argument unpackings precede keyword arguments and keyword argument 
unpackings.  Then, a tool like autopep8 is free to reorganize argument lists.  
Such reorganization will not be possible if f(*a(), b=b()) is different than 
f(b=b(), *a()).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

(I also suggest that the evaluation order within a function argument list to be 
defined to be positional and iterable before keyword, otherwise left-to-right — 
rather than strictly left-to-right).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[SilentGhost]

SilentGhost added the comment:

It seems, if I read https://docs.python.org/3/reference/expressions.html#calls 
correctly that the evaluation order of the function arguments is not defined in 
general, as it depends on your use of keyword argument and exact function 
signature. Naturally, f(a(), b()) would be evaluated in the order you're 
expecting.

--
nosy: +SilentGhost

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[R. David Murray]

R. David Murray added the comment:

The resolution of issue 16967 argues that this should probably be considered a 
bug.  It certainly goes against normal Python expectations.  I think it should 
also be considered to be of low priority, though.

--
nosy: +r.david.murray
priority: normal - low
stage:  - needs patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Joshua Landau]

New submission from Joshua Landau:

It is claimed that all expressions are evaluated left-to-right, including in 
functions¹. However,

f(*a(), b=b())

will evaluate b() before a().

¹ https://docs.python.org/3/reference/expressions.html#evaluation-order

--
components: Interpreter Core
messages: 234672
nosy: Joshua.Landau
priority: normal
severity: normal
status: open
title: Incorrect evaluation order of function arguments with *args
type: behavior
versions: Python 3.4, Python 3.5

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

actually, we accept alternation, so maybe a bit to say whether we start with a 
list or a dict followed by a length of the alternating sequence?

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[R. David Murray]

R. David Murray added the comment:

Neil: I presume you are speaking of your in-progress PEP patch, and not the 
current python code?  If so, please keep the discussion of handling this in the 
context of the PEP to the PEP issue.  This issue should be for resolving the 
bug in the current code (if we choose to do so...if the PEP gets accepted for 
3.5 this issue may become irrelevant, as I'm not sure we'd want to fix it in 
3.4 for backward compatibility reasons).

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

Yes, sorry David.  I got linked here from the other issue.  In any case, in the 
current code, the longest alternating sequence possible is 4.  So one way to 
solve this is to change the CALL_FUNCTION parameters to be lists and dicts only 
and then process the final merging in CALL_FUNCTION.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

another option is to add a LIST_EXTEND(stack_difference) opcode that would 
allow us to take the late iterable and extend a list at some arbitrary stack 
position.  I had to add something like that for dicts for the other issue, so 
it would follow that pattern.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23316] Incorrect evaluation order of function arguments with *args

[Neil Girdhar]

Neil Girdhar added the comment:

I assume this is the problem:

 dis.dis('f(*a(), b=b())')
  1   0 LOAD_NAME0 (f)
  3 LOAD_NAME1 (a)
  6 CALL_FUNCTION0 (0 positional, 0 keyword pair)
  9 LOAD_CONST   0 ('b')
 12 LOAD_NAME2 (b)
 15 CALL_FUNCTION0 (0 positional, 0 keyword pair)
 18 CALL_FUNCTION_VAR  256 (0 positional, 1 keyword pair)
 21 RETURN_VALUE

— looks fine.

 dis.dis('f(b=b(), *a())')
  1   0 LOAD_NAME0 (f)
  3 LOAD_NAME1 (a)
  6 CALL_FUNCTION0 (0 positional, 0 keyword pair)
  9 LOAD_CONST   0 ('b')
 12 LOAD_NAME2 (b)
 15 CALL_FUNCTION0 (0 positional, 0 keyword pair)
 18 CALL_FUNCTION_VAR  256 (0 positional, 1 keyword pair)
 21 RETURN_VALUE

Joshua, we could make function calls take:

x lists
y dictionaries
one optional list
z dictionaries

but we as well do all the merging in advance:

one optional list
one optional dictionary
one optional list
one optional dictionary

which is representable in three bits, but four is easier to decode I think.

--
nosy: +neil.g

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23316
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

Meaning of * in the function arguments list

[ast]

Hi

Consider the following to_bytes method from integer class:

int.to_bytes(length, byteorder, *, signed=False)

What doest the '*' in the arguments list means ?
--
https://mail.python.org/mailman/listinfo/python-list

Re: Meaning of * in the function arguments list

[Peter Otten]

ast wrote:

 Consider the following to_bytes method from integer class:
 
 int.to_bytes(length, byteorder, *, signed=False)
 
 What doest the '*' in the arguments list means ?

A bare * indicates that the arguments that follow it are keyword-only:

 def f(a, b=2, *, c=3):
... print(a = {}, b = {}, c = {}.format(a, b, c))
... 
 f(10)
a = 10, b = 2, c = 3
 f(10, 20)
a = 10, b = 20, c = 3
 f(10, 20, 30)
Traceback (most recent call last):
  File stdin, line 1, in module
TypeError: f() takes from 1 to 2 positional arguments but 3 were given
 f(10, 20, c=30)
a = 10, b = 20, c = 30


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Meaning of * in the function arguments list

[ast]

Peter Otten __pete...@web.de a écrit dans le message de 
news:mailman.15291.1414574006.18130.python-l...@python.org...


A bare * indicates that the arguments that follow it are keyword-only:



ok, thx 


--
https://mail.python.org/mailman/listinfo/python-list

Re: Meaning of * in the function arguments list

[Terry Reedy]

On 10/29/2014 4:56 AM, ast wrote:


Consider the following to_bytes method from integer class:
int.to_bytes(length, byteorder, *, signed=False)
What doest the '*' in the arguments list means ?


If you go to the online doc index page for Symbols,
https://docs.python.org/3/genindex-Symbols.html
there a 3 entries for the use of * as an operator, in statements (in 
particular, def for functions), and in function calls.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Roundup Robot]

Roundup Robot added the comment:

New changeset f87c2c4f03da by Antoine Pitrou in branch 'default':
Issue #11271: concurrent.futures.Executor.map() now takes a *chunksize*
https://hg.python.org/cpython/rev/f87c2c4f03da

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Antoine Pitrou added the comment:

Sorry for the delay. I simplified the patch a tiny bit (yield from wasn't 
needed, it was sufficient to result the itertools.chain.from_iterable() 
result), and committed it. Thank you!

--
resolution:  - fixed
stage: patch review - resolved
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

Hey, my first committed patch :) Thanks for helping to push this through, 
Antoine!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

A couple of small updates based on comments from Charles-François Natali:

* Use itertools.chain.from_iterable to yield from the result chunks instead 
of a for loop.

* Add more tests with varying chunksizes.

--
Added file: http://bugs.python.org/file36352/map_chunksize_docs_update.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +neologix
stage:  - patch review
versions: +Python 3.5 -Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thanks for the update! This looks basically good, I'll wait a bit to see if 
other people have comments, otherwise I'll commit.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

Here's an updated patch that adds documentation and Antoine's requested code 
changes.

--
Added file: http://bugs.python.org/file36306/map_chunksize_with_docs.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Josh Rosenberg]

Changes by Josh Rosenberg shadowranger+pyt...@gmail.com:


--
nosy: +josh.rosenberg

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

I've attached an updated patch based on your review comments; there's now a 
unit test with a non-default chunksize, the chunking algorithm is more 
readable, and the same code path is used no matter what chunksize is provided.

I've also updated the test script to match the implementation changes.

--
Added file: http://bugs.python.org/file36184/map_chunksize_with_test.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Removed file: http://bugs.python.org/file36065/test_mult.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Added file: http://bugs.python.org/file36185/test_mult.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thank you for the patch. I've posted some review comments. Two further remarks:

- this makes the ProcessPool API slightly different from the ThreadPool one. I 
wonder if this is acceptable or not. Thoughts?

- the patch would need unit tests for the additional functionality

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

re: Diverging ThreadPoolExecutor and ProcessPoolExecutor APIs. I thought about 
this as well. It would of course be possible to make ThreadPoolExecutor's API 
match, but it would serve no useful purpose that I can think of. I guess we 
could make the ThreadPoolExecutor API accept the chunksize argument just so the 
APIs match, but then not actually use it (and document that it's not used, of 
course). That would make it easier to switch between the two APIs, at least.

On the other hand, the two APIs are slightly different already: 
ThreadPoolExecutor requires max_workers be supplied, while ProcessPoolExecutor 
will default to multiprocessing.cpu_count(). So this wouldn't completely be 
without precedent.

Anyway, I will review your comments on the patch, add unit tests, and 
re-submit. Thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Antoine Pitrou added the comment:

Thank you for posting this, I'm reopening the issue.

--
nosy: +sbt
resolution: out of date - 
status: closed - open

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Removed file: http://bugs.python.org/file36059/map_chunksize.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Added file: http://bugs.python.org/file36064/map_chunksize.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Removed file: http://bugs.python.org/file36058/test_mult.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Added file: http://bugs.python.org/file36065/test_mult.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Removed file: http://bugs.python.org/file36064/map_chunksize.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Changes by Dan O'Reilly oreil...@gmail.com:


Added file: http://bugs.python.org/file36067/map_chunksize.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

I've added new versions of the patch/demonstration that ensures we actually 
submit all the futures before trying to yield from the returned iterator. 
Previously we were just returning a generator object when map was called, 
without actually submitting any futures.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

I'm seeing an even larger difference between multiprocessing.Pool and 
ProcessPoolExecutor on my machine, with Python 3.4:

Starting multiproc...done in 2.160644769668579 s.
Starting futures...done in 67.953957319259644 s.
Starting futures fixed...done in 2.134932041168213 s.

I've updated the initial patch to address the comments Antoine made; the 
chunksize now defaults to 1, and itertools is used to chunk the input 
iterables, rather than building a list. Attached is an updated benchmark script:

Starting multiproc...done in 2.2295100688934326 s.
Starting futures...done in 68.5991039276123 s.
Starting futures fixed (no chunking)...done in 69.18992304801941 s.
Starting futures fixed (with chunking)...done in 2.352942705154419 s.

The new implementation of map has essentially identical performance to the 
original with chunksize=1, but it performs much better with a larger chunksize 
provided.

--
nosy: +dan.oreilly
Added file: http://bugs.python.org/file36058/test_mult.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Dan O'Reilly]

Dan O'Reilly added the comment:

Here's a patch that adds the new map implementation from the benchmark script 
to concurrent.futures.process.

--
keywords: +patch
Added file: http://bugs.python.org/file36059/map_chunksize.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

ALL function arguments in a common dictionary

[Wolfgang Maier]

Hi everybody,
what is the recommended way of stuffing *all* function arguments (not just
the ones passed by **kwargs) into a common dictionary?

The following sort of works when used as the first block in a function:
try:
kwargs.update(locals())
except NameError:
kwargs = locals().copy()

except that it's nesting pre-existing kwargs.

Thanks for your help,
Wolfgang

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ALL function arguments in a common dictionary

[Fábio Santos]

A quick hack:

 from inspect import getargspec

 def func(a, b, c=3, **kwargs):
... out_dict = {}
... args, _, keywords, _ = getargspec(func)
... for argname in args:
... out_dict[argname] = locals()[argname]
... if keywords:
... out_dict.update(locals()[keywords])
... return out_dict
...
 func(1,2, gah=123)
{'a': 1, 'c': 3, 'b': 2, 'gah': 123}


On Wed, Apr 24, 2013 at 2:36 PM, Wolfgang Maier
wolfgang.ma...@biologie.uni-freiburg.de wrote:
 Hi everybody,
 what is the recommended way of stuffing *all* function arguments (not just
 the ones passed by **kwargs) into a common dictionary?

 The following sort of works when used as the first block in a function:
 try:
 kwargs.update(locals())
 except NameError:
 kwargs = locals().copy()

 except that it's nesting pre-existing kwargs.

 Thanks for your help,
 Wolfgang

 --
 http://mail.python.org/mailman/listinfo/python-list



-- 
Fábio Santos
-- 
http://mail.python.org/mailman/listinfo/python-list

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Brian Quinlan]

Changes by Brian Quinlan br...@sweetapp.com:


--
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14918] Incorrect TypeError message for wrong function arguments

[Jesús Cea Avión]

Changes by Jesús Cea Avión j...@jcea.es:


--
nosy: +jcea

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14918
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14918] Incorrect TypeError message for wrong function arguments

[Éric Araujo]

Éric Araujo mer...@netwok.org added the comment:

I think this was fixed in 3.3, and maybe backported.  Could you test with the 
latest releases?

--
nosy: +benjamin.peterson, eric.araujo
title: Incorrect explanation of TypeError exception - Incorrect TypeError 
message for wrong function arguments
versions: +Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14918
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14918] Incorrect TypeError message for wrong function arguments

[R. David Murray]

R. David Murray rdmur...@bitdance.com added the comment:

It is fixed.  It was not backported.

--
nosy: +r.david.murray
resolution:  - duplicate
stage:  - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue14918
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Brian Quinlan]

Brian Quinlan br...@sweetapp.com added the comment:

I'm closing this since tbrink didn't respond to pitrou's comments.

--
resolution:  - out of date

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue11271] concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments by chunks

[Antoine Pitrou]

Antoine Pitrou pit...@free.fr added the comment:

Using your test script fixed (on Python 3.3), I get the following numbers:

Starting multiproc...done in 2.1014609336853027 s.
Starting futures...done in 20.209479093551636 s.
Starting futures fixed...done in 2.026125907897949 s.

So there's a 0.2ms overhead per remote function call here 
(20/(10010-1)).

Can't your chunks() function use itertools.islice()?

Also, the chunksize can't be anything else than 1 by default, since your 
approach is increasing latency of returning results.

--
nosy: +pitrou
title: concurrent.futures.ProcessPoolExecutor.map() slower than 
multiprocessing.Pool.map() for fast function argument - 
concurrent.futures.ProcessPoolExecutor.map() doesn't batch function arguments 
by chunks
versions: +Python 3.3 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11271
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com