Re: [Python-Dev] chained assignment weirdity

[Ned Batchelder]

On 11/7/2012 5:11 PM, Terry Reedy wrote:

On 11/7/2012 4:39 PM, Ned Batchelder wrote:


Just to be clear: the reference guide says that the behavior *SHOULD BE*
(but is not yet) this:

Python 3.3.0
 >>> {print("a"):print("b")}
a
b
{None: None}
 >>> d = {}
 >>> d[print("a")] = print("b")
b
a
 >>>

Is this or is this not "weird" to you?


Not weird. Expressions and assignment targets are each consistently 
evaluated left to right (except as *necessarily* alter by precedence), 
with expressions evaluated before targets.


Sorry, I should have been clearer: I was asking about weird not to say, 
"This is weird and should be changed!", but to get clarification from 
Serhiy about his statement, " It will be weird if a dict comprehension 
and a plain loop will be inconsistent."  I honestly didn't know which 
behavior he considered inconsistent and therefore weird.


--Ned.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] urlretrieve regression no.2, now in Python 3.3

[Gregory P. Smith]

On Wed, Nov 7, 2012 at 2:42 PM, Terry Reedy  wrote:
> On 11/7/2012 5:57 AM, anatoly techtonik wrote:
>>
>> urlretrieve has a callback parameter, which takes function with the
>> following prototype:
>>
>>def callback(block_number, block_size, total_size):
>>  pass
>>
>> Where block_size was constant and block_size*block_number gave an exact
>> number of transferred bytes.
>
>
> The 3.2 and 3.3 docs both say "The third argument, if present, is a hook
> function that will be called once on establishment of the network connection
> and once after each block read thereafter. The hook will be passed three
> arguments; a count of blocks transferred so far, a block size in bytes, and
> the total size of the file. The third argument may be -1 on older FTP
> servers which do not return a file size in response to a retrieval request."
>
> The word 'constant' does not appear. The product is still the same.
>
>
>> Recent change in Python 3.3 changed the semantic of block_size and broke
>> my `wget` package. http://bugs.python.org/issue16409
>
>
> The only change is that blocksize is now reported as 0 before any blocks
> have been transmitted. It is a side-effect of commits for
> http://bugs.python.org/issue10050.

two changes.  the first block size is 0.  the last block size is the
number of bytes read.

It is a trivial two line change to undo this and I think it would be
wise.  It is difficult to even find documentation on what the
arguments to that callback mean.  Look up the urlretrieve() function
docs in 2.7 and 3.3 and it doesn't tell you there or refer you to
anywhere else to find out.

I'm in favor of restoring and documenting the old behavior for this
legacy urlretrieve API to make existing code being ported from 2.7 not
break.

-gps

>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #16218: skip test if filesystem doesn't support required encoding

[R. David Murray]

On Wed, 07 Nov 2012 23:47:13 +0100, Victor Stinner  
wrote:
> 2012/11/7 Alexandre Vassalotti :
> > The Unicode code points in the U+DC00-DFFF range (low surrogate area) can't
> > be encoded in UTF-8. Quoting from RFC 3629:
> >
> > The definition of UTF-8 prohibits encoding character numbers between U+D800
> > and U+DFFF, which are reserved for use with the UTF-16 encoding form (as
> > surrogate pairs) and do not directly represent characters.
> >
> >
> > It looks like this test was doing something specific with regards to this.
> > So, I am curious as well about this change.
> 
> os.fsencode() uses the surrogateescape error handler (PEP 393) on UNIX.
> 
> >>> os.fsencode('\udcf1\udcea\udcf0\udce8\udcef\udcf2')
> b'\xf1\xea\xf0\xe8\xef\xf2'
> 
> I replaced this arbitrary string (and other similar constant strings)
> with support.FS_NONASCII which is more portable (should be available
> on all locale encodings... except ASCII) and documented.
> 
> I rewrote test_cmd_line_script.test_non_ascii() (and other tests) in
> Python 3.4 to use support.FS_NONASCII.
> 
> This change should improve code coverage on heterogeneous environments.

Alexandre's point was that the string did not appear to be arbitrary,
but rather appeared to specifically be a string containing surrogates.
Is this not the case?

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #16218: skip test if filesystem doesn't support required encoding

[Victor Stinner]

2012/11/7 Alexandre Vassalotti :
> The Unicode code points in the U+DC00-DFFF range (low surrogate area) can't
> be encoded in UTF-8. Quoting from RFC 3629:
>
> The definition of UTF-8 prohibits encoding character numbers between U+D800
> and U+DFFF, which are reserved for use with the UTF-16 encoding form (as
> surrogate pairs) and do not directly represent characters.
>
>
> It looks like this test was doing something specific with regards to this.
> So, I am curious as well about this change.

os.fsencode() uses the surrogateescape error handler (PEP 393) on UNIX.

>>> os.fsencode('\udcf1\udcea\udcf0\udce8\udcef\udcf2')
b'\xf1\xea\xf0\xe8\xef\xf2'

I replaced this arbitrary string (and other similar constant strings)
with support.FS_NONASCII which is more portable (should be available
on all locale encodings... except ASCII) and documented.

I rewrote test_cmd_line_script.test_non_ascii() (and other tests) in
Python 3.4 to use support.FS_NONASCII.

This change should improve code coverage on heterogeneous environments.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] urlretrieve regression no.2, now in Python 3.3

[Terry Reedy]

On 11/7/2012 5:57 AM, anatoly techtonik wrote:

urlretrieve has a callback parameter, which takes function with the
following prototype:

   def callback(block_number, block_size, total_size):
 pass

Where block_size was constant and block_size*block_number gave an exact
number of transferred bytes.


The 3.2 and 3.3 docs both say "The third argument, if present, is a hook 
function that will be called once on establishment of the network 
connection and once after each block read thereafter. The hook will be 
passed three arguments; a count of blocks transferred so far, a block 
size in bytes, and the total size of the file. The third argument may be 
-1 on older FTP servers which do not return a file size in response to a 
retrieval request."


The word 'constant' does not appear. The product is still the same.


Recent change in Python 3.3 changed the semantic of block_size and broke
my `wget` package. http://bugs.python.org/issue16409


The only change is that blocksize is now reported as 0 before any blocks 
have been transmitted. It is a side-effect of commits for

http://bugs.python.org/issue10050.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Steven D'Aprano]

On 08/11/12 08:39, Ned Batchelder wrote:


Just to be clear: the reference guide says that the behavior *SHOULD BE* (but 
is not yet) this:

Python 3.3.0

>> {print("a"):print("b")}

a
b
{None: None}



That was the behaviour of Python 2.4:

py> def pr(x):
... print x
...
py> {pr(1): pr(2), pr(3): pr(4)}
1
2
3
4
{None: None}


2.5 changed to the behaviour seen now, that is, it prints 2 1 4 3
in that order.



>> d = {}
>> d[print("a")] = print("b")

b
a

Is this or is this not "weird" to you?


Not weird to me. The first case has no assignment, so it operates
left to right without exception. The second case has an assignment,
so it operates left to right with a single exception, the right
hand side of the assignment is evaluated before the left hand side(s).

This gives a single, intuitive[1] order of evaluation (left to right),
with the fewest number of exceptions necessary[2]. Using Python 2.4
again:


py> d = {}
py> d[pr(1)] = d[pr(2)] = d[pr(3)] = pr(4) is pr(5)
4
5
1
2
3





[1] Well, intuitive to those whose native language reads left to right.

[2] I assume it is necessary.



--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Issue #16218: skip test if filesystem doesn't support required encoding

[Alexandre Vassalotti]

The Unicode code points in the
U+DC00-DFFF
 range (low surrogate area) can't be encoded in UTF-8. Quoting from
RFC 3629
:

*The definition of UTF-8 prohibits encoding character numbers between
U+D800 and U+DFFF, which are reserved for use with the UTF-16 encoding form
(as surrogate pairs) and do not directly represent characters.*


It looks like this test was doing something specific with regards to this.
So, I am curious as well about this change.



On Sat, Nov 3, 2012 at 10:13 AM, Antoine Pitrou  wrote:

> On Sat,  3 Nov 2012 13:37:48 +0100 (CET)
> andrew.svetlov  wrote:
> > http://hg.python.org/cpython/rev/95d1adf144ee
> > changeset:   80187:95d1adf144ee
> > user:Andrew Svetlov 
> > date:Sat Nov 03 14:37:37 2012 +0200
> > summary:
> >   Issue #16218: skip test if filesystem doesn't support required encoding
> >
> > files:
> >   Lib/test/test_cmd_line_script.py |  7 ++-
> >   1 files changed, 6 insertions(+), 1 deletions(-)
> >
> >
> > diff --git a/Lib/test/test_cmd_line_script.py
> b/Lib/test/test_cmd_line_script.py
> > --- a/Lib/test/test_cmd_line_script.py
> > +++ b/Lib/test/test_cmd_line_script.py
> > @@ -366,7 +366,12 @@
> >  def test_non_utf8(self):
> >  # Issue #16218
> >  with temp_dir() as script_dir:
> > -script_basename = '\udcf1\udcea\udcf0\udce8\udcef\udcf2'
> > +script_basename = '\u0441\u043a\u0440\u0438\u043f\u0442'
>
> Why exactly did you change the tested name here?
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/alexandre%40peadrop.com
>
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Terry Reedy]

On 11/7/2012 4:39 PM, Ned Batchelder wrote:


Just to be clear: the reference guide says that the behavior *SHOULD BE*
(but is not yet) this:

Python 3.3.0
 >>> {print("a"):print("b")}
a
b
{None: None}
 >>> d = {}
 >>> d[print("a")] = print("b")
b
a
 >>>

Is this or is this not "weird" to you?


Not weird. Expressions and assignment targets are each consistently 
evaluated left to right (except as *necessarily* alter by precedence), 
with expressions evaluated before targets.


What is weird -- to me ;-) -- is using side-effects in either example above.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Guido van Rossum]

The dict display is considered an *expression* and thus must follow
the L2R rule. The assignment is explicitly covered by the R2L rule for
assignments (only). Weird or not, those are the rules, and I don't
want to change them.

On Wed, Nov 7, 2012 at 1:39 PM, Ned Batchelder  wrote:
>
> On 11/7/2012 12:08 PM, Serhiy Storchaka wrote:
>
> On 07.11.12 17:12, Nick Coghlan wrote:
>
> Since you've indicated the implementation is in the wrong here and you
> also want to preserve opcode semantics, I think Skip's patch is
> correct, but also needs to be applied to dict comprehensions (now we
> have them). The extra bytecode is only ROT_TWO, which is one of the
> cheapest we have kicking around :)
>
>
> Not only to dict comprehensions, but also to item assignments.  It will be
> weird if a dict comprehension and a plain loop will be inconsistent.
>
>
> Just to be clear: the reference guide says that the behavior *SHOULD BE*
> (but is not yet) this:
>
> Python 3.3.0
 {print("a"):print("b")}
> a
> b
> {None: None}
 d = {}
 d[print("a")] = print("b")
> b
> a

>
> Is this or is this not "weird" to you?
>
> --Ned.
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Ned Batchelder]

On 11/7/2012 12:08 PM, Serhiy Storchaka wrote:

On 07.11.12 17:12, Nick Coghlan wrote:

Since you've indicated the implementation is in the wrong here and you
also want to preserve opcode semantics, I think Skip's patch is
correct, but also needs to be applied to dict comprehensions (now we
have them). The extra bytecode is only ROT_TWO, which is one of the
cheapest we have kicking around :)


Not only to dict comprehensions, but also to item assignments.  It 
will be weird if a dict comprehension and a plain loop will be 
inconsistent.



Just to be clear: the reference guide says that the behavior *SHOULD BE* 
(but is not yet) this:


   Python 3.3.0
>>> {print("a"):print("b")}
   a
   b
   {None: None}
>>> d = {}
>>> d[print("a")] = print("b")
   b
   a
>>>

Is this or is this not "weird" to you?

--Ned.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/ned%40nedbatchelder.com




___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Terry Reedy]

On 11/7/2012 10:17 AM, Guido van Rossum wrote:

Ok, somebody go for it! (Also please refer to my pronouncement in the
bug -- I've gotta run.)


Done. http://bugs.python.org/issue11205?@ok_message=msg 175120

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Terry Reedy]

On 11/7/2012 9:54 AM, Guido van Rossum wrote:


Hm. I really don't think that is a good development for Python to
compromise in the area of expression evaluation order where side
effects are involved.


I agreee. I think Python's simple left to right evaluation order is one 
of its virtues.



A good compiler should be able to detect the
absence of potential side effects. E.g. it might reorder when only
constants and simple variable references are present, or (in the case
of a JIT, which has run-time type information) when it knows enough
about the types of the operands involved to determine that operations
like getattr or getitem are guaranteed side-effect-free.


I call this the 'as-if' rule: the compiler can take shortcuts if the 
result is 'as-if' no shortcut.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Serhiy Storchaka]

On 07.11.12 17:12, Nick Coghlan wrote:

Since you've indicated the implementation is in the wrong here and you
also want to preserve opcode semantics, I think Skip's patch is
correct, but also needs to be applied to dict comprehensions (now we
have them). The extra bytecode is only ROT_TWO, which is one of the
cheapest we have kicking around :)


Not only to dict comprehensions, but also to item assignments.  It will 
be weird if a dict comprehension and a plain loop will be inconsistent.



___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Guido van Rossum]

Ok, somebody go for it! (Also please refer to my pronouncement in the bug
-- I've gotta run.)


On Wed, Nov 7, 2012 at 7:12 AM, Nick Coghlan  wrote:

> On Thu, Nov 8, 2012 at 12:54 AM, Guido van Rossum 
> wrote:
> > On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder 
> wrote:
> >> If the bug report is accurate, CPython and the reference manual have
> >> disagreed since Python 2.5, and many of us are now surprised to hear it,
> >> which means there can't have been much broken code.
> >
> > Give that it was discussed before and fixed before, I think the intent
> > is clear: we should fix the code, not the docs.
>
> Almost certainly, it was broken in the migration to the AST compiler
> and there was no regression test to pick up the change.
>
> > I haven't looked at the proposed fixes, but I think correctness is
> > more important than saving an extra bytecode (OTOH keeping the set of
> > opcodes the same trumps both). I can't imagine that this extra opcode
> > will be significant in many cases.
>
> Since you've indicated the implementation is in the wrong here and you
> also want to preserve opcode semantics, I think Skip's patch is
> correct, but also needs to be applied to dict comprehensions (now we
> have them). The extra bytecode is only ROT_TWO, which is one of the
> cheapest we have kicking around :)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Nick Coghlan]

On Thu, Nov 8, 2012 at 12:54 AM, Guido van Rossum  wrote:
> On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder  wrote:
>> If the bug report is accurate, CPython and the reference manual have
>> disagreed since Python 2.5, and many of us are now surprised to hear it,
>> which means there can't have been much broken code.
>
> Give that it was discussed before and fixed before, I think the intent
> is clear: we should fix the code, not the docs.

Almost certainly, it was broken in the migration to the AST compiler
and there was no regression test to pick up the change.

> I haven't looked at the proposed fixes, but I think correctness is
> more important than saving an extra bytecode (OTOH keeping the set of
> opcodes the same trumps both). I can't imagine that this extra opcode
> will be significant in many cases.

Since you've indicated the implementation is in the wrong here and you
also want to preserve opcode semantics, I think Skip's patch is
correct, but also needs to be applied to dict comprehensions (now we
have them). The extra bytecode is only ROT_TWO, which is one of the
cheapest we have kicking around :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Chris Angelico]

On Thu, Nov 8, 2012 at 1:54 AM, Guido van Rossum  wrote:
> On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder  wrote:
>> We've gone out of our way to
>> maintain backward compatibility with the implemented behavior before
>> (ordering of dict keys, for example).
>
> Not sure what you're referencing here -- didn't we just start
> randomizing hashing?

Hash randomization could have been quietly introduced as a security
fix and backported to old versions. But since applications might have
depended on dict iteration order, it wasn't - old versions of Python
won't randomize hashes unless you set an environment variable. That's
taking care of old code, even when that old code is depending on
non-spec behaviour.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[Nick Coghlan]

On Wed, Nov 7, 2012 at 11:57 PM, Ulrich Eckhardt
 wrote:
> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created on
> line 11"? Or "implied by line 11"? Or how about "Local variable FONT_NAMES
> (implied by line 11) doesn't refer to an object", to avoid the multiple
> interpretations of the term "assignment"?

Unfortunately, we don't track the information we would need in order
to emit that kind of error message. However, you did give me an idea:
I believe the compiler is actually in a position to emit SyntaxWarning
for functions that have a high chance of triggering UnboundLocalError
when called. With output pointing to *both* problematic lines,
beginners should stand a better chance of figuring out what is going
on. (http://bugs.python.org/issue16429)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Guido van Rossum]

On Wed, Nov 7, 2012 at 4:13 AM, Ned Batchelder  wrote:
> I think it's unfortunate that the current patch in the referenced bug (
> http://bugs.python.org/issue11205 ) fixes the "problem" by adding one more
> bytecode to the compiled Python.  The other alternative seems to be changing
> the meaning of two opcodes. Neither of these are great alternatives.  I know
> we don't promise to maintain backward compatibility in the meanings of
> opcodes, but I'll bet actual code will be broken by changing the meaning of
> STORE_MAP and MAP_ADD.  Slowing down dict displays (just slightly) to get
> this right seems unfortunate also.

I agree this would be unfortunate and I recommend that you add this to
the bug. I agree that we should be *very* conservative in changing the
meaning of existing opcodes (adding new one is a different story).

But it was fixed before (http://bugs.python.org/issue448679 has a
simple fix that doesn't change opcode meanings and was applied in
2001) -- what happened?

> If the bug report is accurate, CPython and the reference manual have
> disagreed since Python 2.5, and many of us are now surprised to hear it,
> which means there can't have been much broken code.

Give that it was discussed before and fixed before, I think the intent
is clear: we should fix the code, not the docs.

> I understand the point about C compilers having more opportunities to take
> advantage of "undefined" in the spec, but Python implementations are getting
> cleverer about how to implement Python semantics as well, perhaps we should
> have some faith in their future cleverness and give them even a little bit
> of leeway.

Hm. I really don't think that is a good development for Python to
compromise in the area of expression evaluation order where side
effects are involved. A good compiler should be able to detect the
absence of potential side effects. E.g. it might reorder when only
constants and simple variable references are present, or (in the case
of a JIT, which has run-time type information) when it knows enough
about the types of the operands involved to determine that operations
like getattr or getitem are guaranteed side-effect-free.

> I don't imagine this little bit will actually be useful to them,
> but making this undefined will already help CPython by avoiding either an
> extra bytecode or a change in the opcodes.

I haven't looked at the proposed fixes, but I think correctness is
more important than saving an extra bytecode (OTOH keeping the set of
opcodes the same trumps both). I can't imagine that this extra opcode
will be significant in many cases.

> There are plenty of places where different Python implementations differ,
> and even careful observers had different ideas about how keys and values
> were ordered in dict displays ("I thought it was like a function call,"  vs,
> "I thought it was like an assignment"). We've gone out of our way to
> maintain backward compatibility with the implemented behavior before
> (ordering of dict keys, for example).

Not sure what you're referencing here -- didn't we just start
randomizing hashing?

> The simplest change to make here is
> to update the reference and keep the implementation.

In this particular case I disagree.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Łukasz Rekucki]

On 7 November 2012 15:16, Chris Angelico  wrote:
> On Thu, Nov 8, 2012 at 1:11 AM, Nick Coghlan  wrote:
>> "The implementation is right, the docs are wrong" sounds good to me,
>> as it's easy to justify the out of order evaluation in terms of the
>> equivalent item assignment statements:
>
> What do other Pythons than CPython do currently? Or is it "The
> reference implementation is right, the docs are wrong"?

PyPy and IronPython are the same as CPython. Only Jython (both 2.5 and
2.7a) follows the docs.

Regards,
Łukasz Rekucki
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[R. David Murray]

On Wed, 07 Nov 2012 14:57:57 +0100, Ulrich Eckhardt 
 wrote:
> Am 31.10.2012 23:15, schrieb Steven D'Aprano:
> > On 01/11/12 06:57, anatoly techtonik wrote:
> > [...]
> >> UnboundLocalError: local variable 'FONT_NAMES' referenced before
> >> assignment
> [...]
> >> I wonder if this message can be improved with a
> >> pointer to the concept on when global variables become local?
> >
> > If you have a suggestion for an improved message, please tell us.
> 
> I'll take a shot, since I was also bitten by this when trying to learn 
> Python. The important point is that some code earlier or later in that 
> function does an assignment, so this location should be referenced in 
> the error.
> 
> How about:
> 
>"UnboundLocalError: Local variable 'FONT_NAMES' (created on
> line 11) referenced before assignment."
> 
> What I don't really like is the term "created". Maybe "implicitly 
> created on line 11"? Or "implied by line 11"? Or how about "Local 
> variable FONT_NAMES (implied by line 11) doesn't refer to an object", to 
> avoid the multiple interpretations of the term "assignment"?

The problem is that when Python executes the statement that raises the
error it has no idea where the assignment was done.  All it knows is that
the variable is local.  Keeping track of the location of every assignment
that makes a variable local and writing it in to the .pyc file is a very
non-trivial change to how the Python bytecode compiler works, I think, and
probably not worth the overhead in order to improve this error message.

(And note that raising an error at compile time would change Python's
semantics.)

--David
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[Skip Montanaro]

> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created
> on line 11"? Or "implied by line 11"? Or how about "Local variable
> FONT_NAMES (implied by line 11) doesn't refer to an object", to avoid the
> multiple interpretations of the term "assignment"?

It's been a long (long) while since I looked at the virtual machine
implementation, but my guess is that at the point where the error is
detected the system won't know what line number corresponds to the
assignment.  There might also be multiple assignments.  How would you
know which one to pick?

As for a better word than "created", I would use "assigned."

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Chris Angelico]

On Thu, Nov 8, 2012 at 1:11 AM, Nick Coghlan  wrote:
> "The implementation is right, the docs are wrong" sounds good to me,
> as it's easy to justify the out of order evaluation in terms of the
> equivalent item assignment statements:

What do other Pythons than CPython do currently? Or is it "The
reference implementation is right, the docs are wrong"?

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[Sam Partington]

On 7 November 2012 13:57, Ulrich Eckhardt
 wrote:
> Am 31.10.2012 23:15, schrieb Steven D'Aprano:
> I'll take a shot, since I was also bitten by this when trying to learn
> Python. The important point is that some code earlier or later in that
> function does an assignment, so this location should be referenced in the
> error.

+1

> How about:
>
>   "UnboundLocalError: Local variable 'FONT_NAMES' (created on
>line 11) referenced before assignment."
>
> What I don't really like is the term "created". Maybe "implicitly created on
> line 11"? Or "implied by line 11"? Or how about "Local variable FONT_NAMES
> (implied by line 11) doesn't refer to an object", to avoid the multiple
> interpretations of the term "assignment"?

Why not make use of the well defined words already there in the error
message, but at the line number

   "UnboundLocalError: Local variable 'FONT_NAMES' referenced before
it has been assigned to on line 11."

Sam
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Nick Coghlan]

On Wed, Nov 7, 2012 at 10:13 PM, Ned Batchelder  wrote:
> There are plenty of places where different Python implementations differ,
> and even careful observers had different ideas about how keys and values
> were ordered in dict displays ("I thought it was like a function call,"  vs,
> "I thought it was like an assignment"). We've gone out of our way to
> maintain backward compatibility with the implemented behavior before
> (ordering of dict keys, for example).  The simplest change to make here is
> to update the reference and keep the implementation.

"The implementation is right, the docs are wrong" sounds good to me,
as it's easy to justify the out of order evaluation in terms of the
equivalent item assignment statements:

x = {a:b, c:d}

vs

x = {}
x[a] = b
x[c] = d

That relationship is quite logical given that (ignoring namespace
details) dict construction from a display [1] pretty much does the
equivalent of:

result = {}
for key_expr, val_expr in display_entries:
result[eval(key_expr)] = eval(val_expr)

This comment [2] from the dict comprehension implementation makes it
explicit that the behaviour of the equivalent Python item assignment
code was taken to be normative.

[1] http://hg.python.org/cpython/file/default/Python/compile.c#l3319
[2] http://hg.python.org/cpython/file/default/Python/compile.c#l3020

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Improve error message "UnboundLocalError: local variable referenced before assignment"

[Ulrich Eckhardt]

Am 31.10.2012 23:15, schrieb Steven D'Aprano:

On 01/11/12 06:57, anatoly techtonik wrote:
[...]

UnboundLocalError: local variable 'FONT_NAMES' referenced before
assignment

[...]

I wonder if this message can be improved with a
pointer to the concept on when global variables become local?


If you have a suggestion for an improved message, please tell us.


I'll take a shot, since I was also bitten by this when trying to learn 
Python. The important point is that some code earlier or later in that 
function does an assignment, so this location should be referenced in 
the error.


How about:

  "UnboundLocalError: Local variable 'FONT_NAMES' (created on
   line 11) referenced before assignment."

What I don't really like is the term "created". Maybe "implicitly 
created on line 11"? Or "implied by line 11"? Or how about "Local 
variable FONT_NAMES (implied by line 11) doesn't refer to an object", to 
avoid the multiple interpretations of the term "assignment"?


=just my 2cc=

Uli

**
Domino Laser GmbH, Fangdieckstra�e 75a, 22547 Hamburg, Deutschland
Gesch�ftsf�hrer: Hans Robert Dapprich, Amtsgericht Hamburg HR B62 932
**
Visit our website at http://www.dominolaser.com
**
Diese E-Mail einschlie�lich s�mtlicher Anh�nge ist nur f�r den 
Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte 
benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte 
Empf�nger sein sollten. Die E-Mail ist in diesem Fall zu l�schen und darf 
weder gelesen, weitergeleitet, ver�ffentlicht oder anderweitig benutzt werden.
E-Mails k�nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte 
�nderungen enthalten. Domino Laser GmbH ist f�r diese Folgen nicht 
verantwortlich.
**

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] chained assignment weirdity

[Ned Batchelder]

On 11/6/2012 5:12 PM, Guido van Rossum wrote:

On Tue, Nov 6, 2012 at 9:58 AM, Ned Batchelder  wrote:

On 11/6/2012 11:26 AM, R. David Murray wrote:

On Tue, 06 Nov 2012 18:14:38 +0200, Serhiy Storchaka 
wrote:

Another counterintuitive (and possible wrong) example:

 >>> {print('foo'): print('bar')}
 bar
 foo
 {None: None}

http://bugs.python.org/issue11205


This seems to me better left undefined, since there's hardly ever a need to
know the precise evaluation sequence between keys and values, and retaining
some amount of "unspecified" to allow for implementation flexibility is a
good thing.

Maybe. Do note that Python tries to be *different* than your average
C++ standard and actually prescribes evaluation orders in most cases.
The idea being that the kind of optimizations that C++ compilers get
to do by moving evaluation order around aren't worth it in Python
anyway, and it's better for the user if there are no arbitrary
differences in this area between Python implementations. Note that I
didn't say "no surprises" -- the post that started this thread shows
that surprises are still possible.



I think it's unfortunate that the current patch in the referenced bug ( 
http://bugs.python.org/issue11205 ) fixes the "problem" by adding one 
more bytecode to the compiled Python.  The other alternative seems to be 
changing the meaning of two opcodes. Neither of these are great 
alternatives.  I know we don't promise to maintain backward 
compatibility in the meanings of opcodes, but I'll bet actual code will 
be broken by changing the meaning of STORE_MAP and MAP_ADD.  Slowing 
down dict displays (just slightly) to get this right seems unfortunate also.


If the bug report is accurate, CPython and the reference manual have 
disagreed since Python 2.5, and many of us are now surprised to hear it, 
which means there can't have been much broken code.


I understand the point about C compilers having more opportunities to 
take advantage of "undefined" in the spec, but Python implementations 
are getting cleverer about how to implement Python semantics as well, 
perhaps we should have some faith in their future cleverness and give 
them even a little bit of leeway.  I don't imagine this little bit will 
actually be useful to them, but making this undefined will already help 
CPython by avoiding either an extra bytecode or a change in the opcodes.


There are plenty of places where different Python implementations 
differ, and even careful observers had different ideas about how keys 
and values were ordered in dict displays ("I thought it was like a 
function call,"  vs, "I thought it was like an assignment"). We've gone 
out of our way to maintain backward compatibility with the implemented 
behavior before (ordering of dict keys, for example).  The simplest 
change to make here is to update the reference and keep the implementation.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Fwd: [issue16424] regression: os.path.split('//hostname/foo/bar.txt')

[anatoly techtonik]

Forwarding to python-dev.

Does everybody agree that the following behavior is not a regression, but a
feature of os.path.split()?

Python 3:
>>> import os.path as osp
>>> osp.split('//hostname/foo/')
('//hostname/foo/', '')

Python 2:
>>> osp.split('//hostname/foo/')
('//hostname/foo', '')


But Python 3 again:
>>> osp.split('//hostname/foo/bar/')
('//hostname/foo/bar', '')


http://bugs.python.org/issue16424
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] urlretrieve regression no.2, now in Python 3.3

[anatoly techtonik]

urlretrieve has a callback parameter, which takes function with the
following prototype:

  def callback(block_number, block_size, total_size):
pass

Where block_size was constant and block_size*block_number gave an exact
number of transferred bytes.

Recent change in Python 3.3 changed the semantic of block_size and broke my
`wget` package. http://bugs.python.org/issue16409


Can anybody tell me if I can wait for the fix in 3.3.1 or should implement
a long-term workaround myself?
-- 
anatoly t.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] 回复： Python-Dev Digest, Vol 112, Issue 8

[Amaury Forgeot d'Arc]

Hi,

2012/11/7 邓九祥 :
> I hava some question about Object which "inital" need ,but return "object"
> in mongo shell. So

This is the python-dev mailing list, which deals with development of
the Python language.
You should ask your question on some MongoDB forum.

>
>  BSONElement initial = p["initial"];
> if ( initial.type()
>  != Object ) {
> errmsg = "initial has to be an object";
> return false;
> }
>  "initial.type() != Object " will be found ,so "group" command will not run
> forever. What can'i do
> Mybe this is my fase, please give me some advise.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/amauryfa%40gmail.com
>



-- 
Amaury Forgeot d'Arc
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com