Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Eric V. Smith
On 08/10/2015 10:18 AM, Victor Stinner wrote:
 Hi,
 
 I read the PEP but I don't understand how it is implemented. For me, it
 should be a simple preprocessor:
 
 - f'x={x}' is replaced with 'x={0}'.format(x) by the compiler
 - f'x={1+1}' is replaced with 'x={0}'.format(1+1)
 - f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo())
 - ...
 
 That's all. No new language, no new function or method.

There is no new function or method being proposed. The pre-processor
is being implemented as the ast is being built. As the PEP says, though,
the expressions supported aren't exactly the same, so a simple
conversion to str.format syntax isn't possible.

 It's unclear to me if arbitrary expressions should be allowed or not. If
 not, we may pass parameters by keywords instead:
 
 f'x={x}' is replaced with 'x={x}'.format(x=x) by the compiler
 
 '...'.format(...) is highly optimized. In the current PEP, I see that
 each parameter is rendered in its own independent buffer (.__format__()
 method called multiple times) and then concateneted by ''.join(...).
 It's less efficient that using a single call to .format().

Well, that's what str.format() does: calls __format__ on each expression
and concatenates the results. Except str.format() uses _PyUncicodeWriter
rather than ''.join, so it skips creating the list.

I guess there is a short-circuit in format() where it has an exact
object check for string, float, double (and I think complex), in which
case it will skip the object allocation and basically call the internals
of the object's __format__ method. I could make a similar optimization
here, but it would require a new opcode. I'd like to look at some
benchmarks first.

I don't think such an optimization should drive acceptance or not: let's
decide on the functionality first.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Branch Prediction And The Performance Of Interpreters - Don't Trust Folklore

2015-08-10 Thread Larry Hastings



This just went by this morning on reddit's /r/programming.  It's a paper 
that analyzed Python--among a handful of other languages--to answer the 
question are branch predictors still that bad at the big switch 
statement approach to interpreters?  Their conclusion: no.


   Our simulations [...] show that, as long as the payload in the
   bytecode remains limited and do not feature significant amount of
   extra indirect branches, then the misprediction rate on the
   interpreter can be even become insignificant (less than 0.5 MPKI).

(MPKI = missed predictions per thousand instructions)

Their best results were on simulated hardware with state-of-the-art 
prediction algorithms (TAGE and ITTAGE), but they also demonstrate 
that branch predictors in real hardware are getting better quickly.  
When running the Unladen Swallow test suite on Python 3.3.2, compiled 
with USE_COMPUTED_GOTOS turned off, Intel's Nehalem experienced an 
average of 12.8 MPKI--but Sandy Bridge drops that to 3.5 MPKI, and 
Haswell reduces it further to a mere *1.4* MPKI.  (AFAICT they didn't 
compare against Python 3.3.2 using computed gotos, either in terms of 
MPKI or in overall performance.)


The paper is here:

   https://hal.inria.fr/hal-01100647/document


I suppose I wouldn't propose removing the labels-as-values opcode 
dispatch code yet.  But perhaps that day is in sight!



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Eric V. Smith
On 08/10/2015 01:26 PM, Steven D'Aprano wrote:
 On Sun, Aug 09, 2015 at 06:54:38PM -0700, David Mertz wrote:
 
 Which brought to mind a certain thought.  While I don't like:

 f'My name is {name}, my age next year is {age+1}'

 I wouldn't have any similar objection to:

'My name is {name}, my age next year is {age+1}'.scope_format()

 Or

   scope_format('My name is {name}, my age next year is {age+1}')

 I realize that these could be completely semantically equivalent... but the
 function or method call LOOKS LIKE a runtime operation, while a one letter
 prefix just doesn't look like that (especially to beginners whom I might
 teach).
 
 I fear that this is actually worse than the f-string concept. f-strings, 
 as far as I understand, are literals. (Well, not exactly literals.) You 
 cannot say:
 
 # this can't happen (I think?)
 expr = 'age + 1'
 result = f'blah blah blah {' + expr + '}'
 
 and inject the expression into the f-string. That makes them a little 
 weaker than eval(), and hence a little safer. 

Correct. f-strings only work on literals. They essentially convert the
f-string literal into an expression (which is not strictly specified in
the PEP, but it has examples).

 But scope_format would
 have to be eval in disguise, since it receives a string as argument, 
 and it can't know where it came from or how it came to be:
 
 # pretend that expr comes from, say, a web form
 expr = 'age + 1}{os.system(echo Pwned!) and '
 result = scope_format(
 'My name is {name}, my age next year is {' + expr + '}'
 )
 
 It's a dilemma, because I'm completely with you in your discomfort in 
 having something which looks like a string literal actually be a 
 function of sorts; but turning it into an actual function makes it more 
 dangerous, not less.
 
 I think I would be happy with f-strings, or perhaps i-strings if we use 
 Nick's ideas about internationalisation, and limit what they can 
 evaluate to name lookups, attribute lookups, and indexing, just like 
 format().
 
 We can always relax that restriction in the future, if necessary, but 
 it's a lot harder to tighten it.

This desire, which many people have expressed, is not completely lost on me.

Eric.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Branch Prediction And The Performance Of Interpreters - Don't Trust Folklore

2015-08-10 Thread Maciej Fijalkowski
On Mon, Aug 10, 2015 at 4:44 PM, Larry Hastings la...@hastings.org wrote:


 This just went by this morning on reddit's /r/programming.  It's a paper
 that analyzed Python--among a handful of other languages--to answer the
 question are branch predictors still that bad at the big switch statement
 approach to interpreters?  Their conclusion: no.

 Our simulations [...] show that, as long as the payload in the bytecode
 remains limited and do not feature significant amount of extra indirect
 branches, then the misprediction rate on the interpreter can be even become
 insignificant (less than 0.5 MPKI).

 (MPKI = missed predictions per thousand instructions)

 Their best results were on simulated hardware with state-of-the-art
 prediction algorithms (TAGE and ITTAGE), but they also demonstrate that
 branch predictors in real hardware are getting better quickly.  When running
 the Unladen Swallow test suite on Python 3.3.2, compiled with
 USE_COMPUTED_GOTOS turned off, Intel's Nehalem experienced an average of
 12.8 MPKI--but Sandy Bridge drops that to 3.5 MPKI, and Haswell reduces it
 further to a mere *1.4* MPKI.  (AFAICT they didn't compare against Python
 3.3.2 using computed gotos, either in terms of MPKI or in overall
 performance.)

 The paper is here:

 https://hal.inria.fr/hal-01100647/document


 I suppose I wouldn't propose removing the labels-as-values opcode dispatch
 code yet.  But perhaps that day is in sight!


 /arry

 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com


Hi Larry

Please also note that as far as I can tell this mostly applies to x86.
The ARM branch prediction is significantly dumber these days and as
long as python performance is considered on such platforms such tricks
do make the situation better. We found it out doing CPython/PyPy
comparison, where the difference PyPy vs cPython was bigger on ARM and
smaller on x86, despite our ARM assembler that we produce being less
well optimized.

Cheers,
fijal
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Steven D'Aprano
On Sun, Aug 09, 2015 at 06:14:18PM -0700, David Mertz wrote:

[...]
 That said, there *is* one small corner where I believe f-strings add
 something helpful to the language.  There is no really concise way to spell:
 
   collections.ChainMap(locals(), globals(), __builtins__.__dict__).

I think that to match the normal name resolution rules, nonlocals() 
needs to slip in there between locals() and globals(). I realise that 
there actually isn't a nonlocals() function (perhaps there should be?). 

 If we could spell that as, say `lgb()`, that would let str.format() or
 %-formatting pick up the full what's in scope.  To my mind, that's the
 only good thing about the f-string idea.

I like the concept, but not the name. Initialisms tend to be hard 
to remember and rarely self-explanatory. How about scope()?


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Steven D'Aprano
On Sun, Aug 09, 2015 at 06:54:38PM -0700, David Mertz wrote:

 Which brought to mind a certain thought.  While I don't like:
 
 f'My name is {name}, my age next year is {age+1}'
 
 I wouldn't have any similar objection to:
 
'My name is {name}, my age next year is {age+1}'.scope_format()
 
 Or
 
   scope_format('My name is {name}, my age next year is {age+1}')
 
 I realize that these could be completely semantically equivalent... but the
 function or method call LOOKS LIKE a runtime operation, while a one letter
 prefix just doesn't look like that (especially to beginners whom I might
 teach).

I fear that this is actually worse than the f-string concept. f-strings, 
as far as I understand, are literals. (Well, not exactly literals.) You 
cannot say:

# this can't happen (I think?)
expr = 'age + 1'
result = f'blah blah blah {' + expr + '}'

and inject the expression into the f-string. That makes them a little 
weaker than eval(), and hence a little safer. But scope_format would 
have to be eval in disguise, since it receives a string as argument, 
and it can't know where it came from or how it came to be:

# pretend that expr comes from, say, a web form
expr = 'age + 1}{os.system(echo Pwned!) and '
result = scope_format(
'My name is {name}, my age next year is {' + expr + '}'
)

It's a dilemma, because I'm completely with you in your discomfort in 
having something which looks like a string literal actually be a 
function of sorts; but turning it into an actual function makes it more 
dangerous, not less.

I think I would be happy with f-strings, or perhaps i-strings if we use 
Nick's ideas about internationalisation, and limit what they can 
evaluate to name lookups, attribute lookups, and indexing, just like 
format().

We can always relax that restriction in the future, if necessary, but 
it's a lot harder to tighten it.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Barry Warsaw
On Aug 11, 2015, at 03:26 AM, Steven D'Aprano wrote:

I think I would be happy with f-strings, or perhaps i-strings if we use
Nick's ideas about internationalisation, and limit what they can evaluate to
name lookups, attribute lookups, and indexing, just like format().

I still think you really only need name lookups, especially for an i18n
context.  Anything else is just overkill, YAGNI, potentially error prone, or
perhaps even harmful.

Remember that the translated strings usually come from only moderately (if at
all) trusted and verified sources, so it's entirely possible that a malicious
translator could sneak in an exploit, especially if you're evaluating
arbitrary expressions.  If you're only doing name substitutions, then the
worst that can happen is an information leak, which is bad, but won't
compromise the integrity of say a server using the translation.

Even if the source strings avoid the use of expressions, if the feature is
available, a translator could still sneak something in.  That pretty much
makes it a non-starter for i18n, IMHO.

Besides, any expression you have to calculate can go in a local that will get
interpolated.  The same goes for any !r or other formatting modifiers.  In an
i18n context, you want to stick to the simplest possible substitution
placeholders.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Eric V. Smith
On 08/10/2015 02:31 PM, Barry Warsaw wrote:
 On Aug 11, 2015, at 03:26 AM, Steven D'Aprano wrote:
 
 I think I would be happy with f-strings, or perhaps i-strings if we use
 Nick's ideas about internationalisation, and limit what they can evaluate to
 name lookups, attribute lookups, and indexing, just like format().
 
 I still think you really only need name lookups, especially for an i18n
 context.  Anything else is just overkill, YAGNI, potentially error prone, or
 perhaps even harmful.
 
 Remember that the translated strings usually come from only moderately (if at
 all) trusted and verified sources, so it's entirely possible that a malicious
 translator could sneak in an exploit, especially if you're evaluating
 arbitrary expressions.  If you're only doing name substitutions, then the
 worst that can happen is an information leak, which is bad, but won't
 compromise the integrity of say a server using the translation.
 
 Even if the source strings avoid the use of expressions, if the feature is
 available, a translator could still sneak something in.  That pretty much
 makes it a non-starter for i18n, IMHO.
 
 Besides, any expression you have to calculate can go in a local that will get
 interpolated.  The same goes for any !r or other formatting modifiers.  In an
 i18n context, you want to stick to the simplest possible substitution
 placeholders.

This is why I think PEP-498 isn't the solution for i18n. I'd really like
to be able to say, in a debugging context:

print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')

without having to create locals to hold these 4 values.

Eric.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Eric V. Smith
On 08/10/2015 01:07 PM, Steven D'Aprano wrote:
 On Sun, Aug 09, 2015 at 06:14:18PM -0700, David Mertz wrote:
 
 [...]
 That said, there *is* one small corner where I believe f-strings add
 something helpful to the language.  There is no really concise way to spell:

   collections.ChainMap(locals(), globals(), __builtins__.__dict__).
 
 I think that to match the normal name resolution rules, nonlocals() 
 needs to slip in there between locals() and globals(). I realise that 
 there actually isn't a nonlocals() function (perhaps there should be?). 
 
 If we could spell that as, say `lgb()`, that would let str.format() or
 %-formatting pick up the full what's in scope.  To my mind, that's the
 only good thing about the f-string idea.
 
 I like the concept, but not the name. Initialisms tend to be hard 
 to remember and rarely self-explanatory. How about scope()?

I don't see how you're going to be able to do this in the general case.
Not all variables end up in locals(). See PEP-498's discussion of
closures, for example. Guido has already said locals() and globals()
would not be part of the solution for string interpolation (also in the
PEP).

PEP-498 handles the non-general case: it parses through the string to
find the variables used in the expressions, and then adds them to the
symbol table.

Eric.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Yury Selivanov



On 2015-08-10 2:37 PM, Eric V. Smith wrote:

Besides, any expression you have to calculate can go in a local that will get
interpolated.  The same goes for any !r or other formatting modifiers.  In an
i18n context, you want to stick to the simplest possible substitution
placeholders.

This is why I think PEP-498 isn't the solution for i18n. I'd really like
to be able to say, in a debugging context:

print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')

without having to create locals to hold these 4 values.


Why can't we restrict expressions in f-strings to
attribute/item getters?

I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
disallow f'{foo.bar(baz=something)}'

Yury
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Eric V. Smith
On 08/10/2015 02:44 PM, Yury Selivanov wrote:
 
 
 On 2015-08-10 2:37 PM, Eric V. Smith wrote:
 Besides, any expression you have to calculate can go in a local that
 will get
 interpolated.  The same goes for any !r or other formatting
 modifiers.  In an
 i18n context, you want to stick to the simplest possible substitution
 placeholders.
 This is why I think PEP-498 isn't the solution for i18n. I'd really like
 to be able to say, in a debugging context:

 print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')

 without having to create locals to hold these 4 values.
 
 Why can't we restrict expressions in f-strings to
 attribute/item getters?
 
 I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
 disallow f'{foo.bar(baz=something)}'

It's possible. But my point is that Barry doesn't even want
attribute/item getters for an i18n solution, and I'm not willing to
restrict it that much.

Eric.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread David Mertz
I know. I elided including the nonexistent `nonlocals()` in there. But it
*should* be `lngb()`.  Or call it scope(). :-)
On Aug 10, 2015 10:09 AM, Steven D'Aprano st...@pearwood.info wrote:

 On Sun, Aug 09, 2015 at 06:14:18PM -0700, David Mertz wrote:

 [...]
  That said, there *is* one small corner where I believe f-strings add
  something helpful to the language.  There is no really concise way to
 spell:
 
collections.ChainMap(locals(), globals(), __builtins__.__dict__).

 I think that to match the normal name resolution rules, nonlocals()
 needs to slip in there between locals() and globals(). I realise that
 there actually isn't a nonlocals() function (perhaps there should be?).

  If we could spell that as, say `lgb()`, that would let str.format() or
  %-formatting pick up the full what's in scope.  To my mind, that's the
  only good thing about the f-string idea.

 I like the concept, but not the name. Initialisms tend to be hard
 to remember and rarely self-explanatory. How about scope()?


 --
 Steve
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Yury Selivanov

Eric,

On 2015-08-07 9:39 PM, Eric V. Smith wrote:
[..]

'f-strings are very awesome!'

I'm open to any suggestions to improve the PEP. Thanks for your feedback.



Congrats for the PEP, it's a cool concept!

Overall I'm +1, because a lot of my own formatting code
looks like this:

   'something ... {var1} .. something ... {var2}'.format(
var1=var1, var2=var2)

However, I'm still -1 on a few things.


1. Naming.  How about renaming f-strings to i-strings
(short for interpolated, and, maybe, later for i18n-ed)?

So instead of f'...' we will have i'...'.

There is a parallel PEP 501 by Nick Coghlan proposing
integrating translation mechanisms, and I think, that
i- prefix would allow us to implement PEP 498 first,
and later build upon it.

And, to my ears, i-string sounds way better than
f-string.


2. I'm still not sold on allowing arbitrary expressions
in strings.  There is something about this idea that
conflicts with Python philosophy and its principles.
Supporting arbitrary expressions means that we give a
blessing to shifting parts of application business logic
to string formatting.

I'd hate to see code like this:

print(f'blah blah {self.foobar(spam=ham)!r} blah')

to me it seems completely unreadable, and should be
refactored to

result = self.foobar(spam=ham)
print(f'blah blah {result!r} blah')

The refactored snippet of code is readable even without
advanced syntax highlighting.

Moreover, if we decide to implement Nick's PEP 501, then
supporting expressions in f-strings will cause more harm
than good, as translators usually aren't programmers.

I think that the main reason behind allowing arbitrary
expressions in f-strings is allowing attribute and
item access:

f'{foo.bar} {spam[ham]}'

If that's the case, then can we just restrict expressions
allowed in f-strings to names, attribute and item lookups?
And if later, there is a strong demand for full expressions,
we can add them in 3.7?


Thanks,
Yury

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Carl Meyer
On 08/10/2015 02:49 PM, Eric V. Smith wrote:
 On 08/10/2015 02:44 PM, Yury Selivanov wrote:


 On 2015-08-10 2:37 PM, Eric V. Smith wrote:
 Besides, any expression you have to calculate can go in a local that
 will get
 interpolated.  The same goes for any !r or other formatting
 modifiers.  In an
 i18n context, you want to stick to the simplest possible substitution
 placeholders.
 This is why I think PEP-498 isn't the solution for i18n. I'd really like
 to be able to say, in a debugging context:

 print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')

 without having to create locals to hold these 4 values.

 Why can't we restrict expressions in f-strings to
 attribute/item getters?

 I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
 disallow f'{foo.bar(baz=something)}'
 
 It's possible. But my point is that Barry doesn't even want
 attribute/item getters for an i18n solution, and I'm not willing to
 restrict it that much.

I don't think attribute access and item access are on the same level
here. In terms of readability of the resulting string literal, it would
be reasonable to allow attribute access but disallow item access. And I
think attribute access is reasonable to allow in the context of an i18n
solution as well (but item access is not). Item access is much harder to
read and easier for translators to mess up because of all the extra
punctuation (and the not-obvious-to-a-non-programmer distinction between
a literal or variable key).

There's also the solution used by the Django and Jinja templating
languages, where dot-notation can mean either attribute access
(preferentially) or item access with literal key (as fallback). That
manages to achieve both a high level of readability of the
literal/template, and a high level of flexibility for the context
provider (who may find it easier to provide a dictionary than an
object), but may fail the too different from Python test.

Carl



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Barry Warsaw
On Aug 10, 2015, at 02:49 PM, Eric V. Smith wrote:

It's possible. But my point is that Barry doesn't even want
attribute/item getters for an i18n solution, and I'm not willing to
restrict it that much.

Actually, attribute chasing is generally fine, and flufl.i18n supports that.
Translators can handle $foo.bar although you still do have to be careful about
information leaks (choose your foo's carefully).   Item getters have been
more YAGNI than anything else.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread MRAB

On 2015-08-10 20:23, Guido van Rossum wrote:

On Mon, Aug 10, 2015 at 8:49 PM, Eric V. Smith e...@trueblade.com
mailto:e...@trueblade.com wrote:

On 08/10/2015 02:44 PM, Yury Selivanov wrote:
 On 2015-08-10 2:37 PM, Eric V. Smith wrote:
 This is why I think PEP-498 isn't the solution for i18n. I'd really like
 to be able to say, in a debugging context:

 print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')

 without having to create locals to hold these 4 values.

 Why can't we restrict expressions in f-strings to
 attribute/item getters?

 I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
 disallow f'{foo.bar(baz=something)}'

It's possible. But my point is that Barry doesn't even want
attribute/item getters for an i18n solution, and I'm not willing to
restrict it that much.


I also don't want to tie this closely to i18n. That is (still) very much
a wold of its own.

What I want with f-strings (by any name) is a way to generalize from
print() calls with multiple arguments. We can write

   print('Todo:', len(self.todo), '; busy:', len(self.busy))

but the same thing is more awkward when you have to pass it as a single
string to a function that just sends one string somewhere. And note that
the above example inserts a space before the ';' which I don't really
like. So it would be nice if instead we could write

   print(f'Todo: {len(self.todo)}; busy: {len(self.busy)}')

which IMO is just as readable as the multi-arg print() call[1], and
generalizes to other functions besides print().

In fact, the latter form has less punctuation noise than the former --
every time you insert an expression in a print() call, you have a
quote+comma before it and a comma+quote after it, compared to a brace
before and one after in the new form. (Note that this is an argument for
using f'{...}' rather than '\{...}' -- for a single interpolation it's
the same amount of typing, but for multiple interpolations,
f'{...}{...}' is actually shorter than '\{...}\{...}', and also the \{
part is ugly.)

Anyway, this generalization from print() is why I want arbitrary
expressions. Wouldn't it be silly if we introduced print() today and
said we don't really like to encourage printing complicated
expressions, but maybe we can introduce them in a future version... :-)

Continuing the print()-generalization theme, if things become too long
to fit on a line we can write

   print('Todo:', len(self.todo),
 '; busy:', len(self.busy))

Can we allow the same in f-strings? E.g.

   print(f'Todo: {len(self.todo)
 }; busy: {len(self.busy)
 }')

or is that too ugly? It could also be solved using implicit
concatenation, e.g.

   print(f'Todo: {len(self.todo)}; '
 f'busy: {len(self.busy)}')

[1] Assuming syntax colorizers catch on.


I'd expect f'...' to follow similar rules to '...'.

You could escape it:

print(f'Todo: {len(self.todo)\
}; busy: {len(self.busy)\
}')

which would be equivalent to:

print(f'Todo: {len(self.todo)}; busy: {len(self.busy)}')

or use triple-quoted a f-string:

print(f'''Todo: {len(self.todo)
}; busy: {len(self.busy)
}''')

which would be equivalent to:

print(f'Todo: {len(self.todo)\n}; busy: {len(self.busy)\n}')

(I think it might be OK to have a newline in the expression because
it's wrapped in {...}.)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Nathaniel Smith
On Aug 10, 2015 11:33 AM, Barry Warsaw ba...@python.org wrote:

 On Aug 11, 2015, at 03:26 AM, Steven D'Aprano wrote:

 I think I would be happy with f-strings, or perhaps i-strings if we use
 Nick's ideas about internationalisation, and limit what they can
evaluate to
 name lookups, attribute lookups, and indexing, just like format().

 I still think you really only need name lookups, especially for an i18n
 context.  Anything else is just overkill, YAGNI, potentially error prone,
or
 perhaps even harmful.

 Remember that the translated strings usually come from only moderately
(if at
 all) trusted and verified sources, so it's entirely possible that a
malicious
 translator could sneak in an exploit, especially if you're evaluating
 arbitrary expressions.  If you're only doing name substitutions, then the
 worst that can happen is an information leak, which is bad, but won't
 compromise the integrity of say a server using the translation.

 Even if the source strings avoid the use of expressions, if the feature is
 available, a translator could still sneak something in.  That pretty much
 makes it a non-starter for i18n, IMHO.

 Besides, any expression you have to calculate can go in a local that will
get
 interpolated.  The same goes for any !r or other formatting modifiers.
In an
 i18n context, you want to stick to the simplest possible substitution
 placeholders.

IIUC what Nick contemplates in PEP 501 is that when you write something like
  iI am ${self.age}
then the python runtime would itself evaluate self.age and pass it on to
the i18n machinery to do the actual substitution; the i18n machinery
wouldn't even contain any calls to eval. The above string could be
translated as
  iTengo ${self.age} años
but
  iTengo ${self.password} años
would be an error, because the runtime did not provide a value for
self.password. So while arbitrarily complex expressions are allowed (at
least as far as the language is concerned -- a given project or i18n
toolkit could impose additional policy restrictions), by the time the
interpolation machinery runs they'll effectively have been reduced to local
variables with funny multi-token names.

This pretty much eliminates all the information leak and exploit concerns,
AFAICT. From your comments about having to be careful about attribute
chasing, it sounds like it might even be more robust than current
flufl.i18n in this regard...?

-n
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Le lundi 10 août 2015, Eric V. Smith e...@trueblade.com a écrit :

 On 08/10/2015 10:18 AM, Victor Stinner wrote:
  Hi,
 
  I read the PEP but I don't understand how it is implemented. For me, it
  should be a simple preprocessor:
 
  - f'x={x}' is replaced with 'x={0}'.format(x) by the compiler
  - f'x={1+1}' is replaced with 'x={0}'.format(1+1)
  - f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo())
  - ...
 
  That's all. No new language, no new function or method.

 There is no new function or method being proposed. The pre-processor
 is being implemented as the ast is being built. As the PEP says, though,
 the expressions supported aren't exactly the same, so a simple
 conversion to str.format syntax isn't possible.


Can you please provide example(s) of f-string(s) which cannot be replaced
by a call to .format() like I did?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
PEP 498:

Leading whitespace in expressions is skipped
https://www.python.org/dev/peps/pep-0498/#id28

Because expressions may begin with a left brace ('{'), there is a problem
when parsing such expressions. For example:

 f'{{k:v for k, v in [(1, 2), (3, 4)]}}'
'{k:v for k, v in [(1, 2), (3, 4)]}'



For me, this example is crazy. You should not add a special case (ignore
spaces) just to support a corner case.

This example can easily be rewritten using a temporary variable and it
makes the code simpler.

items={k:v for k, v in [(1, 2), (3, 4)]; f'{items}'

Seriously, a dict-comprehension inside a f-string should be considered as
an abuse of the feature. Don't you think so?

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
By the way, I don't think that fu'...' syntax should be allowed. IMHO
u'...' was only reintroduced to Python 3.3 to ease transition from Python 2
to Python 3 of the existing u'...' Syntax. Since f'...' is a new syntax,
backward compatibility doesn't matter here.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread MRAB

On 2015-08-11 00:26, Victor Stinner wrote:

Le mardi 11 août 2015, Eric V. Smith e...@trueblade.com
mailto:e...@trueblade.com a écrit :

It sounds like you want to disallow leading spaces just to
disallow this one type of expression.


  I would like to reduce the number of subtle differences between
f-string and str.format().


I'm a little bit surprised at seeing this:

 '{0}'.format('foo')
'foo'
 '{ 0}'.format('foo')
Traceback (most recent call last):
  File stdin, line 1, in module
KeyError: ' 0'
 '{a}'.format(a='foo')
'foo'
 '{ a}'.format(a='foo')
Traceback (most recent call last):
  File stdin, line 1, in module
KeyError: ' a'

In some other cases, leading and trailing spaces are ignored:

 int(' 0 ')
0

Outside string literals, they're also ignored.

But, then:

 '{-1}'.format('foo')
Traceback (most recent call last):
  File stdin, line 1, in module
KeyError: '-1'

It's a string key, even though it looks like an int position.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread MRAB

On 2015-08-10 23:54, Victor Stinner wrote:


PEP 498:




Leading whitespace in expressions is skipped
https://www.python.org/dev/peps/pep-0498/#id28

Because expressions may begin with a left brace ('{'), there is a
problem when parsing such expressions. For example:


f'{{k:v for k, v in [(1, 2), (3, 4)]}}'  '{k:v for k, v in [(1, 2), (3, 4)]}'




For me, this example is crazy. You should not add a special case (ignore
spaces) just to support a corner case.


Is it a special case? Don't we already ignore leading spaces in
bracketed expressions?


This example can easily be rewritten using a temporary variable and it
makes the code simpler.

items={k:v for k, v in [(1, 2), (3, 4)]; f'{items}'

Seriously, a dict-comprehension inside a f-string should be considered
as an abuse of the feature. Don't you think so?


True. Or we can wrap it in parentheses. :-)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, Eric V. Smith e...@trueblade.com a écrit :

 Oops, I was thinking of going the other way (str.format - f''). Yes, I
 think you're correct.


Ah ok.

But in any event, I don't see the distinction between calling
 str.format(), and calling each object's __format__ method. Both are
 compliant with the PEP, which doesn't specify exactly how the
 transformation is done.


When I read the PEP for the first time, I understood that you reimplemented
str.format() using the __format__() methods. So i understood that it's a
new formatting language and it would be tricky to reimplement it, for
example in a library providing i18n with f-string syntax (I'm not sure that
it's feasible, it's just an example). I also expected many subtle
differences between .format() and f-string.

In fact, f-string is quite standard and not new, it's just a compact syntax
to call .format() (well, with some minor and acceptable subtle
differences). For me, it's a good thing to rely on the existing .format()
method because it's well known (no need to learn a new formatting language).

Maybe you should rephrase some parts of your PEP and rewrite some examples
to say that's it's just a compact syntax to call .format().

--

For me, calling __format__() multiple times or format() once matters, for
performances, because I contributed to the implementation of
_PyUnicodeWriter. I spent a lot of time to keep good performances when the
implementation of Unicode was rewritten for the PEP 393. With this PEP,
writing an efficient implementation is much harder. The dummy benchmark is
to compare Python 2.7 str.format() (bytes!) to Python 3 str.format()
(Unicode!). Users want similar performances! If I recall correctly, Python
3 is not bad (faster is some corner cases).

Concatenate temporary strings is less efficient Than _PyUnicodeWriter
(single buffer) when you have UCS-1, UCS-2 and UCS-4 strings (1/2/4 bytes
per character). It's more efficient to write directly into the final format
(UCS-1/2/4), even if you may need to convert the buffer from UCS-1 to UCS-2
(and maybe even one more time to UCS-4).

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Eric V. Smith
On 8/10/2015 7:26 PM, Victor Stinner wrote:
 Le mardi 11 août 2015, Eric V. Smith e...@trueblade.com
 mailto:e...@trueblade.com a écrit :
 
 It sounds like you want to disallow leading spaces just to
 disallow this one type of expression.
 
 
  I would like to reduce the number of subtle differences between
 f-string and str.format().

The expressions supported are so vastly different that I don't think the
whitespace issue matters.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Eric V. Smith
On 8/10/2015 8:04 PM, Victor Stinner wrote:
 
 
 Le mardi 11 août 2015, MRAB pyt...@mrabarnett.plus.com
 mailto:pyt...@mrabarnett.plus.com a écrit :
 
 I'm a little bit surprised at seeing this: (...)
 
 
 We may modify str.format to ignore leading spaces, but IMHO it should
 not be motivated by the PEP.

Agreed.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Eric V. Smith
On 8/10/2015 6:22 PM, Victor Stinner wrote:
 Le lundi 10 août 2015, Eric V. Smith e...@trueblade.com
 mailto:e...@trueblade.com a écrit :
 
 On 08/10/2015 10:18 AM, Victor Stinner wrote:
  Hi,
 
  I read the PEP but I don't understand how it is implemented. For
 me, it
  should be a simple preprocessor:
 
  - f'x={x}' is replaced with 'x={0}'.format(x) by the compiler
  - f'x={1+1}' is replaced with 'x={0}'.format(1+1)
  - f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo())
  - ...
 
  That's all. No new language, no new function or method.
 
 There is no new function or method being proposed. The pre-processor
 is being implemented as the ast is being built. As the PEP says, though,
 the expressions supported aren't exactly the same, so a simple
 conversion to str.format syntax isn't possible.
 
 
 Can you please provide example(s) of f-string(s) which cannot be
 replaced by a call to .format() like I did?

Oops, I was thinking of going the other way (str.format - f''). Yes, I
think you're correct.

But in any event, I don't see the distinction between calling
str.format(), and calling each object's __format__ method. Both are
compliant with the PEP, which doesn't specify exactly how the
transformation is done.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Eric V. Smith
On 8/10/2015 6:54 PM, Victor Stinner wrote:
 
 PEP 498:
 
 
 
 
 Leading whitespace in expressions is skipped
 https://www.python.org/dev/peps/pep-0498/#id28
 
 Because expressions may begin with a left brace ('{'), there is a
 problem when parsing such expressions. For example:
 
 f'{{k:v for k, v in [(1, 2), (3, 4)]}}' '{k:v for k, v in [(1, 2), (3, 
 4)]}'
 
 
 
 For me, this example is crazy. You should not add a special case (ignore
 spaces) just to support a corner case.
 
 This example can easily be rewritten using a temporary variable and it
 makes the code simpler.
 
 items={k:v for k, v in [(1, 2), (3, 4)]; f'{items}'
 
 Seriously, a dict-comprehension inside a f-string should be considered
 as an abuse of the feature. Don't you think so?

Yes, it's absolutely an abuse and should never be written. But if the
only cost to allowing it is skipping leading spaces, I don't see the
harm. It sounds like you want to disallow leading spaces just to
disallow this one type of expression.

My other use case for spaces, which I've not added to the PEP yet, is
for alignment, like:

f' x={ x:15}'
f'xx={xx:15}'

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, Eric V. Smith e...@trueblade.com a écrit :

 It sounds like you want to disallow leading spaces just to
 disallow this one type of expression.


 I would like to reduce the number of subtle differences between f-string
and str.format().

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Eric V. Smith
On 8/10/2015 7:23 PM, Victor Stinner wrote:
 But in any event, I don't see the distinction between calling
 str.format(), and calling each object's __format__ method. Both are
 compliant with the PEP, which doesn't specify exactly how the
 transformation is done.
 
 
 When I read the PEP for the first time, I understood that you
 reimplemented str.format() using the __format__() methods. So i
 understood that it's a new formatting language and it would be tricky to
 reimplement it, for example in a library providing i18n with f-string
 syntax (I'm not sure that it's feasible, it's just an example). I also
 expected many subtle differences between .format() and f-string.
 
 In fact, f-string is quite standard and not new, it's just a compact
 syntax to call .format() (well, with some minor and acceptable subtle
 differences). For me, it's a good thing to rely on the existing
 .format() method because it's well known (no need to learn a new
 formatting language).
 
 Maybe you should rephrase some parts of your PEP and rewrite some
 examples to say that's it's just a compact syntax to call .format().

Okay. I'll look at it.

 For me, calling __format__() multiple times or format() once matters,
 for performances, because I contributed to the implementation of
 _PyUnicodeWriter. I spent a lot of time to keep good performances
 when the implementation of Unicode was rewritten for the PEP 393. With
 this PEP, writing an efficient implementation is much harder. The dummy
 benchmark is to compare Python 2.7 str.format() (bytes!) to Python 3
 str.format() (Unicode!). Users want similar performances! If I recall
 correctly, Python 3 is not bad (faster is some corner cases).

'{} {}'.format(datetime.datetime.now(), decimal.Decimal('100'))

calls __format__() twice. It's only special cased to not call __format__
for str, int, float, and complex. I'll grant you that most of the cases
it will ever be used for are thus special cased.

 Concatenate temporary strings is less efficient Than _PyUnicodeWriter
 (single buffer) when you have UCS-1, UCS-2 and UCS-4 strings (1/2/4
 bytes per character). It's more efficient to write directly into the
 final format (UCS-1/2/4), even if you may need to convert the buffer
 from UCS-1 to UCS-2 (and maybe even one more time to UCS-4).

As I said, after it's benchmarked, I'll look at it. It's not a
user-visible change.

And thanks for your work on _PyUnicodeWriter.

Eric.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Cameron Simpson

On 11Aug2015 01:00, Victor Stinner victor.stin...@gmail.com wrote:

By the way, I don't think that fu'...' syntax should be allowed. IMHO
u'...' was only reintroduced to Python 3.3 to ease transition from Python 2
to Python 3 of the existing u'...' Syntax. Since f'...' is a new syntax,
backward compatibility doesn't matter here.


There's another reason to resist the fu'...' prefix: political correctness.

To illustrate, there's a consumer rights TV snow here with a segment called 
F.U. Tube, where members of the public describe ripoffs and other product 
failures in video form. While a phonetic play on the name YouTube, the 
abbreviation also colloquially means just what you think it might. I can just 
imagine reciting one of these new strings out loud...


Cheers,
Cameron Simpson c...@zip.com.au

People shouldn't be allowed to build overpasses ...
   - Dianne I know what's best for you Feinstein after the '94 LA quake.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Victor Stinner
Le mardi 11 août 2015, MRAB pyt...@mrabarnett.plus.com a écrit :

 I'm a little bit surprised at seeing this: (...)


We may modify str.format to ignore leading spaces, but IMHO it should not
be motivated by the PEP.

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP 498 f-string: please remove the special case for spaces

2015-08-10 Thread Eric V. Smith
On 8/10/2015 8:00 PM, MRAB wrote:
 On 2015-08-11 00:26, Victor Stinner wrote:
 Le mardi 11 août 2015, Eric V. Smith e...@trueblade.com
 mailto:e...@trueblade.com a écrit :

 It sounds like you want to disallow leading spaces just to
 disallow this one type of expression.


   I would like to reduce the number of subtle differences between
 f-string and str.format().

 I'm a little bit surprised at seeing this:
 
 '{0}'.format('foo')
 'foo'
 '{ 0}'.format('foo')
 Traceback (most recent call last):
   File stdin, line 1, in module
 KeyError: ' 0'
 '{a}'.format(a='foo')
 'foo'
 '{ a}'.format(a='foo')
 Traceback (most recent call last):
   File stdin, line 1, in module
 KeyError: ' a'
 
 In some other cases, leading and trailing spaces are ignored:
 
 int(' 0 ')
 0
 
 Outside string literals, they're also ignored.
 
 But, then:
 
 '{-1}'.format('foo')
 Traceback (most recent call last):
   File stdin, line 1, in module
 KeyError: '-1'
 
 It's a string key, even though it looks like an int position.

I think there are bug tracker issues for both of these. I think the
argument against changing them is that people might be depending on this
behavior. I'll grant you it seems unlikely, but you never know.

Eric.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Mike Miller

Here are my notes on PEP 498.

1. Title:  Literal String Formatting

- String Literal Formatting
- Format String Expressions
?

2. Let's call them format strings not f-strings.
   The latter sounds slightly obnoxious, and also inconsistent with the
   others:

r'' raw string
u'' unicode object (string)
f'' format string

3.  This PEP does not propose to remove or deprecate any of the existing
string formatting mechanisms. 

Should we put this farther up with the section talking about them,
it seems out of place where it is.

4. The existing ways of formatting are either error prone, inflexible, or
cumbersome.

I would tone this down a bit, they're not so bad, quite verbose is a
phrase I might use instead.

5. Discussion Section
How to designate f-strings, and how specify the locaton of expressions
^ typo

6. Perhaps mention string literal functionality, like triple quotes, line-ending 
backslashes, as MRAB mentions, in addition to the concatenation rules.


-Mike


On 08/07/2015 06:39 PM, Eric V. Smith wrote:
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Wes Turner
On Mon, Aug 10, 2015 at 2:04 PM, Carl Meyer c...@oddbird.net wrote:

 On 08/10/2015 02:49 PM, Eric V. Smith wrote:
  On 08/10/2015 02:44 PM, Yury Selivanov wrote:
 
 
  On 2015-08-10 2:37 PM, Eric V. Smith wrote:
  Besides, any expression you have to calculate can go in a local that
  will get
  interpolated.  The same goes for any !r or other formatting
  modifiers.  In an
  i18n context, you want to stick to the simplest possible substitution
  placeholders.
  This is why I think PEP-498 isn't the solution for i18n. I'd really
 like
  to be able to say, in a debugging context:
 
  print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')
 
  without having to create locals to hold these 4 values.
 
  Why can't we restrict expressions in f-strings to
  attribute/item getters?
 
  I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
  disallow f'{foo.bar(baz=something)}'
 
  It's possible. But my point is that Barry doesn't even want
  attribute/item getters for an i18n solution, and I'm not willing to
  restrict it that much.

 I don't think attribute access and item access are on the same level
 here. In terms of readability of the resulting string literal, it would
 be reasonable to allow attribute access but disallow item access. And I
 think attribute access is reasonable to allow in the context of an i18n
 solution as well (but item access is not). Item access is much harder to
 read and easier for translators to mess up because of all the extra
 punctuation (and the not-obvious-to-a-non-programmer distinction between
 a literal or variable key).

 There's also the solution used by the Django and Jinja templating
 languages, where dot-notation can mean either attribute access
 (preferentially) or item access with literal key (as fallback). That
 manages to achieve both a high level of readability of the
 literal/template, and a high level of flexibility for the context
 provider (who may find it easier to provide a dictionary than an
 object), but may fail the too different from Python test.


References for (these) PEPs:

One advantage of Python HAVING required
explicit template format interpolation string contexts
is that to do string language formatting correctly
(e.g. *for anything other than printing strings to console*
or with formats with defined field/record boundary delimiters
 (which, even then, may contain shell control escape codes))
we've had to write and use external modules
which are specific to the output domain (JSON, HTML, CSS, SQL, SPARQL, CSS,
[...]).

There are a number of posts about operator syntax, which IMHO, regardless,
it's not convenient enough to lose this distinctive 'security' feature
(explicit variable bindings for string interpolation) of Python as a
scripting language as compared to e.g. Perl, Ruby.

Jinja2 reimplements and extends Django template syntax -{% for
%}{{variable_or_expr | filtercallable}}-{% endfor %}

* Jinja2 supports configurable operators {{ can instead be !! or !{ or ${
or ??
* Because it is a compilable function composition, Jinja2 supports
extensions:
https://github.com/mitsuhiko/jinja2/blob/master/tests/test_ext.py
* Jinja2 supports {% trans %}, _(''), and gettext()  babel-style i18n
  http://jinja.pocoo.org/docs/dev/templates/#i18n
* Jinja2 supports autoescaping:
  http://jinja.pocoo.org/docs/dev/api/#autoescaping

(e.g. 'jinja2.ext.autoescape' AutoEscapeExtension
[ScopedEvalContextModifier])
https://github.com/mitsuhiko/jinja2/blob/master/jinja2/ext.py#L434

* preprocessors and things are then just jinja2.ext.Extension s.

* Jinja2 accepts an explicit context
  (where merge(globals, locals, kwargs) just feels wrong because it is,
... [ ] lookup(**kwargs), lngb(**kwargs)) (salt pillar merges))

  ~ collections.abc.MutableMapping:

https://docs.python.org/3/library/collections.abc.html#collections.abc.MutableMapping

* Jinja2 marks strings with MarkupSafe (in order to prevent e.g. multiple
escaping, lack of escaping)

  https://pypi.python.org/pypi/MarkupSafe

f-strings would make it too easy for me to do the wrong thing;
which other language don't prevent (this does occur often [CWE Top 25
2011]),
and I regard this as a current feature of Python.


 Carl


 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Guido van Rossum
On Mon, Aug 10, 2015 at 8:49 PM, Eric V. Smith e...@trueblade.com wrote:

 On 08/10/2015 02:44 PM, Yury Selivanov wrote:
  On 2015-08-10 2:37 PM, Eric V. Smith wrote:
  This is why I think PEP-498 isn't the solution for i18n. I'd really like
  to be able to say, in a debugging context:
 
  print('a:{self.a} b:{self.b} c:{self.c} d:{self.d}')
 
  without having to create locals to hold these 4 values.
 
  Why can't we restrict expressions in f-strings to
  attribute/item getters?
 
  I.e. allow f'{foo.bar.baz}' and f'{self.foo[bar]}' but
  disallow f'{foo.bar(baz=something)}'

 It's possible. But my point is that Barry doesn't even want
 attribute/item getters for an i18n solution, and I'm not willing to
 restrict it that much.


I also don't want to tie this closely to i18n. That is (still) very much a
wold of its own.

What I want with f-strings (by any name) is a way to generalize from
print() calls with multiple arguments. We can write

  print('Todo:', len(self.todo), '; busy:', len(self.busy))

but the same thing is more awkward when you have to pass it as a single
string to a function that just sends one string somewhere. And note that
the above example inserts a space before the ';' which I don't really like.
So it would be nice if instead we could write

  print(f'Todo: {len(self.todo)}; busy: {len(self.busy)}')

which IMO is just as readable as the multi-arg print() call[1], and
generalizes to other functions besides print().

In fact, the latter form has less punctuation noise than the former --
every time you insert an expression in a print() call, you have a
quote+comma before it and a comma+quote after it, compared to a brace
before and one after in the new form. (Note that this is an argument for
using f'{...}' rather than '\{...}' -- for a single interpolation it's the
same amount of typing, but for multiple interpolations, f'{...}{...}' is
actually shorter than '\{...}\{...}', and also the \{ part is ugly.)

Anyway, this generalization from print() is why I want arbitrary
expressions. Wouldn't it be silly if we introduced print() today and said
we don't really like to encourage printing complicated expressions, but
maybe we can introduce them in a future version... :-)

Continuing the print()-generalization theme, if things become too long to
fit on a line we can write

  print('Todo:', len(self.todo),
'; busy:', len(self.busy))

Can we allow the same in f-strings? E.g.

  print(f'Todo: {len(self.todo)
}; busy: {len(self.busy)
}')

or is that too ugly? It could also be solved using implicit concatenation,
e.g.

  print(f'Todo: {len(self.todo)}; '
f'busy: {len(self.busy)}')

[1] Assuming syntax colorizers catch on.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Wes Turner
On Aug 10, 2015 4:52 PM, Nathaniel Smith n...@pobox.com wrote:

 On Aug 10, 2015 11:33 AM, Barry Warsaw ba...@python.org wrote:
 
  On Aug 11, 2015, at 03:26 AM, Steven D'Aprano wrote:
 
  I think I would be happy with f-strings, or perhaps i-strings if we use
  Nick's ideas about internationalisation, and limit what they can
evaluate to
  name lookups, attribute lookups, and indexing, just like format().
 
  I still think you really only need name lookups, especially for an i18n
  context.  Anything else is just overkill, YAGNI, potentially error
prone, or
  perhaps even harmful.
 
  Remember that the translated strings usually come from only moderately
(if at
  all) trusted and verified sources, so it's entirely possible that a
malicious
  translator could sneak in an exploit, especially if you're evaluating
  arbitrary expressions.  If you're only doing name substitutions, then
the
  worst that can happen is an information leak, which is bad, but won't
  compromise the integrity of say a server using the translation.
 
  Even if the source strings avoid the use of expressions, if the feature
is
  available, a translator could still sneak something in.  That pretty
much
  makes it a non-starter for i18n, IMHO.
 
  Besides, any expression you have to calculate can go in a local that
will get
  interpolated.  The same goes for any !r or other formatting modifiers.
In an
  i18n context, you want to stick to the simplest possible substitution
  placeholders.

 IIUC what Nick contemplates in PEP 501 is that when you write something
like
   iI am ${self.age}
 then the python runtime would itself evaluate self.age and pass it on to
the i18n machinery to do the actual substitution; the i18n machinery
wouldn't even contain any calls to eval. The above string could be
translated as
   iTengo ${self.age} años
 but
   iTengo ${self.password} años
 would be an error, because the runtime did not provide a value for
self.password. So while arbitrarily complex expressions are allowed (at
least as far as the language is concerned -- a given project or i18n
toolkit could impose additional policy restrictions), by the time the
interpolation machinery runs they'll effectively have been reduced to local
variables with funny multi-token names.

 This pretty much eliminates all the information leak and exploit
concerns, AFAICT. From your comments about having to be careful about
attribute chasing, it sounds like it might even be more robust than current
flufl.i18n in this regard...?

No, those remain; but minimizing calls to eval is good, too.

I prefer explicit template context for good reason:

* scope / variable binding in list comprehensions,
* it was called 'cmd' two nested scopes ago

Again, convenient but dangerous (Django and Jinja can/do autoescaping) and
making it far too easy to wrongly quote and not escape strings (which often
contain domain-specific) control characters.


 -n


 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Steven D'Aprano
On Mon, Aug 10, 2015 at 09:23:15PM +0200, Guido van Rossum wrote:

[...]
 Anyway, this generalization from print() is why I want arbitrary
 expressions. Wouldn't it be silly if we introduced print() today and said
 we don't really like to encourage printing complicated expressions, but
 maybe we can introduce them in a future version... :-)

That's a straw-man argument. Nobody is arguing against allowing 
arbitrary expressions as arguments to functions.

If you want a fair analogy, how about the reluctance to allow arbitrary 
expressions as decorators?

@[spam, eggs, cheese][switch]
def function():
...


As far as I can see, the non-straw argument is that f-strings be limited 
to the same subset of expressions that format() accepts: name and 
attribute look-ups, and indexing.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Wes Turner
On Mon, Aug 10, 2015 at 1:52 PM, David Mertz me...@gnosis.cx wrote:

 I know. I elided including the nonexistent `nonlocals()` in there. But it
 *should* be `lngb()`.  Or call it scope(). :-)
 On Aug 10, 2015 10:09 AM, Steven D'Aprano st...@pearwood.info wrote:

 On Sun, Aug 09, 2015 at 06:14:18PM -0700, David Mertz wrote:

 [...]
  That said, there *is* one small corner where I believe f-strings add
  something helpful to the language.  There is no really concise way to
 spell:
 
collections.ChainMap(locals(), globals(), __builtins__.__dict__).

 I think that to match the normal name resolution rules, nonlocals()
 needs to slip in there between locals() and globals(). I realise that
 there actually isn't a nonlocals() function (perhaps there should be?).

  If we could spell that as, say `lgb()`, that would let str.format() or
  %-formatting pick up the full what's in scope.  To my mind, that's the
  only good thing about the f-string idea.

 I like the concept, but not the name. Initialisms tend to be hard
 to remember and rarely self-explanatory. How about scope()?


#letsgoblues!

scope(**kwargs), lngb(**kwargs), lookup(**kwargs) could allow for local
attr override.




 --
 Steve
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/mertz%40gnosis.cx


 ___
 Python-Dev mailing list
 Python-Dev@python.org
 https://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe:
 https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] [RELEASED] Python 3.5.0rc1 is now available

2015-08-10 Thread Larry Hastings



On behalf of the Python development community and the Python 3.5 release 
team, I'm relieved to announce the availability of Python 3.5.0rc1, also 
known as Python 3.5.0 Release Candidate 1.


Python 3.5 has now entered feature freeze.  By default new features 
may no longer be added to Python 3.5.


This is a preview release, and its use is not recommended for production 
settings.



You can find Python 3.5.0rc1 here:

https://www.python.org/downloads/release/python-350rc1/

Windows and Mac users: please read the important platform-specific 
Notes on this release section near the end of that page.



Happy hacking,


/arry

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Sorry folks, minor hiccup for Python 3.5.0rc1

2015-08-10 Thread Larry Hastings



I built the source tarballs with a slightly-out-of-date tree.  We 
slipped the release by a day to get two fixes in, but the tree I built 
from didn't have those two fixes.


I yanked the tarballs off the release page as soon as I suspected 
something.  I'm rebuilding the tarballs and the docs now.  If you 
grabbed the tarball as soon as it appeared, it's slightly out of date, 
please re-grab.


Sorry for the palaver,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Instructions on the new push request workflow for 3.5.0rc1+ through 3.5.0 final

2015-08-10 Thread Brett Cannon
A quick hg tip for making sure you check out the right branch: end the URL
on #3.5 and it will start the repo out with the 3.5 as the active branch.

On Mon, Aug 10, 2015, 01:28 Larry Hastings la...@hastings.org wrote:


 As of Python 3.5.0rc1, the canonical repository for Python 3.5.0 is
 *no longer* on hg.python.org.  Instead, it's hosted on Bitbucket on
 my personal account, here:

  https://bitbucket.org/larry/cpython350

 Since 3.5.0rc1 isn't out yet I'm keeping the repository private for now.
 Once 3.5.0 rc1 is released (hopefully Monday) I'll flip the switch and make
 the repository public.  (I'll email python-dev and python-committers when
 that happens.)


 Putting it succinctly, here's a table of versions and where you'd check in
 for your change to go there:

  3.5.0 : https://bitbucket.org/larry/cpython350 (branch 3.5)
  3.5.1 : hg.python.org/cpython (branch 3.5)
  3.6.0 : hg.python.org/cpython (branch default)

 You'll notice nobody but myself has checkin permissions for my 3.5.0 repo
 on
 Bitbucket.  That's on purpose.  The only way you can get changes in to
 3.5.0
 now is by sending me a Bitbucket pull request.  This is a workflow
 experiment, to see if we as a community like this sort of new-fangled
 gizmo.

 For now, we're only using Bitbucket for the actual final checking-in stage.
 Requests for fixes to be accepted into 3.5.0 and code review will all still
 happen on the Python issue tracker.

 Also, I'm officially now asking you folks to do the forward-merge into
 3.5.1
 and 3.6.0 yourselves.


 Here's how to get a fix checked in for 3.5.0, starting with 3.5.0rc1+ and
 continuing through until 3.5.0 final.

 Pre-requisites:
* You must have a Bitbucket account.
* You must have commit rights to the CPython repository.

   1. Create an issue on the Python issue tracker for the problem.

   2. Submit a patch that fixes the problem.

   3. Add me to the issue and get me to agree that it needs fixing in 3.5.0.
  (You can attempt this step before 2 if you prefer.)

   4. Fork my repository into your Bitbucket account using their web GUI.
  To do that, go to Bitbucket, log in, then go to my 3.5.0 repo:

  https://bitbucket.org/larry/cpython350

  and press the Fork link in the left column.  Bitbucket has a
 tutorial
  on how to do this, here:


 https://confluence.atlassian.com/display/BITBUCKET/Fork+a+teammate%27s+repository

  Note: DO NOT start with a conventional CPython trunk cloned from
  hg.python.org.  The 3.5 branch in my repo and the 3.5 branch in
 normal
  CPython trunk have intentionally diverged and *need* to stay
 out-of-sync.

   5. Make a local clone of your fork on your computer.

  Bitbucket has a tutorial on how to do that, here:


 https://confluence.atlassian.com/display/BITBUCKET/Copy+your+Mercurial+repository+and+add+source+files

  Reminder: switch to the 3.5 branch!

   6. Apply your change to the 3.5 branch and check in.

  Reminder: check in to the 3.5 branch!

   7. Make sure you checked in your change to the 3.5 branch.

  Reminder: Seriously.  I keep messing this up.  I say, the more
 reminders,
  the better.

   8. Push your change back to *your* fork on *your* Bitbucket account.

  Just normal hg push should work here.

  In case it helps, I recommend using the https protocol for this
 step, as
  it sidesteps ssh authentication and prompts you for your Bitbucket
 username
  and password.

   9. Create a pull request using Bitbucket's web GUI.

  Bitbucket has a tutorial on how to create a pull request, here:

 https://confluence.atlassian.com/display/BITBUCKET/Create+a+pull+request

  On the Create pull request web GUI, make sure that you specify
  branch 3.5 for *both* your repo *and* my repo.  Also, make sure
  you *don't* check the Close 3.5 after the pull request is merged
  check box.

  (If you use the Compare page, you also need to select 3.5 in both
  drop-down lists--one for my repo, and one for yours.)

 10. Paste a link to the pull request into the issue tracker issue for this
  change request.

 11. Wait for confirmation that I've accepted your pull request into the
  3.5.0 repo.

 12. Pull your accepted change from your local Bitbucket fork repo into
  a normal hg.cpython.org CPython repo, merge into 3.5, then merge
  into 3.6, then push.


 For the record, here's what *my* workflow looks like when I accept your
 pull request:

 1. Click on the URL you pasted into the pull request.

 2. Visually check that the diff matches the approved diff in the issue
 on the issue tracker.

 3. Click on the Merge button.


 Frequently Asked Questions
 ==

 Q: What if someone sends me a pull request for a change that doesn't
 merge cleanly?
 A: Then I'll decline it, and ask you on the issue tracker to rebase
 and resubmit.

 Q: What if someone sends me a pull request but they don't have commit
  

Re: [Python-Dev] Instructions on the new push request workflow for 3.5.0rc1+ through 3.5.0 final

2015-08-10 Thread Larry Hastings


On 08/10/2015 01:27 AM, Larry Hastings wrote:


As of Python 3.5.0rc1, the canonical repository for Python 3.5.0 is
*no longer* on hg.python.org.  Instead, it's hosted on Bitbucket on
my personal account, here:

https://bitbucket.org/larry/cpython350

Since 3.5.0rc1 isn't out yet I'm keeping the repository private for now.
Once 3.5.0 rc1 is released (hopefully Monday) I'll flip the switch and 
make

the repository public.  (I'll email python-dev and python-committers when
that happens.)


Python 3.5.0rc1 just went live.  So, as promised, I've flipped the 
switch--my cpython350 repository is now public.


En garde,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Sorry folks, minor hiccup for Python 3.5.0rc1

2015-08-10 Thread Larry Hastings

On 08/10/2015 05:55 PM, Larry Hastings wrote:
I yanked the tarballs off the release page as soon as I suspected 
something.  I'm rebuilding the tarballs and the docs now.  If you 
grabbed the tarball as soon as it appeared, it's slightly out of date, 
please re-grab.


p.s. I should have mentioned--the Mac and Windows builds should be 
fine.  They, unlike me, updated their tree ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Python 3.5.0rc1 is delayed by a day

2015-08-10 Thread Larry Hastings



We retagged Python 3.5.0rc1 today to fix two bugs that popped up late in 
the process.  Release candidates are supposed to be software you 
genuinely would release, and I couldn't release Python with both those 
bugs.  This delay rippled through the whole process, so it just isn't 
going out tonight (late Sunday / early Monday in my timezone).  I have 
every expectation it'll go out Monday.


In case you're interested, the bugs are (were!):

   http://bugs.python.org/issue24745
   http://bugs.python.org/issue24835

My thanks to the folks who stepped up and fixed the bugs on short 
notice, and my apologies to the community for the delay.  We're just 
trying to make the best Python we can, for you!


See you tomorrow,


//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Instructions on the new push request workflow for 3.5.0rc1+ through 3.5.0 final

2015-08-10 Thread Larry Hastings


As of Python 3.5.0rc1, the canonical repository for Python 3.5.0 is
*no longer* on hg.python.org.  Instead, it's hosted on Bitbucket on
my personal account, here:

https://bitbucket.org/larry/cpython350

Since 3.5.0rc1 isn't out yet I'm keeping the repository private for now.
Once 3.5.0 rc1 is released (hopefully Monday) I'll flip the switch and make
the repository public.  (I'll email python-dev and python-committers when
that happens.)


Putting it succinctly, here's a table of versions and where you'd check in
for your change to go there:

3.5.0 : https://bitbucket.org/larry/cpython350 (branch 3.5)
3.5.1 : hg.python.org/cpython (branch 3.5)
3.6.0 : hg.python.org/cpython (branch default)

You'll notice nobody but myself has checkin permissions for my 3.5.0 repo on
Bitbucket.  That's on purpose.  The only way you can get changes in to 3.5.0
now is by sending me a Bitbucket pull request.  This is a workflow
experiment, to see if we as a community like this sort of new-fangled gizmo.

For now, we're only using Bitbucket for the actual final checking-in stage.
Requests for fixes to be accepted into 3.5.0 and code review will all still
happen on the Python issue tracker.

Also, I'm officially now asking you folks to do the forward-merge into 3.5.1
and 3.6.0 yourselves.


Here's how to get a fix checked in for 3.5.0, starting with 3.5.0rc1+ and
continuing through until 3.5.0 final.

Pre-requisites:
  * You must have a Bitbucket account.
  * You must have commit rights to the CPython repository.

 1. Create an issue on the Python issue tracker for the problem.

 2. Submit a patch that fixes the problem.

 3. Add me to the issue and get me to agree that it needs fixing in 3.5.0.
(You can attempt this step before 2 if you prefer.)

 4. Fork my repository into your Bitbucket account using their web GUI.
To do that, go to Bitbucket, log in, then go to my 3.5.0 repo:

https://bitbucket.org/larry/cpython350

and press the Fork link in the left column.  Bitbucket has a tutorial
on how to do this, here:

https://confluence.atlassian.com/display/BITBUCKET/Fork+a+teammate%27s+repository

Note: DO NOT start with a conventional CPython trunk cloned from
hg.python.org.  The 3.5 branch in my repo and the 3.5 branch in normal
CPython trunk have intentionally diverged and *need* to stay 
out-of-sync.


 5. Make a local clone of your fork on your computer.

Bitbucket has a tutorial on how to do that, here:

https://confluence.atlassian.com/display/BITBUCKET/Copy+your+Mercurial+repository+and+add+source+files

Reminder: switch to the 3.5 branch!

 6. Apply your change to the 3.5 branch and check in.

Reminder: check in to the 3.5 branch!

 7. Make sure you checked in your change to the 3.5 branch.

Reminder: Seriously.  I keep messing this up.  I say, the more 
reminders,

the better.

 8. Push your change back to *your* fork on *your* Bitbucket account.

Just normal hg push should work here.

In case it helps, I recommend using the https protocol for this 
step, as
it sidesteps ssh authentication and prompts you for your Bitbucket 
username

and password.

 9. Create a pull request using Bitbucket's web GUI.

Bitbucket has a tutorial on how to create a pull request, here:

https://confluence.atlassian.com/display/BITBUCKET/Create+a+pull+request

On the Create pull request web GUI, make sure that you specify
branch 3.5 for *both* your repo *and* my repo.  Also, make sure
you *don't* check the Close 3.5 after the pull request is merged
check box.

(If you use the Compare page, you also need to select 3.5 in both
drop-down lists--one for my repo, and one for yours.)

10. Paste a link to the pull request into the issue tracker issue for this
change request.

11. Wait for confirmation that I've accepted your pull request into the
3.5.0 repo.

12. Pull your accepted change from your local Bitbucket fork repo into
a normal hg.cpython.org CPython repo, merge into 3.5, then merge
into 3.6, then push.


For the record, here's what *my* workflow looks like when I accept your
pull request:

1. Click on the URL you pasted into the pull request.

2. Visually check that the diff matches the approved diff in the issue
   on the issue tracker.

3. Click on the Merge button.


Frequently Asked Questions
==

Q: What if someone sends me a pull request for a change that doesn't
   merge cleanly?
A: Then I'll decline it, and ask you on the issue tracker to rebase
   and resubmit.

Q: What if someone sends me a pull request but they don't have commit
   rights to CPython?
A: I'll ignore it.  I'll only pay attention to pull requests pasted into
   the issue tracker by someone with commit rights.

Q: Whose name goes on the commit?
A: It gets the name the checkin was made with.  Don't worry, your name
   will stay on your commit.

Q: This seems like a lot more work than the old way.
A: For you guys, 

Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Alexander Walters

On 8/10/2015 01:29, Sven R. Kunze wrote:
The best solution would be without prefix and '{var}' only syntax. 
Not sure if that is possible at all; I cannot remember using '{...}' 
anywhere else than for formatting.


My JSON string literal 'test fixtures' weep at that idea.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] PEP-498: Literal String Formatting

2015-08-10 Thread Barry Warsaw
On Aug 09, 2015, at 06:14 PM, David Mertz wrote:

That said, there *is* one small corner where I believe f-strings add
something helpful to the language.  There is no really concise way to spell:

  collections.ChainMap(locals(), globals(), __builtins__.__dict__).

If we could spell that as, say `lgb()`, that would let str.format() or
%-formatting pick up the full what's in scope.  To my mind, that's the
only good thing about the f-string idea.

That would certainly be useful to avoid sys._getframe() calls in my library,
although I'd probably want the third argument to be optional (I wouldn't use
it).

If '{foo}' or '${foo}' syntax is adopted (with no allowance for '$foo'), it's
very unlikely I'd use that over string.Template for internationalization, but
the above would still be useful.

Cheers,
-Barry
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] PEP 498 f-string: is it a preprocessor?

2015-08-10 Thread Victor Stinner
Hi,

I read the PEP but I don't understand how it is implemented. For me, it
should be a simple preprocessor:

- f'x={x}' is replaced with 'x={0}'.format(x) by the compiler
- f'x={1+1}' is replaced with 'x={0}'.format(1+1)
- f'x={foo()!r}' is replaced with 'x={0!r}'.format(foo())
- ...

That's all. No new language, no new function or method.

It's unclear to me if arbitrary expressions should be allowed or not. If
not, we may pass parameters by keywords instead:

f'x={x}' is replaced with 'x={x}'.format(x=x) by the compiler

'...'.format(...) is highly optimized. In the current PEP, I see that each
parameter is rendered in its own independent buffer (.__format__() method
called multiple times) and then concateneted by ''.join(...). It's less
efficient that using a single call to .format().

Victor

PS: it looks like the gmail application changed the font size in the middle
of my email. I don't know how to use plain text, sorry about that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com