[Python-Dev] f-strings in the grammar

[Pablo Galindo Salgado]

Hi,

I have started a project to move the parsing off-strings to the parser and
the grammar. Appart
from some maintenance improvements (we can drop a considerable amount of
hand-written code),
there are some interesting things we **could** (emphasis on could) get out
of this and I wanted
to discuss what people think about them.

* The parser will likely have "\n" characters and backslashes in f-strings
expressions, which currently is impossible:

>>> f"blah blah {'\n'} blah"
  File "", line 1
f"blah blah {'\n'} blah"
^
SyntaxError: f-string expression part cannot include a backslash

* The parser will allow nesting quote characters. This means that we
**could** allow reusing the same quote type in nested expressions
like this:

f"some text { my_dict["string1"] } more text"

* The parser will naturally allow more control over error messages and AST
positions.

* The **grammar** of the f-string will be fully specified without
ambiguity. Currently, the "grammar" that we have in the docs
(
https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals)
is not really formal grammar because
not only is mixing lexing details with grammar details (the definition of "
literal_char") but also is not compatible with the current python
lexing schema (for instance, it recognizes "{{" as its own token, which the
language doesn't allow because something like "{{a:b}:c}"
is tokenized as "{", "{", "a" ... not as "{{", "a". Adding a formal grammar
could help syntax highlighters, IDEs, parsers and other tools
to make sure they properly recognize everything that there is.

There may be some other advantages that we have not explored still.

The work is at a point where the main idea works (all the grammar is
already there and working), but we need to make sure that all existing
errors and specifics are properly ported to the new code, which is a
considerable amount of work still so I wanted to make sure we are on the
same page before we decide to invest more time on this (Batuhan is helping
me with this and Lyssandros will likely join us). We are doing
this work in this branch:
https://github.com/we-like-parsers/cpython/blob/fstring-grammar

Tell me what you think.

P.S. If you are interested to help with this project, please reach out to
me. If we decide to go ahead we can use your help! :)

Regards from cloudy London,
Pablo Galindo Salgado
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/54N3MOYVBDSJQZTU6MTCPLUPIFSDN5IS/
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-Dev] f-strings

[Stephen J. Turnbull]

Eric Fahlgren writes:
 > On Tue, Dec 19, 2017 at 8:47 AM, Stephen J. Turnbull <
 > turnbull.stephen...@u.tsukuba.ac.jp> wrote:
 > 
 > > If I were Bach, I'd compose a more-itertools-like module to be named
 > > Variations_on_the_F_String. :-)
 > >
 > 
 > Would that be P.D.Q. Bach to whom you are referring?

It's a curried function, supply your own arguments.  "No, I won't!"

Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Eric Fahlgren]

On Tue, Dec 19, 2017 at 8:47 AM, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> If I were Bach, I'd compose a more-itertools-like module to be named
> Variations_on_the_F_String. :-)
>

​Would that be P.D.Q. Bach to whom you are referring?​
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Chris Barker]

On Tue, Dec 19, 2017 at 8:47 AM, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> I don't see any reason not to document tips and tricks with f-strings,
> and this is a nice and useful example.  But it looks like TOOWTDI to
> me.  The syntax is documented (6.1.3.1 in the Library Reference),
> along with a specific relevant example ("Aligning the text and
> specifying a width" in 6.1.3.2).
>
> So -1 on putting the recipe in the reference docs.  I really don't
> think this kind of information belongs in a PEP for sure, and probably
> not even in the Library Reference.


The docs (and I think PEP) have been updated to clearly state that
f-strings use the same formatting specifiers as .format(), and have links
to those docs.

So I think we're good.


> The Tutorial might be a good place
> for it, though.
>

yup.

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Stephen J. Turnbull]

Mariatta Wijaya writes:
 > I agree it's useful info :)
 > 
 > I went ahead and made a PR [1].
 > In my PR, I simply linked to the Format Specification Mini Language[2] from
 > f-strings documentation[3].
 > 
 > Not sure about updating PEP 498 at this point..

I don't see any reason not to document tips and tricks with f-strings,
and this is a nice and useful example.  But it looks like TOOWTDI to
me.  The syntax is documented (6.1.3.1 in the Library Reference),
along with a specific relevant example ("Aligning the text and
specifying a width" in 6.1.3.2).

So -1 on putting the recipe in the reference docs.  I really don't
think this kind of information belongs in a PEP for sure, and probably
not even in the Library Reference.  The Tutorial might be a good place
for it, though.

If I were Bach, I'd compose a more-itertools-like module to be named
Variations_on_the_F_String. :-)

Steve

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Wagner Herculano]

Hi,
Thank you all for your time.
In fact I'm really a newbie and I need to study so hard (including English) to 
become someone like you all. I don't know if is possible, but I need to try.
Once again, thank you all for your time and I'm sorry if I'm bother both of you.

Kind regards,
   Wagner Herculano.


From: Chris Barker - NOAA Federal <chris.bar...@noaa.gov>
Sent: Friday, December 15, 2017 2:49:08 PM
To: Mariatta Wijaya
Cc: Wagner Herculano; Python Dev
Subject: Re: [Python-Dev] f-strings




That's covered under "format specifiers" I think.
The PEP mentions this: 
https://www.python.org/dev/peps/pep-0498/#format-specifiers

I can see how a newbie might not realize that that means that f-strings use the 
same formatting language as the .format() method, and or where to find 
documentation for it.

So somewhere in the docs making that really clear, with a link to the 
formatting spec documentation would be good.

Not sure where though — a PEP is not designed to be user documentation.

-CHB


That specific example is not mentioned in the docs, but there other examples of 
using format specifiers with f-strings.
https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals


On Dec 15, 2017 7:39 AM, "Wagner Herculano" 
<waghercul...@hotmail.com<mailto:waghercul...@hotmail.com>> wrote:
Good evening,
I'm Wagner Herculano from Brazil.
I was trying to do a table exercise with number 5 and tried formatting spaces 
and did not find it in PEP 498 documentation.
Finally I found a way, if possible, include this example in the documentation 
please.

Below is my script with the desired formatting about table of 5.

n = 5
for i in range(1,11):
print(f'{n} x {i:>2} = {n*i:>2}')

Result
5 x  1 =  5
5 x  2 = 10
5 x  3 = 15
5 x  4 = 20
5 x  5 = 25
5 x  6 = 30
5 x  7 = 35
5 x  8 = 40
5 x  9 = 45
5 x 10 = 50
---
Sorry my English, I needed to use Google Translate

Best Regards,
Wagner Herculano

___
Python-Dev mailing list
Python-Dev@python.org<mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/mariatta.wijaya%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org<mailto:Python-Dev@python.org>
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Chris Barker]

On Fri, Dec 15, 2017 at 9:39 AM, Mariatta Wijaya 
wrote:

> I agree it's useful info :)
>
> I went ahead and made a PR [1].
>

Thanks! I added a couple comments to that PR.


> Not sure about updating PEP 498 at this point..
>

A little clarification text would be nice. I made a PR for that:

https://github.com/python/peps/pull/514


-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Mariatta Wijaya]

I agree it's useful info :)

I went ahead and made a PR [1].
In my PR, I simply linked to the Format Specification Mini Language[2] from
f-strings documentation[3].

Not sure about updating PEP 498 at this point..

[1] https://github.com/python/cpython/pull/4888

[2] https://docs.python.org/3.6/library/string.html#format-s
pecification-mini-language

[3]  https://docs.python.org/3/reference/lexical_analysis.html#f-strings
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Chris Barker - NOAA Federal]

That's covered under "format specifiers" I think.
The PEP mentions this:
https://www.python.org/dev/peps/pep-0498/#format-specifiers


I can see how a newbie might not realize that that means that f-strings use
the same formatting language as the .format() method, and or where to find
documentation for it.

So somewhere in the docs making that really clear, with a link to the
formatting spec documentation would be good.

Not sure where though — a PEP is not designed to be user documentation.

-CHB


That specific example is not mentioned in the docs, but there other
examples of using format specifiers with f-strings.
https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals


On Dec 15, 2017 7:39 AM, "Wagner Herculano" 
wrote:

> Good evening,
> I'm Wagner Herculano from Brazil.
> I was trying to do a table exercise with number 5 and tried formatting
> spaces and did not find it in PEP 498 documentation.
> Finally I found a way, if possible, include this example in the
> documentation please.
>
> Below is my script with the desired formatting about table of 5.
>
> *n = 5*
>
>
>
>
>
>
>
>
>
>
>
>
>
> *for i in range(1,11): print(f'{n} x {i:>2} = {n*i:>2}') Result5
> x  1 =  5 5 x  2 = 10 5 x  3 = 15 5 x  4 = 20 5 x  5 = 25 5 x  6 = 30 5 x
> 7 = 35 5 x  8 = 40 5 x  9 = 45 5 x 10 = 50*
> *---*
> *Sorry my English, I needed to use Google Translate*
>
> Best Regards,
> Wagner Herculano
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mariatta.
> wijaya%40gmail.com
>
> ___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] f-strings

[Mariatta Wijaya]

That's covered under "format specifiers" I think.
The PEP mentions this:
https://www.python.org/dev/peps/pep-0498/#format-specifiers

That specific example is not mentioned in the docs, but there other
examples of using format specifiers with f-strings.
https://docs.python.org/3/reference/lexical_analysis.html#formatted-string-literals


On Dec 15, 2017 7:39 AM, "Wagner Herculano" 
wrote:

> Good evening,
> I'm Wagner Herculano from Brazil.
> I was trying to do a table exercise with number 5 and tried formatting
> spaces and did not find it in PEP 498 documentation.
> Finally I found a way, if possible, include this example in the
> documentation please.
>
> Below is my script with the desired formatting about table of 5.
>
> *n = 5*
>
>
>
>
>
>
>
>
>
>
>
>
>
> *for i in range(1,11): print(f'{n} x {i:>2} = {n*i:>2}') Result5
> x  1 =  5 5 x  2 = 10 5 x  3 = 15 5 x  4 = 20 5 x  5 = 25 5 x  6 = 30 5 x
> 7 = 35 5 x  8 = 40 5 x  9 = 45 5 x 10 = 50*
> *---*
> *Sorry my English, I needed to use Google Translate*
>
> Best Regards,
> Wagner Herculano
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mariatta.
> wijaya%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] f-strings

[Wagner Herculano]

Good evening,
I'm Wagner Herculano from Brazil.
I was trying to do a table exercise with number 5 and tried formatting spaces 
and did not find it in PEP 498 documentation.
Finally I found a way, if possible, include this example in the documentation 
please.

Below is my script with the desired formatting about table of 5.

n = 5
for i in range(1,11):
print(f'{n} x {i:>2} = {n*i:>2}')

Result
5 x  1 =  5
5 x  2 = 10
5 x  3 = 15
5 x  4 = 20
5 x  5 = 25
5 x  6 = 30
5 x  7 = 35
5 x  8 = 40
5 x  9 = 45
5 x 10 = 50
---
Sorry my English, I needed to use Google Translate

Best Regards,
Wagner Herculano
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com