[issue25885] ast Str type does not annotate the string type when it parses a python document

[STINNER Victor]

STINNER Victor added the comment:

> Would it be prudent to add a Parse flag to the AST module that could provide 
> one of two types of AST's an optimized AST or a complete AST

Adding syntax ("formatting", call it as you want) info to AST requires
to add new attributes to existing AST nodes and probably add new AST
nodes. It will make the code more complex, not only the code to
produce AST, but also code using AST (static code analyzer, my AST
optimizer project, etc.).

I don't think that it's worth it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[STINNER Victor]

STINNER Victor added the comment:

If you consider that we are wrong, please follow the advice of
starting a discussion on python-ideas. This is how Python is
developed, we have a workflow. Proposing ideas on the bug tracker
works in some cases, but AST is really a *core* feature of Python. You
need deep discussions to change the core. To be cristal clear: *if*
anyone is going to change AST, IMHO a PEP is needed. Writing a PEP
requires a specific workflow starting at python-ideas.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Eric V. Smith]

Eric V. Smith added the comment:

I agree that the proposed change would require a PEP, and this should be 
discussed on python-ideas.

I also think there's very little chance such a change would be accepted, but 
that doesn't mean it's impossible.

I think using a external library is your best bet here. If you want to pursue 
this on python-ideas, you should discuss why being in the stdlib's ast module 
would be an improvement over using an external library.

--
nosy: +eric.smith
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Myron Walker]

New submission from Myron Walker:

The 'ast' module does not indicate the type of string, ''' or '"' or '"""', 
that it has encountered when it parses a python document.

This prevents accurate reproduction of the original parsed document by a writer 
walking over an instance of a abstract syntax tree.

--
components: Extension Modules
messages: 256529
nosy: Myron Walker
priority: normal
severity: normal
status: open
title: ast Str type does not annotate the string type when it parses a python 
document
type: behavior
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[SilentGhost]

Changes by SilentGhost :


--
components: +Library (Lib)
nosy: +benjamin.peterson, brett.cannon, georg.brandl, ncoghlan
stage:  -> test needed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Brett Cannon]

Brett Cannon added the comment:

Two things. One, this won't change in Python 2.7 due to compatibility, so this 
only applies to Python 3.6. Two, the AST has not been designed to support 
round-trip syntax transpiling, e.g. there is no way to get back comments in the 
source code.

Because the AST is not meant to round-trip on source code I'm going to close 
this as "not a bug" since I don't see any benefit in supporting this even if we 
gain comment nodes in the AST.

--
resolution:  -> not a bug
status: open -> closed
versions: +Python 3.6 -Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[R. David Murray]

R. David Murray added the comment:

Unlike the tokenizer, I don't think the AST module makes any guarantee about 
being able to reproduce the original source.  This might be a reasonable 
enhancement request, though.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[R. David Murray]

R. David Murray added the comment:

Please respect the decisions of the python core developers as to issue status 
(you can argue with us until we ask you to stop, but let us make the status 
change if you convince us :)

We aren't rejecting your ideas, just this bug report (as not being a bug).  To 
pursue your ideas you will want to subscribe to the python-ideas mailing list 
and begin a discussion there.  This is far too big a topic for the bug tracker.

--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Myron Walker]

Myron Walker added the comment:

I am re-opening this as I believe this is an important issue for work I would 
like to eventually push into the python core which is python code that recode 
themselves as declarations or as instance representation.

"I don't see any benefit in supporting this even if we gain comment nodes in 
the AST."

There would be a huge benefit to having accurate original syntax information of 
the document and being able to re-write the documents as it would enable self 
modifying code or software written code paradigms.

TestCases that can update themselves in various modes, full auto, 
interactive, etc.

TestData Pattern Generation object or Data Pattern Generation object that 
can be easily or efficiently modified and then asked to write themselves back 
out to files. 

The fact that there are other projects being worked on to provide round trip 
document to syntax tree and syntax tree to document transformations also 
indicates that the lack of accurate syntax storage in 'ast' is a problem that
needs to be addressed.

I would prefer to work with the core python modules in order to provide dynamic 
code modification functionality rather than relying on a third party module as 
eventually I would like to push this into core python.

If a python object has script behind it, I would like to eventually be able to 
tell a the object to write itself to a file as a object declaration or as a 
object instance.

As Declaration:

class SomeObject(SomeBase):
   def some method(self):
   print "blah blah"

As Instance:

   SomeObject()

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Myron Walker]

Myron Walker added the comment:

The purpose of a syntax tree is to represent the syntax and not a final 
processed result of processing of syntax.  The current information stored for 
strings is losing syntax information which seems to defeat the purpose of 
offering the information in a syntax tree.  I filed a separate bug because it 
is also combining strings and losing operators for string literals.

   "Hello" + " World"

>From the looks of the code, the above would result in one string type with 
>"Hello World" and syntax information associated with the operator would be 
>lost.

And as indicated, string type information is being lost as well.  The user of 
the AST then has no way of getting the lost syntax information back once it is 
lost.

--
components: +Interpreter Core -Extension Modules, Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[R. David Murray]

R. David Murray added the comment:

Oh, that was weird.  I got a weird error message from roundup.  Must have been 
because I was posting at the same time as Brett.  I'll defer to his decision on 
this.

--
stage: test needed -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[STINNER Victor]

STINNER Victor added the comment:

"Would adding proper storage of syntax information in the AST cause performance 
issues?"

Exactly. And storing syntax information is useless for 90% of usages of AST. So 
Python is optimized for the most common cases.

Again, use a different project (like RedBaron) if you need *all* information in 
the AST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25885] ast Str type does not annotate the string type when it parses a python document

[Myron Walker]

Myron Walker added the comment:

Would it be prudent to add a Parse flag to the AST module that could provide 
one of two types of AST's an optimized AST or a complete AST

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com