Re: Parsing files in python
Chris Angelico wrote:
> On Mon, Dec 24, 2012 at 9:32 PM, Kene Meniru
> wrote:
>> You are saying I can create a python module that can parse this file
>> format without using a system like python-ply? I know how to parse
>> strings using python but considering that text files that describe a
>> whole building may be quite large I thought perhaps the re module may not
>> be adequate.
>
> Effectively, what you do is leverage the Python parser. Your script
> would look like this:
>
> possible user file content for parsing
> # Boiler-plate to make this work
> from pypovray import *
>
> # in the following the python interface program reads
> # the contents of the file "other.file" as if its content
> # were located at this point.
> import other.file
>
> #In the following the python interface makes "snap_size" a
> # global parameter
> snap_size = 10
>
>
> # In the following "buildingLevel" is a class (or function) that is
> # called and passed the parameters in parenthesis.
> buildingLevel("FirstLevel", 3000)
>
> # In the following "snapOffset" is a class that is
> # called and passed the parameters in parenthesis.
> snapOffset("Closet-S1_r1", "Closet-S2_r3", (0,0,0))
> end of user file content
>
> Note the extreme similarity to your original example. Everything
> between the two snip-lines is perfectly legal Python code. (The
> semantics of a Python import aren't quite the same as a C preprocessor
> #include, so that might need a little tweaking, depending on what you
> wanted to achieve there. Possibly "from other.file import *" would do
> it.) Instead of writing a file parser, with all the complexities that
> that entails, all you need to write is a set of functions/classes that
> can be invoked.
>
> The only part that doesn't work cleanly is the vector, since its
> syntax doesn't work in Python. You'll need to use round brackets
> instead of angle ones, as in the above example, and on output to
> Python, translate them. But that's fairly straight-forward, and by
> this method, you get *everything else* done for you - parsing, nesting
> of function calls, the entire Python standard library... the works.
>
> ChrisA
Thanks. This makes sense and it is something I can start right away porting
my code. Sincerely glad I voiced my thoughts. The import directive will have
to be tackled later but that is not for at least a year or so :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
On 12/23/2012 11:05 PM, Chris Angelico wrote: But other than that, yes, Python's a good choice for this. (I find it amusing how I said "yeah, good idea to make a DSL, I wonder if you can capitalize on Python" and you said "don't make a DSL, maybe you can capitalize on Python" - opposite opening argument, same conclusion and recommendation.) I am aware that every substantial module, let alone package, defines a domain-specific extension or vocabulary. str.format and struct even have their own mini-language (which people tend to forget if not used regularly). What I meant was to not invent a domain-specific base language and syntax that is a complete replacement for an existing one. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
On Mon, Dec 24, 2012 at 9:32 PM, Kene Meniru wrote:
> You are saying I can create a python module that can parse this file format
> without using a system like python-ply? I know how to parse strings using
> python but considering that text files that describe a whole building may be
> quite large I thought perhaps the re module may not be adequate.
Effectively, what you do is leverage the Python parser. Your script
would look like this:
possible user file content for parsing
# Boiler-plate to make this work
from pypovray import *
# in the following the python interface program reads
# the contents of the file "other.file" as if its content
# were located at this point.
import other.file
#In the following the python interface makes "snap_size" a
# global parameter
snap_size = 10
# In the following "buildingLevel" is a class (or function) that is
# called and passed the parameters in parenthesis.
buildingLevel("FirstLevel", 3000)
# In the following "snapOffset" is a class that is
# called and passed the parameters in parenthesis.
snapOffset("Closet-S1_r1", "Closet-S2_r3", (0,0,0))
end of user file content
Note the extreme similarity to your original example. Everything
between the two snip-lines is perfectly legal Python code. (The
semantics of a Python import aren't quite the same as a C preprocessor
#include, so that might need a little tweaking, depending on what you
wanted to achieve there. Possibly "from other.file import *" would do
it.) Instead of writing a file parser, with all the complexities that
that entails, all you need to write is a set of functions/classes that
can be invoked.
The only part that doesn't work cleanly is the vector, since its
syntax doesn't work in Python. You'll need to use round brackets
instead of angle ones, as in the above example, and on output to
Python, translate them. But that's fairly straight-forward, and by
this method, you get *everything else* done for you - parsing, nesting
of function calls, the entire Python standard library... the works.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
Chris Angelico wrote: > I'm hoping you meant for that to be public; if not, my apologies for > forwarding a private message. > > On Mon, Dec 24, 2012 at 8:45 PM, Kene Meniru > wrote: >> Chris Angelico wrote: >>> from povray_macros import * >>> >> >> Am afraid you misunderstood my post. The file format I described is not >> an attempt to re-create or modify a python environment. I do not wish to >> be able to "import" python macros but other text files similar to the one >> I described. > > Yep. There are two possibilities: Either you create a program that > reads in a file of format you invent, or you make a set of Python > functions and classes that mean that the format is actually a Python > script. Instead of writing a file parser, you use Python's, and the > program you write is actually a module rather than a top-level > application. > Actually, I think I mean what you are saying. Let me repeat what I understand maybe I am understanding it wrong. You are saying I can create a python module that can parse this file format without using a system like python-ply? I know how to parse strings using python but considering that text files that describe a whole building may be quite large I thought perhaps the re module may not be adequate. > Producing output on stdout is one of the easiest and most standard > ways to export content. > > ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
I'm hoping you meant for that to be public; if not, my apologies for forwarding a private message. On Mon, Dec 24, 2012 at 8:45 PM, Kene Meniru wrote: > Chris Angelico wrote: >> from povray_macros import * >> > > Am afraid you misunderstood my post. The file format I described is not an > attempt to re-create or modify a python environment. I do not wish to be > able to "import" python macros but other text files similar to the one I > described. Yep. There are two possibilities: Either you create a program that reads in a file of format you invent, or you make a set of Python functions and classes that mean that the format is actually a Python script. Instead of writing a file parser, you use Python's, and the program you write is actually a module rather than a top-level application. Producing output on stdout is one of the easiest and most standard ways to export content. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
On Mon, Dec 24, 2012 at 12:46 PM, Terry Reedy wrote:
>> snapOffset("Closet-S1_r1", "Closet-S2_r3", <0,0,0>)
>
>
> Already legal Python
>
Not quite. This is the one part that *doesn't* work directly. In
POV-Ray, a vector eg is used to represent points,
transformations, and sometimes colors. The closest Python equivalent
is the tuple, but that requires that the brackets be changed:
snapOffset("Closet-S1_r1", "Closet-S2_r3", (0,0,0))
It would also require some smart footwork at the export end,
recognizing that a tuple needs to be output with angle brackets.
But other than that, yes, Python's a good choice for this. (I find it
amusing how I said "yeah, good idea to make a DSL, I wonder if you can
capitalize on Python" and you said "don't make a DSL, maybe you can
capitalize on Python" - opposite opening argument, same conclusion and
recommendation.)
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
On 12/23/2012 12:19 PM, Kene Meniru wrote:
Hello: I am writing a program that is made up of a collection of
POV-Ray macros. POV-Ray is available at povray.org. It is a
ray-tracing program that reads a scene description language (SDL) to
create photo-realistic images. At this time my program (for modeling
building information) is so huge that I am finding it difficult
managing the macros and I am not even near completion.
I am hoping to move this program to python and am wondering the best
way to approach this.
I would like to model my program after LaTeX. Basically the user
writes a text file using certain key words and numbers and my python
program reads this file, calls the classes that will then work
together to calculate the information that is needed to create an
accurate model. The result of this calculation will be an output to
another text file in the appropriate format such as POV-Ray SDL,
OpenSCAD script, etc. This file output can then be rendered by the
corresponding program to produce the actual 3D model. The macros I
have now currently does this but like I said it is getting tedious
and most importantly the fun factor is losing its strength for me.
I have been advised to check out python-ply and I have come across
others. I have not really tried any yet and before I dive into any
one of them I was wondering what else I should know. The following is
a sample of what the text file that will be processed by this
proposed system will contain. I appreciate any pointers and
suggestions.
Mine is "Don't do that." Seriously. What I understand is that you are
proposing to design and write a parser for yet another Domain Specific
Language -- that will require knowledge to use that is useless outside
the specific domain. I expect that is will come to duplicate the basic
facilities of existing languages. Users will want to be able to
calculate, make conditional calculations and constructions, iterate*,
and define functions (subroutines, macros). Why bother to reinvent all
that? It often becomes a mess. (Or you will offer or people will want to
mix Python with the dsl. That also can become a mess.)
Instead, write a pypovray package incorporating the POV macros, that can
be imported into a python program. Write a tutorial for the specific
parts of Python that users will need.
For instances, someone wants to place duplicate or parameterized models
on a rectangular grid, along an arc, or even at random locations.
possible user file content for parsing // in
the following the python interface program reads //+ the contents of
the file "other.file" as if its content //+ were located at this
point.
# this is a python comment. trivial difference from //
include other.file
import other.file # with Python's variations
# or
exec(open('other.file')) # but it is nearly always better to
# keep the separate namespace. What if a name in other.file
# is the same as used below?
import pypovray as ppr
//In the following the python interface makes "snap_size" a //+
global parameter
> declare snap_size = 10
snap_size = 10 # the extra word is just noise
// In the following "buildingLevel" is a class that is //+ called
and passed the parameters in parenthesis. buildingLevel("FirstLevel",
3000)
// In the following "snapOffset" is a class that is //+ called and
passed the parameters in parenthesis.
snapOffset("Closet-S1_r1", "Closet-S2_r3", <0,0,0>)
Already legal Python
# at the end of the file
ppr.run(args)
# Reads globals(), which python has nicely created for you, to create
the master scene description and output whatever is needed for povray.
It could be part of a template.py file you provide.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing files in python
On Mon, Dec 24, 2012 at 4:19 AM, Kene Meniru wrote:
> Hello: I am writing a program that is made up of a collection of POV-Ray
> macros. POV-Ray is available at povray.org. It is a ray-tracing program that
> reads a scene description language (SDL) to create photo-realistic images. At
> this time my program (for modeling building information) is so huge that I am
> finding it difficult managing the macros and I am not even near completion.
I love POV-Ray! Great software, but the input language does at times
lack something, so I'm not surprised that you're wanting a pre-parser.
> possible user file content for parsing
> // in the following the python interface program reads
> //+ the contents of the file "other.file" as if its content
> //+ were located at this point.
> include other.file
>
> //In the following the python interface makes "snap_size" a
> //+ global parameter
> declare snap_size = 10
>
>
> // In the following "buildingLevel" is a class that is
> //+ called and passed the parameters in parenthesis.
> buildingLevel("FirstLevel", 3000)
>
> // In the following "snapOffset" is a class that is
> //+ called and passed the parameters in parenthesis.
> snapOffset("Closet-S1_r1", "Closet-S2_r3", <0,0,0>)
> end of user file content
>
> It should also be possible to include comments using double-slashes, etc.
Hmm. That's a fairly complex file format you have there. I wonder if
it'd be possible to use an actual language parser for it - for
instance, to make this a real Python program. You'd have to use # for
a comment rather than //, and vector syntax may be a problem, but for
the rest, you should be able to do it all with just one extra line at
the top:
from povray_macros import *
You then write all your macros in a file called povray_macros.py and
they'll be conveniently available. For instance:
def buildingLevel(name, altitude):
print("... whatever POV-Ray code is needed ...")
Unfortunately POV-Ray doesn't seem to support reading from stdin, so
you can't simply pipe your program into the renderer. But you can do
it this way:
my_file_whatever_it_is.py >temp.pov
povray +Itemp.pov
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Parsing files in python
Hello: I am writing a program that is made up of a collection of POV-Ray
macros. POV-Ray is available at povray.org. It is a ray-tracing program that
reads a scene description language (SDL) to create photo-realistic images. At
this time my program (for modeling building information) is so huge that I am
finding it difficult managing the macros and I am not even near completion.
I am hoping to move this program to python and am wondering the best way to
approach this.
I would like to model my program after LaTeX. Basically the user writes a text
file using certain key words and numbers and my python program reads this file,
calls the classes that will then work together to calculate the information
that is needed to create an accurate model. The result of this calculation will
be an output to another text file in the appropriate format such as POV-Ray
SDL, OpenSCAD script, etc. This file output can then be rendered by the
corresponding program to produce the actual 3D model. The macros I have now
currently does this but like I said it is getting tedious and most importantly
the fun factor is losing its strength for me.
I have been advised to check out python-ply and I have come across others. I
have not really tried any yet and before I dive into any one of them I was
wondering what else I should know. The following is a sample of what the text
file that will be processed by this proposed system will contain. I appreciate
any pointers and suggestions. Thank you very much.
possible user file content for parsing
// in the following the python interface program reads
//+ the contents of the file "other.file" as if its content
//+ were located at this point.
include other.file
//In the following the python interface makes "snap_size" a
//+ global parameter
declare snap_size = 10
// In the following "buildingLevel" is a class that is
//+ called and passed the parameters in parenthesis.
buildingLevel("FirstLevel", 3000)
// In the following "snapOffset" is a class that is
//+ called and passed the parameters in parenthesis.
snapOffset("Closet-S1_r1", "Closet-S2_r3", <0,0,0>)
end of user file content
It should also be possible to include comments using double-slashes, etc.
Sincerely,
Kene (kemen...@gmail.com)
--
http://mail.python.org/mailman/listinfo/python-list
