Re: [Tutor] python parser
I'm forwarding this to the list. Please always reply-all so a copy goes to the list. johnf wrote: On Monday 22 December 2008 03:03:44 pm you wrote: More thoughts on converting VFP to Python: line-by-line can take into account control structures. One just increases or decreases the indent when encountering the start or end of a structure. a bigger challenge is handling the work-area related commands such as Scan, Replace, Skip, Set Relation, Actually Bob I'm using Dabo which has the above covered. You should take a close look at Dabo. I am able to do most everything I've ever done in VFP. I don't use IDE Dabo provides and I'm able to get everything done. At this point I can almost copy any VFP form. the concept of data sessions So far I'm able to read the dbc and have most of what is required for the data sessions macro substitution That looks like a problem for the moment. all the GUI stuff would be translated to one of the python GUI packages. Not a trivial task! So far it has not been to bad. The real issue is converting to sizers. But the windows side complete. An outline: create initial dictionary of all initial special characters, keywords including their abbreviations, ...? create dictionary to hold all user-generated names (variables, classes, procedures, windows, ...?) awareness while scanning of #define and blocks such as text .. endtext to ignore scan file for procedure/function/define certain initial characters take precedence = ? & # * && @ \ this list may be extended temporarily by commands like text ; is a special final character assignment statement? initial word a keyword or function? if none of the above error? and on and on. If I had a decent parser I think the rest would be easy. But I don't and I do not understand the all the CS stuff. -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
More thoughts on converting VFP to Python: line-by-line can take into account control structures. One just increases or decreases the indent when encountering the start or end of a structure. a bigger challenge is handling the work-area related commands such as Scan, Replace, Skip, Set Relation, the concept of data sessions macro substitution all the GUI stuff would be translated to one of the python GUI packages. Not a trivial task! An outline: create initial dictionary of all initial special characters, keywords including their abbreviations, ...? create dictionary to hold all user-generated names (variables, classes, procedures, windows, ...?) awareness while scanning of #define and blocks such as text .. endtext to ignore scan file for procedure/function/define certain initial characters take precedence = ? & # * && @ \ this list may be extended temporarily by commands like text ; is a special final character assignment statement? initial word a keyword or function? if none of the above error? and on and on. -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
Hi John! I have written a (currently mostly defunct) compiler in Python for a specialized programming language. The parser may serve as an example for you. Introduction: http://freeode.berlios.de/ The parser: https://svn.berlios.de/wsvn/freeode/trunk/freeode_py/freeode/simlparser.py The method --- Parser._defineLanguageSyntax --- creates the Pyparsing object. There are gigantic amounts of (mostly correct) comments in the file. Excuse the bad English and the typos. The style is somewhat adapted to the IDE Pydev. The parser reads the program text (no scanner) and creates a tree structure from it. I call it AST or Parse Tree (I can't decide how to call it). This tree is used in subsequent passes of the compiler. I use a modified version of Pyparsing where I fixed things that I considered bugs, and where I implemented some of my wishes: https://svn.berlios.de/wsvn/freeode/trunk/freeode_py/freeode/third_party/pyparsing.py Kind regards, Eike. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
On Mon, 22 Dec 2008 09:39:37 -0800, johnf wrote: > Well I'm attempting to convert VFP and MsSQL (stored procedures and triggers) > to Python and Postgres (functions and triggers). Actually, python is very > close to VFP and I would convert very well. MsSQL is another question. Hi John, I hope this message finds you well. You might be interested in the following: http://dabodev.com/ "Dabo is a 3-tier, cross-platform application development framework, written in Python atop the wxPython GUI toolkit. And while Dabo is designed to create database-centric apps, that is not a requirement. Lots of people are using Dabo for the GUI tools to create apps that have no need to connect to a database at all." NB: The VFP-angle is well and truely catered for: "Background Dabo's authors, Ed Leafe and Paul McNett, have strong backgrounds in database application development using the awesome and underrated Microsoft Visual FoxPro development environment. While Visual FoxPro shines at developing data-centric applications, it has one limitation that cannot be ignored: it only runs on Microsoft Windows, and Ed and Paul both have clients that want their applications to run on Linux and Macintosh. We are sure we are not alone in this regard: it is a multi-platform world with more diverse needs than one vendor can fulfill. Ed and Paul got to talking one day: Paul had been researching various multiplatform GUI toolkits for about 18 months, and Ed has lots of experience developing the Visual FoxPro Codebook framework. We decided to work together to make a framework for developing robust data-centric applications for multi-platform deployment. We've come up with a design that is simple, flexible, and robust, and we've begun developing our own client applications using the Dabo framework." G. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
On Mon, Dec 22, 2008 at 12:39 PM, johnf wrote: > I paid for and downloaded a paper from O'Reilly books on what I > thought was going to be on 'pyparser'. But that turned out to be mostly > theory. And nothing about the use of pyparser. Was that "Getting Started with Pyparsing" ? I would be very surprised if that was all theory, it is written by the author of pyparsing and he likes examples :-) and the ToC looks like it is mostly extended examples. > So far no links to tutorials? David Mertz has written some columns on parsing, see http://www.ibm.com/developerworks/views/linux/libraryview.jsp?sort_by=Relevance&show_abstract=true&show_all=false&search_flag=true&topic_by=All+topics+and+related+products&type_by=Articles&search_by=charming+python&Submit.x=0&Submit.y=0 You might also try his book: http://gnosis.cx/TPiP/ Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
On Monday 22 December 2008 10:34:12 am Alan Gauld wrote: > "johnf" wrote > > > So I ask you guys is there a link to a practical tutorial that > > provides hands > > on information on the use of a python parser. > > One place to loook for all things to do with processing text is Mertz' > Text > Processing in Python. He discuissed parsers in some detail although, > infuriatingly, doesn't spend a lot of time on the standard library > parsers. > But the principles are common and the modules he uses readily > available. > > Its available online at: > > http://gnosis.cx/TPiP/ > > HTH, Interesting - I'll start reading right now. -- John Fabiani ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
johnf wrote: On Monday 22 December 2008 09:15:13 am bob gailer wrote: johnf wrote: I've been in the programming business for over 20 years and I have never had a need for a parser. But recently I have need to convert code from one language to a my new language python. Pray tell: what is the other language, and why do you want to convert programs? I assume you want to convert so you can then modify / extend the old programs. If you just plan to run them in Python "as is" I see no value in converting! It is hard to map programs in other languages to Python because Python is so different. A direct translation would not take advantage of python's uniqueness. What a better way to learn the new language python than a new project. I decided it might be time to learn a little something about parsers too. However, I soon discovered that I was walking into the world of compiler writers and theories of computer scientist. I paid for and downloaded a paper from O'Reilly books on what I thought was going to be on 'pyparser'. But that turned out to be mostly theory. And nothing about the use of pyparser. So I ask you guys is there a link to a practical tutorial that provides hands on information on the use of a python parser. I'd like to see something that demo's converting a real language to python. When I google parsers I have found a few simple code examples of parsing a float. Not really much help (I could have done that using Regex) when you want to parse 'if,then' statements that can be recursive. FWIW I have written a parser for parts of the CMS Pipelines language. I provide a form of BNF for the syntax, generate parsers from that, run a "program" through that, which generates lists or dictionaries. I have no need to create a program from that. The Pipelines language is very simple, so I am not dealing with control structures, just sequences of words and symbols. Well I'm attempting to convert VFP and MsSQL (stored procedures and triggers) to Python and Postgres (functions and triggers). Actually, python is very close to VFP and I would convert very well. MsSQL is another question. I started writing a VFP - Python converter a few years ago. I will see if I have it around. It has been suggested that I just read one line at a time and convert. I could do that but I did want to expand my understanding by using a parser. And the one line at a time does not lend it self to control structures. Even if I do a get the convert to work there would still be a need for hand coding. But it would provide a good start. So far no links to tutorials? -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
"johnf" wrote So I ask you guys is there a link to a practical tutorial that provides hands on information on the use of a python parser. One place to loook for all things to do with processing text is Mertz' Text Processing in Python. He discuissed parsers in some detail although, infuriatingly, doesn't spend a lot of time on the standard library parsers. But the principles are common and the modules he uses readily available. Its available online at: http://gnosis.cx/TPiP/ HTH, -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
On Monday 22 December 2008 09:15:13 am bob gailer wrote: > johnf wrote: > > I've been in the programming business for over 20 years and I have never > > had a need for a parser. But recently I have need to convert code from > > one language to a my new language python. > > Pray tell: what is the other language, and why do you want to convert > programs? > > I assume you want to convert so you can then modify / extend the old > programs. If you just plan to run them in Python "as is" I see no value > in converting! > > It is hard to map programs in other languages to Python because Python > is so different. A direct translation would not take advantage of > python's uniqueness. > > > What a better way to learn the new > > language python than a new project. I decided it might be time to learn > > a little something about parsers too. However, I soon discovered that I > > was walking into the world of compiler writers and theories of computer > > scientist. I paid for and downloaded a paper from O'Reilly books on what > > I thought was going to be on 'pyparser'. But that turned out to be > > mostly theory. And nothing about the use of pyparser. > > > > So I ask you guys is there a link to a practical tutorial that provides > > hands on information on the use of a python parser. I'd like to see > > something that demo's converting a real language to python. When I > > google parsers I have found a few simple code examples of parsing a > > float. Not really much help (I could have done that using Regex) when > > you want to parse 'if,then' statements that can be recursive. > > FWIW I have written a parser for parts of the CMS Pipelines language. I > provide a form of BNF for the syntax, generate parsers from that, run a > "program" through that, which generates lists or dictionaries. I have no > need to create a program from that. > > The Pipelines language is very simple, so I am not dealing with control > structures, just sequences of words and symbols. Well I'm attempting to convert VFP and MsSQL (stored procedures and triggers) to Python and Postgres (functions and triggers). Actually, python is very close to VFP and I would convert very well. MsSQL is another question. It has been suggested that I just read one line at a time and convert. I could do that but I did want to expand my understanding by using a parser. And the one line at a time does not lend it self to control structures. Even if I do a get the convert to work there would still be a need for hand coding. But it would provide a good start. So far no links to tutorials? -- John Fabiani ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
Am Mon, 22 Dec 2008 07:18:41 -0800 schrieb johnf : > I've been in the programming business for over 20 years and I have > never had a need for a parser. But recently I have need to convert > code from one language to a my new language python. What a better > way to learn the new language python than a new project. I decided > it might be time to learn a little something about parsers too. > However, I soon discovered that I was walking into the world of > compiler writers and theories of computer scientist. I paid for and > downloaded a paper from O'Reilly books on what I thought was going to > be on 'pyparser'. But that turned out to be mostly theory. And > nothing about the use of pyparser. Well, scanning and parsing are rather one of the older fields in C.S. So if I got that correct you want to convert programs from language A to language Python, and on tops you'll probably want the generated program to be human-readable and as fast as possible. Without knowing what A is, it's hard to tell you more, but writing a parser for almost any programming language is hard, and emitting fast and human readable code is even harder. Furthermore, doing it without the theoretical background is not simpler either. Btw, possible implementations of a parser depend upon a number of "theoretical" properties of the grammer you want to implement. > > So I ask you guys is there a link to a practical tutorial that > provides hands on information on the use of a python parser. I'd > like to see something that demo's converting a real language to > python. When I google parsers I have found a few simple code > examples of parsing a float. Not really much help (I could have done > that using Regex) when you want to parse 'if,then' statements that > can be recursive. Take a look at the examples at the pyparsing website, it even includes a python parser :) Andreas > > Thanks in advance. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] python parser
johnf wrote: I've been in the programming business for over 20 years and I have never had a need for a parser. But recently I have need to convert code from one language to a my new language python. Pray tell: what is the other language, and why do you want to convert programs? I assume you want to convert so you can then modify / extend the old programs. If you just plan to run them in Python "as is" I see no value in converting! It is hard to map programs in other languages to Python because Python is so different. A direct translation would not take advantage of python's uniqueness. What a better way to learn the new language python than a new project. I decided it might be time to learn a little something about parsers too. However, I soon discovered that I was walking into the world of compiler writers and theories of computer scientist. I paid for and downloaded a paper from O'Reilly books on what I thought was going to be on 'pyparser'. But that turned out to be mostly theory. And nothing about the use of pyparser. So I ask you guys is there a link to a practical tutorial that provides hands on information on the use of a python parser. I'd like to see something that demo's converting a real language to python. When I google parsers I have found a few simple code examples of parsing a float. Not really much help (I could have done that using Regex) when you want to parse 'if,then' statements that can be recursive. FWIW I have written a parser for parts of the CMS Pipelines language. I provide a form of BNF for the syntax, generate parsers from that, run a "program" through that, which generates lists or dictionaries. I have no need to create a program from that. The Pipelines language is very simple, so I am not dealing with control structures, just sequences of words and symbols. -- Bob Gailer Chapel Hill NC 919-636-4239 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
