Re: [sage-devel] Re: A Mathematica parser for Sage.

[Dr. David Kirkby]

On 09/ 9/10 03:32 AM, Tim Daly wrote:

Some of the questions you have about why lisp are answered in:
http://channel9.msdn.com/shows/Going+Deep/Expert-to-Expert-Rich-Hickey-and-Brian-Beckman-Inside-Clojure/

which is about Clojure, a more recent lisp although the ideas are
essentially the same in Common Lisp.

Tim Daly


Thank you Tim.

I think you, RJF and a few others have convinced me Lisp would be the best for 
this. I'll revisit this once the 64-bit Solaris ports are done.


If someone wants to make a parser from Python, let them do it. But I'm pretty 
convinced it is not the best way to approach it.


However, I've had no success running RJF's code. I would have thought the ANSI 
Common Lisp would have covered how commands are loaded, but I am told that is 
not so. If Richard could suggest how his code might be modified to run with ECL, 
then I'd like to give it a quick try and post my results here.


Dave


David Kirkby wrote:

On 5 September 2010 10:14, Mitesh Patel qed...@gmail.com wrote:

Hi David,

On 09/05/2010 03:52 AM, David Kirkby wrote:

I'm quite happy to be that someone who learns Lisp - I'm serious
thinking of buying a book on it. Unfortunately, they tend to be quite
expensive, as do books on writing compilers.

Some time ago, I found Practical Common Lisp by P. Seibel, which is
available for free online:

http://www.gigamonkeys.com/book/

But I have not read it and am not familar with Lisp.

Sincerely,
Mitesh


Thank you for that - it is one of the books on Amazon I was considering.

http://www.amazon.com/Practical-Common-Lisp-Peter-Seibel/dp/1590592395

It's $52.29 on Amazon - to find it free is quite nice.

I also found Common Lisp: A Gentle Introduction to Symbolic
Computation - all

http://www.cs.cmu.edu/~dst/LispBook/index.html

I want to finish the Solaris port first, getting it building properly
on 64-bit systems. But then I might take a look trying to parse
Mathematica and do something useful with it.

Dave





--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[rjf]

 However, I've had no success running RJF's code. I would have thought the ANSI
 Common Lisp would have covered how commands are loaded, but I am told that is
 not so. If Richard could suggest how his code might be modified to run with 
 ECL,
 then I'd like to give it a quick try and post my results here.

Reading this thread, it seems that tkosan installed my code. Did you
follow his suggestions?
If you have it running on ABCL,
there should be no modification to run on ECL, but frankly, I have no
copy of ECL and do not intend to get one.

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Tim Daly]

Some of the questions you have about why lisp are answered in:
http://channel9.msdn.com/shows/Going+Deep/Expert-to-Expert-Rich-Hickey-and-Brian-Beckman-Inside-Clojure/
which is about Clojure, a more recent lisp although the ideas are
essentially the same in Common Lisp.

Tim Daly

David Kirkby wrote:

On 5 September 2010 10:14, Mitesh Patel qed...@gmail.com wrote:
  

Hi David,

On 09/05/2010 03:52 AM, David Kirkby wrote:


I'm quite happy to be that someone who learns Lisp - I'm serious
thinking of buying a book on it. Unfortunately, they tend to be quite
expensive, as do books on writing compilers.
  

Some time ago, I found Practical Common Lisp by P. Seibel, which is
available for free online:

http://www.gigamonkeys.com/book/

But I have not read it and am not familar with Lisp.

Sincerely,
Mitesh



Thank you for that - it is one of the books on Amazon I was considering.

http://www.amazon.com/Practical-Common-Lisp-Peter-Seibel/dp/1590592395

It's $52.29 on Amazon - to find it free is quite nice.

I also found Common Lisp: A Gentle Introduction to Symbolic Computation - all

http://www.cs.cmu.edu/~dst/LispBook/index.html

I want to finish the Solaris port first, getting it building properly
on 64-bit systems. But then I might take a look trying to parse
Mathematica and do something useful with it.

Dave

  


--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Dr. David Kirkby]

On 09/ 6/10 01:40 AM, rjf wrote:



On Sep 5, 2:40 pm, David Kirkbydavid.kir...@onetel.net  wrote:



It is however very clear there are far more people know Python than
Lisp, so use of Python is more attractive to more developers.

Dave


More people know C than Python.  More people use Windows than Linux.
More people have IQs below 110 than above 110.


I assume the first is true. I don't know that for a fact.

Certainly the second and third are true.


There are many reasons to favor lisp for writing a parser for
Mathematica, but one
that is hard to deny is that it is already available free.


I agree the fact one exists is a major advantage. But conversely, as I said to 
Tim, there are advantages in doing something in a well known language (e.g. 
Python).


But can you list objectively what other advantages Lisp has for this task. I'd 
be interested in what they are. Two I can think of are


1) There is already one existing. I don't know how comprehensive it is yet.

2) There's an ANSI standard for Lisp, whereas Python changes. If one wrote to 
the ANSI standard, then the code should be portable and if the list 
implementation used (e.g. ECL) was to become unmaintained, then switching to 
another would not be a problem.


But what other advantages are there?

Doing it in C was quite attractive to me personally until you pointed out that 
lex/yacc would solve the problem.



There is, by the way, another way to parse Mathematica, if you have a
whole bunch of
stuff you want to convert to (say) lisp. You read it in to Mathematica
and print it out
in FullForm,  or run a 3-line program (in Mathematica) to print it out
in lisp in the first
place.


But that does not help with an open-source implementation.


I think Python is better than many other languages for some purposes,
and not better for some others.
   I think we already had this discussion.


Yes, but I'd like to see an objective comparison of Python vs Lisp. Unlike some, 
I am not discounting Lisp. Without knowing the languge though, it is hard for me 
to form an objective comparison.



RJF


Dave

--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Dr. David Kirkby]

On 09/ 6/10 07:37 AM, Dr. David Kirkby wrote:


Doing it in C was quite attractive to me personally until you pointed
out that lex/yacc would solve the problem.


Oops - you said lex/yacc would not solve the problem.

Dave

--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Dr. David Kirkby]

On 09/ 2/10 11:41 PM, rjf wrote:



On Sep 2, 2:23 pm, Dr. David Kirkbydavid.kir...@onetel.net  wrote:

On 09/ 2/10 06:10 AM, rjf wrote:




the mathematica syntax parser that I wrote appears to run inside
Maxima, so
you can, if you wish, feed such text to the mma-in-maxima system.


Sorry to sound green, but I barely know Maxima, and do not know Lisp at all.

I do know Mathematica - though I'm certainly not at expert at it.

There's no README file in the source code I found of yours, so it's far from
obvious to me how I would use it.

um, I don't know where you looked, but

here is one place..
http://www.cs.berkeley.edu/~fateman/mma1.6/

I forget exactly, but I think it is approximately,  get all those
files in that directory. start a lisp in that directory.

  (load capsonlyparser.lisp)
(in-package :mma)

(p)


then type some mathematica and hit two returns.


It would be really useful if you could put your code in a tar file with a 
README.txt. I had quite a problem downloading this. For some reason wget -r 
started to wonder off into other directories, so:


wget -r http://www.cs.berkeley.edu/~fateman/mma1.6/

did not work.

Anyway, I think I have the code OK now, but I did not get very far with that at 
all.


I used 'ecl' which is built as part of Sage, set up the environment to run run 
sage, but did not run it. Instead I called ecl directly after sticking your lisp 
file in the same directory as the list executable. The ecl interpreter seems to 
work - I could add 1000 to 100 and get 1100, but I got nowhere trying to load 
your package.


(sage subshell) hawk:bin drkirkby$ ./ecl
ECL (Embeddable Common-Lisp) 10.2.1
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.
Top level.
 (+ 1000 100)

1100
 (load capsonlyparser.lisp)

;;; Loading 
/export/home/drkirkby/4/sage-4.5.3.alpha2/local/bin/capsonlyparser.lisp


There exists no package with name MMA
Available restarts:

1. (RESTART-TOPLEVEL) Go back to Top-Level REPL.

Broken at SI:BYTECODES. [Evaluation of: (IN-PACKAGE :MMA)]
 File: 
#P/export/home/drkirkby/4/sage-4.5.3.alpha2/local/bin/capsonlyparser.lisp 
(Position #761)



--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[tkosan]

On Sep 5, 5:40 pm, David Kirkby david.kir...@onetel.net wrote:

What seems to be a common theme is using Python, but it's not clear to
me this is optimal. It might be simpler to pass the unmodified input
from Sage to Lisp and do the rest there.

RJF thinks Lisp is the best language
William thinks Python is God

It needs an objective look at this.

I started learning Lisp a couple of years ago when I left the Sage
project and created a fork of a Lisp-based CAS called Yacas.  The fork
is called MathPiper (http://mathpiper.org) and I spent a full year
refactoring it and documenting it as part of the process of learning
how it worked.

I have experience using a compiler compiler like Yacc to create a
programming language interpreter and I also have some experience
writing programs in Python.  Here are some of my thoughts on using
Python vs. Lisp for developing CAS related programs.

I think one of the most critical differences between Python and Lisp
is that Python is a programming language, but Lisp isn't really a
programming language for reasons which are described in this blog
post:

http://lispy.wordpress.com/2007/10/24/lisp-isnt-really-a-programming-language/

The post begins by stating that ...practically any program written in
Lisp could be considered to be a DSL.  For those who don't know, the
term DSL stands for Domain Specific Language and as an example,
MathPiper is a DSL which is written in lisp.


The whole post is informative, but the following paragraph gets to the
heart of what I think makes Lisp fundamentally different than Python:

Now there’s an interesting thing about those data structures you make
with car, cons, and cdr. They’re an abstract syntax tree. Now in most
other languages, if someone wants to do something cool, they convert
what they’re working with to an abstract syntax tree and then go to
town with it. If you want to convert something from in-fix notation to
something more lisp-like, its trivial once you can get that original
notation into an abstract syntax tree. But the thing is, if you’re
using lisp, you’re using lists. And Lisp’s lists are essentially
abstract syntax trees by definition. So in Lisp, a lot hard things are
easy… by default.  Finally, your Lisp code is itself written in terms
of list data structures. This means its easy to write code to execute
transformations of your code. This is what macros do and this is why
other languages can’t do this.

This isn’t a good place to go into the details of what an Abstract
Syntax Tree (AST) is or how with Lisp one works with an AST directly,
but perhaps I can a provide bit of a feel for this.

Here is a simple while loop in MathPiper syntax, along with the Lisp
AST code it is transformed into:

%mathpiper,title=The output fold contains the internal Lisp form of
the MathPiper while loop.
LispForm(Hold([
x := 1;
While(x = 10)
[
  Write(x,,);
  x := x + 1;
];
]));
%/mathpiper

%output,preserve=false
  Result: snip

  Side Effects:
  (Prog
  (:= x 1 )
  (While
(= x 10 )
(Prog
  (Write x , )
  (:= x
(+ x 1 )
.   %/output


For people who have not programmed in Lisp yet, it is usually
challenging to get one's mind around what is being indicated by the
Lisp syntax.  I experienced this difficulty too when I first started
learning MathPiper's Lisp, so I made a function called ViewList which
displays Lisp code in graphic box diagram form.  Here is the above
Lisp code displayed in this graphic form:

https://sites.google.com/site/mathpiper/screenshots/graphic_view_of_while_loop_lisp_code.png?attredirects=0


Its difficult to explain how amazingly powerful it is to be able to
directly work with the AST form of the code one is developing.  In the
blog post it states You don’t write Lisp code so much as you extend
the Lisp language until it becomes a custom language for whatever it
is you’re trying to do.  This ability enables a developer to
customize and extend the MathPiper syntax to an astonishing degree.
If you were interested in experiencing some of the power of Lisp
before diving in and learning Lisp syntax, MathPiper provides a
relatively easy was to do this.


Now that I understand MathPiper's Lisp fairly well, I have been
spending time learning Common Lisp.  Recently, I have been
experimenting with using Mockmma to translate Mathematica expressions
into Lisp format so that I can read them into MathPiper.  For example,
here is a short Mathmatica expression:

-PolyLog[2,1-a*c-b*c*x]/e

And here is the Lisp code that Mockmma translates it into:

(TIMES
 (TIMES -1
  (POLYLOG 2
   (PLUS 1 (TIMES -1 (TIMES A C)) (TIMES -1 (TIMES B C X)
 (POWER E -1))


The following program uses MathPiper's Lisp code reader to read the
code into a MathPiper internal AST structure:

%mathpiper,title=MMA to Lisp
lispCode := PipeFromString(
(TIMES
 (TIMES -1
  (POLYLOG 2
   (PLUS 1 (TIMES -1 (TIMES A C)) (TIMES -1 (TIMES B C 

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Mitesh Patel]

On 09/05/2010 05:23 AM, David Kirkby wrote:
 On 5 September 2010 10:14, Mitesh Patel qed...@gmail.com wrote:
 On 09/05/2010 03:52 AM, David Kirkby wrote:
 I'm quite happy to be that someone who learns Lisp - I'm serious
 thinking of buying a book on it. Unfortunately, they tend to be quite
 expensive, as do books on writing compilers.

 Some time ago, I found Practical Common Lisp by P. Seibel, which is
 available for free online:

 http://www.gigamonkeys.com/book/

 But I have not read it and am not familar with Lisp.


Unless Scheme counts.  Another potentially useful and freely available
text is Structure and Interpretation of Computer Programs (SICP):

http://mitpress.mit.edu/sicp/


 I also found Common Lisp: A Gentle Introduction to Symbolic Computation - 
 all
 
 http://www.cs.cmu.edu/~dst/LispBook/index.html

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[tkosan]

On Sep 6, 3:49 am, Dr. David Kirkby david.kir...@onetel.net wrote:

 Anyway, I think I have the code OK now, but I did not get very far with that 
 at
 all.

 I used 'ecl' which is built as part of Sage, set up the environment to run run
 sage, but did not run it. Instead I called ecl directly after sticking your 
 lisp
 file in the same directory as the list executable. The ecl interpreter seems 
 to
 work - I could add 1000 to 100 and get 1100, but I got nowhere trying to load
 your package. snip

I put a copy of ABCL (Armed Bear Common Lisp) into the directory which
contains the copy of Mockmma that I have been playing with, placed the
directory into a zip file named mockmma_with_abcl.zip, and then
uploaded it to the MathPiper development releases server:

http://www.mathpiper.org/development-releases


ABCL is written in Java, so you will need a copy of Java installed if
you want to try running Mockmma with it.  After you unzip the archive,
just change into the mockmma_with_abcl directory and enter the
following commands to launch abcl, initialize the Mockmma parser, and
have it parse some Mathematica expressions:

mockmma_with_abcl$ java -jar abcl.jar

Armed Bear Common Lisp 0.21.0
Java 1.6.0_20 Sun Microsystems Inc.
Java HotSpot(TM) 64-Bit Server VM
Low-level initialization completed in 0.57 seconds.
Startup completed in 1.667 seconds.
Type :help for a list of available commands.

CL-USER(1): (load mma)
T

CL-USER(2): (load parser)
T

CL-USER(3): (in-package :mma)
#PACKAGE MMA

MMA(4): (p)
2+2
(PLUS 2 2)

MMA(5): (p)
Sin[3]
(SIN 3)

MMA(6): (p)
-PolyLog[2,1-a*c-b*c*x]/e
(TIMES
 (TIMES -1
  (POLYLOG 2
   (PLUS 1 (TIMES -1 (TIMES A C)) (TIMES -1 (TIMES B C X)
 (POWER E -1))
MMA(7):


Hope this helps,

Ted

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[David Kirkby]

On 2 September 2010 23:41, rjf fate...@gmail.com wrote:

 There's no README file in the source code I found of yours, so it's far from
 obvious to me how I would use it.
 um, I don't know where you looked, but

 here is one place..
 http://www.cs.berkeley.edu/~fateman/mma1.6/

Adding a README file would help a lot

 I forget exactly, but I think it is approximately,  get all those
 files in that directory. start a lisp in that directory.

  (load capsonlyparser.lisp)
 (in-package :mma)

 (p)

Thank you - I will give it a try.

 then type some mathematica and hit two returns.

 Better lisps that don't do everything in caps would use the regular
 parser file, which is easier to read and modify.

Hopefully ECL would be able to use the simpler file.

 Certainly from reading the paper, it gives no indication that this runs as 
 part
 of Maxima.

 Right. The adaptation was the work of someone else to load it into
 Maxima. Check sourceforge.
 John Lapeyre was the author.

Thank you.

 It gives the impression to me that this is written in lisp, to run
 under the lisp interpreter directly - not under Maxima. I'm a bit confused on
 this issue.

 My program can run without Maxima.


  The intent in that project is mainly to take mma syntax for
  expressions and map it into
  maxima,  and not take the big step of having a more-or-less full
  mathematica
  evaluator in Maxima.

 How complex in your opinion would it be to create a parser for Mathematica in
 C/C++/Python/Lisp/whaterver-is-most-suitable which could call the right part 
 of
 Sage for a particular tasks? In other words, call Maxima for integration,
 perhaps the GSL for Bessel functions, etc.

 Well, if you want to load GSL or whatever into lisp, then you can use
 the parser that
 is already in Lisp.

Can you elaborate more on this - I do not know what you mean.

 As for a whatever is most suitable to create a parser for
 Mathematica, that would have
 to be Mathematica, since it is already done.  If you want an #2, I
 would say Lisp.

 I'm pretty sure this is a non-trivial project, but my background is 
 engineering,
 not maths or computer science, so I don't have a lot of understanding of the
 complexity involved.

 I just bought a book on Lex + Yacc (around the US equivalent of $1 used). 
 I've
 no idea if they would be suitable tools or not, but at 1$ I'm not going to go
 bankrupt.

 Unfortunately, WRI does not give you a context free grammar for
 Mathematica, nor
 does it give you a description of the tokens that you can use.  So Lex
 and Yacc are
 not directly applicable.

How far could one get with Lex/Yacc? Some of the constructs I see in
your paper, such as the confusion that can arrise with ! for note and
!! for double factorial, are a little odd. One could easily re-write
small parts of code that are ambiguous.  If one was able to write
something able to parse 99% of Mathematica code, that would probably
be enough for me. Even if the parser was 100% perfect, one would only
be able to implement a small subset of Mathematica's features in Sage
using the current Sage code base.


 I gather there are several types of parser LL, LR etc. I've no idea what 
 type is
 best for parsing Mathematica.

 Since Mathematica's language is not a context free language, as far as
 I can tell,
 none of these abstract classes of parsers will entirely work.  There
 are constructs in
 Mathematica that appear to require ad hoc parsing techniques.  Though
 much of it
 can be done by some kind of table-driven parser, e.g. LALR or LL.  The
 MockMMA
 lexer and parser are modified from a recursive descent parser with
 variable look-ahead.

This is making me think perhaps Lex/Yacc approach may not be so entirely dumb.

 My suggestion is that you find someone who knows lisp, or is willing
 to learn it, and
 then use the parser, free.
 or I suppose you could have someone write a parser in Python, but then
 you'd might
 have to write a substantial portion of Lisp, too, for the
 representation and manipulation
 of the intermediate expressions.  So Lisp turns out to be a pretty
 substantial win.



 RJF

I'm quite happy to be that someone who learns Lisp - I'm serious
thinking of buying a book on it. Unfortunately, they tend to be quite
expensive, as do books on writing compilers.

However, one very practical problem is that if one writes code in
Lisp, the number of people that would be willing to extend that code
drops dramatically, as the number of people knowing the language
drops.

I'm not one who preaches Python is God. There have been many languages
come and go. I don't know what the lifetime of Python will be, but I
suspect it will be found only is history books when C is still a
common language. I know you don't happen to like C either.

I'm not denying Lisp my be technically the easiest language to do this
in, but that is not the only factor to be considered if code is going
to be edited by a number of people. Of course, lex/yacc falls into the
same problem as Lisp - they 

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Mitesh Patel]

Hi David,

On 09/05/2010 03:52 AM, David Kirkby wrote:
 I'm quite happy to be that someone who learns Lisp - I'm serious
 thinking of buying a book on it. Unfortunately, they tend to be quite
 expensive, as do books on writing compilers.

Some time ago, I found Practical Common Lisp by P. Seibel, which is
available for free online:

http://www.gigamonkeys.com/book/

But I have not read it and am not familar with Lisp.

Sincerely,
Mitesh

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[David Kirkby]

On 5 September 2010 10:14, Mitesh Patel qed...@gmail.com wrote:
 Hi David,

 On 09/05/2010 03:52 AM, David Kirkby wrote:
 I'm quite happy to be that someone who learns Lisp - I'm serious
 thinking of buying a book on it. Unfortunately, they tend to be quite
 expensive, as do books on writing compilers.

 Some time ago, I found Practical Common Lisp by P. Seibel, which is
 available for free online:

 http://www.gigamonkeys.com/book/

 But I have not read it and am not familar with Lisp.

 Sincerely,
 Mitesh

Thank you for that - it is one of the books on Amazon I was considering.

http://www.amazon.com/Practical-Common-Lisp-Peter-Seibel/dp/1590592395

It's $52.29 on Amazon - to find it free is quite nice.

I also found Common Lisp: A Gentle Introduction to Symbolic Computation - all

http://www.cs.cmu.edu/~dst/LispBook/index.html

I want to finish the Solaris port first, getting it building properly
on 64-bit systems. But then I might take a look trying to parse
Mathematica and do something useful with it.

Dave

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[rjf]

On Sep 5, 2:40 pm, David Kirkby david.kir...@onetel.net wrote:
 On 5 September 2010 22:13, William Stein wst...@gmail.com wrote:

  On Sunday, September 5, 2010, David Kirkby david.kir...@onetel.net wrote:
  RJF thinks Lisp is the best language
  William thinks Python is God

  No I don't.

  I chose Python for Sage because in 2004 it was the closest popular
  open source language to Magma.

 God was the wrong word. But you have said Python is a beautiful
 language.  Your view of the language contrasts greatly with that of
 RJF.

 It's not clear to me the best way to tackle something able to parse
 Mathematica.

 It is however very clear there are far more people know Python than
 Lisp, so use of Python is more attractive to more developers.

 Dave

More people know C than Python.  More people use Windows than Linux.
More people have IQs below 110 than above 110.

There are many reasons to favor lisp for writing a parser for
Mathematica, but one
that is hard to deny is that it is already available free.

There is, by the way, another way to parse Mathematica, if you have a
whole bunch of
stuff you want to convert to (say) lisp. You read it in to Mathematica
and print it out
in FullForm,  or run a 3-line program (in Mathematica) to print it out
in lisp in the first
place.

This would work for, say Rubi.

I think Python is better than many other languages for some purposes,
and not better for some others.
  I think we already had this discussion.

RJF

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[Dr. David Kirkby]

On 09/ 2/10 06:10 AM, rjf wrote:


the mathematica syntax parser that I wrote appears to run inside
Maxima, so
you can, if you wish, feed such text to the mma-in-maxima system.


Sorry to sound green, but I barely know Maxima, and do not know Lisp at all.

I do know Mathematica - though I'm certainly not at expert at it.

There's no README file in the source code I found of yours, so it's far from 
obvious to me how I would use it.


Are there any differences between the paper I found on your web site, and the 
paper published in a peer reviewed journal?


Certainly from reading the paper, it gives no indication that this runs as part 
of Maxima. It gives the impression to me that this is written in lisp, to run 
under the lisp interpreter directly - not under Maxima. I'm a bit confused on 
this issue.



The intent in that project is mainly to take mma syntax for
expressions and map it into
maxima,  and not take the big step of having a more-or-less full
mathematica
evaluator in Maxima.


How complex in your opinion would it be to create a parser for Mathematica in 
C/C++/Python/Lisp/whaterver-is-most-suitable which could call the right part of 
Sage for a particular tasks? In other words, call Maxima for integration, 
perhaps the GSL for Bessel functions, etc.


I realise a complete parser would be impossible due to the fact Sage simply does 
not have all the functionality of Mathematica.


I'm pretty sure this is a non-trivial project, but my background is engineering, 
not maths or computer science, so I don't have a lot of understanding of the 
complexity involved.


I just bought a book on Lex + Yacc (around the US equivalent of $1 used). I've 
no idea if they would be suitable tools or not, but at 1$ I'm not going to go 
bankrupt.


I gather there are several types of parser LL, LR etc. I've no idea what type is 
best for parsing Mathematica.


Dave

--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[rjf]

On Sep 2, 2:23 pm, Dr. David Kirkby david.kir...@onetel.net wrote:
 On 09/ 2/10 06:10 AM, rjf wrote:



  the mathematica syntax parser that I wrote appears to run inside
  Maxima, so
  you can, if you wish, feed such text to the mma-in-maxima system.

 Sorry to sound green, but I barely know Maxima, and do not know Lisp at all.

 I do know Mathematica - though I'm certainly not at expert at it.

 There's no README file in the source code I found of yours, so it's far from
 obvious to me how I would use it.
um, I don't know where you looked, but

here is one place..
http://www.cs.berkeley.edu/~fateman/mma1.6/

I forget exactly, but I think it is approximately,  get all those
files in that directory. start a lisp in that directory.

 (load capsonlyparser.lisp)
(in-package :mma)

(p)


then type some mathematica and hit two returns.

Better lisps that don't do everything in caps would use the regular
parser file, which is easier to read and modify.




 Are there any differences between the paper I found on your web site, and the
 paper published in a peer reviewed journal?

probably not


 Certainly from reading the paper, it gives no indication that this runs as 
 part
 of Maxima.

Right. The adaptation was the work of someone else to load it into
Maxima. Check sourceforge.
John Lapeyre was the author.

It gives the impression to me that this is written in lisp, to run
 under the lisp interpreter directly - not under Maxima. I'm a bit confused on
 this issue.

My program can run without Maxima.


  The intent in that project is mainly to take mma syntax for
  expressions and map it into
  maxima,  and not take the big step of having a more-or-less full
  mathematica
  evaluator in Maxima.

 How complex in your opinion would it be to create a parser for Mathematica in
 C/C++/Python/Lisp/whaterver-is-most-suitable which could call the right part 
 of
 Sage for a particular tasks? In other words, call Maxima for integration,
 perhaps the GSL for Bessel functions, etc.

Well, if you want to load GSL or whatever into lisp, then you can use
the parser that
is already in Lisp.

As for a whatever is most suitable to create a parser for
Mathematica, that would have
to be Mathematica, since it is already done.  If you want an #2, I
would say Lisp.


 I realise a complete parser would be impossible due to the fact Sage simply 
 does
 not have all the functionality of Mathematica.

You could still have a complete parser.You would not have an evaluator
that was complete.


 I'm pretty sure this is a non-trivial project, but my background is 
 engineering,
 not maths or computer science, so I don't have a lot of understanding of the
 complexity involved.

 I just bought a book on Lex + Yacc (around the US equivalent of $1 used). I've
 no idea if they would be suitable tools or not, but at 1$ I'm not going to go
 bankrupt.

Unfortunately, WRI does not give you a context free grammar for
Mathematica, nor
does it give you a description of the tokens that you can use.  So Lex
and Yacc are
not directly applicable.


 I gather there are several types of parser LL, LR etc. I've no idea what type 
 is
 best for parsing Mathematica.

Since Mathematica's language is not a context free language, as far as
I can tell,
none of these abstract classes of parsers will entirely work.  There
are constructs in
Mathematica that appear to require ad hoc parsing techniques.  Though
much of it
can be done by some kind of table-driven parser, e.g. LALR or LL.  The
MockMMA
lexer and parser are modified from a recursive descent parser with
variable look-ahead.

My suggestion is that you find someone who knows lisp, or is willing
to learn it, and
then use the parser, free.
or I suppose you could have someone write a parser in Python, but then
you'd might
have to write a substantial portion of Lisp, too, for the
representation and manipulation
of the intermediate expressions.  So Lisp turns out to be a pretty
substantial win.



RJF

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[kcrisman]

On Sep 1, 11:55 am, David Kirkby david.kir...@onetel.net wrote:
 Has anyone given thought for making Sage read Mathematica syntax? I've
 seen a recent video from William stating it is NOT an aim of Sage to
 be clone of any of the 4 M's - in contrast, Octave is a clone of
 MATLAB.

 Whilst parsing Mathematica code is not an aim of Sage, I doubt many
 would are argue that it would be a useful addition if Sage could read
 Mathematica code, and do something useful with it.

 I could see several way one might attack such a problem.

 1)) Use yacc/lex to create a C library, then call that from Sage.
 2) Write the parser in Python.
 3) Pass the input directly to ECL (Lisp interpreter) and do the
 parsing in Lisp.

 Personally I'd be most comfortable in using lex/yacc/C, less
 comfortable in Python and even less so in Lisp, as I don't have a clue
 about Lisp.

 But actually, the latter approach, using Lisp, which personally I'd be
 *least* comfortable with, might actually be the *most* sensible
 approach. Why?

 1) Most computer algebra systems are based on some form of Lisp -
 including Mathematica I believe, so it can't be a bad choice.

 One might even argue Lisp might have been a better choice than Python
 than Sage, though there are far more Python programmers than Lisp
 programmers, so that's possibly not true. In fact, I think it's fair
 to sage having Python skills on ones CV is more worthwhile than Lisp
 skills.

 (flames for even considering not using Python to /dev/null).

 2) I think there's a library interface to ECL, which could probably
 avoid the need for messing around with pexpect. Send the input
 directly to ECL.

 3) There is also a paper describing a Mathematica parser in Lisp in a
 peer reviewed journal.

 http://portal.acm.org/citation.cfm?id=1089419.1089421

 For those of us not affiliated with an academic institution and so
 have no access to a library (boy, do I miss that), a paper by the same
 name at

 http://www.eecs.berkeley.edu/~fateman/papers/lmath.ps

 is probably almost identical

 4) The source code for that Mathematica parser is available.

 http://www.cs.berkeley.edu/~fateman/mma1.6/

 whilst most obviously written for an old version of Mathematica, a few
 source files were updated in 2006, suggesting it probably works with
 semi-recent versions of Mathematica. In any case, Mathematica is
 reasonably backwards compatible.

 5) The author of the Mathematica parser in Lisp is alive and kicking,
 and a reader of sage-devel.

 Getting Sage to read Mathematica and do useful things with it, should
 make an interesting project for a computer science student. Although I
 don't know much about this, I would be guess this would have to be an
 MSc project, not an undergraduate one as I doubt doing any of this
 would be trivial.

 Once such a parser existed which was usable from Sage, a mathematician
 could probably do useful things with it.

 Of course, I'm aware Sage has an interface to Mathematica, but that
 needs a Mathematica license to be of any use at all.

 Realistically, in my personal opinion, Sage will never have the
 combined functionality of MATLAB, Maple and Mathematica. In fact, I
 don't believe it will ever have the full functionality of any of those
 3. (I also accept that Sage is far ahead of Mathematica in some areas
 such as number theory). On that basis, there's little point in
 worrying about a 100% complete implemention of a Mathematica parser.

 Has anyone got any comments?

whuss at some point added something like this for both Mma and Maple,
though very basic, as part of another ticket (symbolic sums?).  I
can't remember where it is and am unfortunately having some internet
issues :( but anyway I believe this code was merged into Sage at some
point.

- kcrisman

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[David Kirkby]

On 1 September 2010 17:45, kcrisman kcris...@gmail.com wrote:


 On Sep 1, 11:55 am, David Kirkby david.kir...@onetel.net wrote:
 Has anyone given thought for making Sage read Mathematica syntax? I've
 seen a recent video from William stating it is NOT an aim of Sage to
 be clone of any of the 4 M's - in contrast, Octave is a clone of
 MATLAB.

snip

 Getting Sage to read Mathematica and do useful things with it, should
 make an interesting project for a computer science student. Although I
 don't know much about this, I would be guess this would have to be an
 MSc project, not an undergraduate one as I doubt doing any of this
 would be trivial.

snip

 Has anyone got any comments?

 whuss at some point added something like this for both Mma and Maple,
 though very basic, as part of another ticket (symbolic sums?).  I
 can't remember where it is and am unfortunately having some internet
 issues :( but anyway I believe this code was merged into Sage at some
 point.

 - kcrisman

Unless I'm *very* mistaken, doing this would be a non-trivial project,
so I doubt someone would have done it for both Mathematica and Maple
as part of another ticket. But of course I may be wrong.


Dave

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[kcrisman]

On Sep 1, 1:30 pm, David Kirkby david.kir...@onetel.net wrote:
 On 1 September 2010 17:45, kcrisman kcris...@gmail.com wrote:



  On Sep 1, 11:55 am, David Kirkby david.kir...@onetel.net wrote:
  Has anyone given thought for making Sage read Mathematica syntax? I've
  seen a recent video from William stating it is NOT an aim of Sage to
  be clone of any of the 4 M's - in contrast, Octave is a clone of
  MATLAB.

 snip

  Getting Sage to read Mathematica and do useful things with it, should
  make an interesting project for a computer science student. Although I
  don't know much about this, I would be guess this would have to be an
  MSc project, not an undergraduate one as I doubt doing any of this
  would be trivial.

 snip

  Has anyone got any comments?

  whuss at some point added something like this for both Mma and Maple,
  though very basic, as part of another ticket (symbolic sums?).  I
  can't remember where it is and am unfortunately having some internet
  issues :( but anyway I believe this code was merged into Sage at some
  point.

  - kcrisman

 Unless I'm *very* mistaken, doing this would be a non-trivial project,
 so I doubt someone would have done it for both Mathematica and Maple
 as part of another ticket. But of course I may be wrong.

If I recall correctly, it was truly basic, and certainly likely to not
work in many situations - but for a few very specific purposes it was
better than nothing.   I think the following was the relevant bit.

- kcrisman

sage: sage.calculus.calculus.symbolic_sum??
snip
   #. Sage can currently only understand a subset of the output of
Maxima, Maple and
  Mathematica, so even if the chosen backend can perform the
summation the
  result might not be convertable into a Sage expression.
snip
sage: sage.interfaces.mathematica??
snip

def _sage_(self):
r
Try to convert a mathematica expression back to a Sage
expression.

This currently does not implement a parser for the Mathematica
output language,
therefore only very simple expressions will convert
successfully.

EXAMPLES::

sage: m = mathematica('x^2 +
5*y')# optional - mathematica
sage:
m.sage()# optional -
mathematica
x^2 + 5*y

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[Felix Lawrence]

   whuss at some point added something like this for both Mma and Maple,
   though very basic, as part of another ticket (symbolic sums?).  I
   can't remember where it is and am unfortunately having some internet
   issues :( but anyway I believe this code was merged into Sage at some
   point.
   - kcrisman
  Unless I'm *very* mistaken, doing this would be a non-trivial project,
  so I doubt someone would have done it for both Mathematica and Maple
  as part of another ticket. But of course I may be wrong.

 If I recall correctly, it was truly basic, and certainly likely to not
 work in many situations - but for a few very specific purposes it was
 better than nothing.   I think the following was the relevant bit.

I think there's some confusion here.  kcrisman seems to be talking
about allowing the Mathematica interface to parse mathematica output,
importing it to Sage.  Dave seems to be proposing writing something
that lets Sage run mathematica code natively, i.e. without calling
Mathematica.

Incidentally, I think whuss's patch implemented parsing for symbolic
variables and broke the existing functionality for parsing mma's
output (trac seems to be down at the moment so I can't check).  A
ticket is ready for review with a patch that improves the mathematica
output parser: it reimplements the old functionality, keeps whuss's
functionality and supports some new things.

The ticket is available here (when trac recovers):
http://trac.sagemath.org/sage_trac/ticket/8495

It doesn't support syntax such as \\, and it is rather cavalier (if
there's no known Sage equivalent to a mma function, it just converts
the function to lower case and hopes for the best!) but it might not
be a bad start for a Mathematica parser.

100% compatibility is an unrealistic goal (especially with things such
as mathlink) since mma's language seems to be a moving target, but it
should be possible to do the basics.

Cheers,
Felix

 - kcrisman

 sage: sage.calculus.calculus.symbolic_sum??
 snip
        #. Sage can currently only understand a subset of the output of
 Maxima, Maple and
           Mathematica, so even if the chosen backend can perform the
 summation the
           result might not be convertable into a Sage expression.
 snip
 sage: sage.interfaces.mathematica??
 snip

     def _sage_(self):
         r
         Try to convert a mathematica expression back to a Sage
 expression.

         This currently does not implement a parser for the Mathematica
 output language,
         therefore only very simple expressions will convert
 successfully.

         EXAMPLES::

             sage: m = mathematica('x^2 +
 5*y')                            # optional - mathematica
             sage:
 m.sage()                                                # optional -
 mathematica
             x^2 + 5*y

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

Re: [sage-devel] Re: A Mathematica parser for Sage.

[David Kirkby]

On 2 September 2010 04:01, Felix Lawrence fe...@physics.usyd.edu.au wrote:

 I think there's some confusion here.  kcrisman seems to be talking
 about allowing the Mathematica interface to parse mathematica output,
 importing it to Sage.  Dave seems to be proposing writing something
 that lets Sage run mathematica code natively, i.e. without calling
 Mathematica.


Yes, I expect there's some confusion too.

 Incidentally, I think whuss's patch implemented parsing for symbolic
 variables and broke the existing functionality for parsing mma's
 output (trac seems to be down at the moment so I can't check).  A
 ticket is ready for review with a patch that improves the mathematica
 output parser: it reimplements the old functionality, keeps whuss's
 functionality and supports some new things.

 The ticket is available here (when trac recovers):
 http://trac.sagemath.org/sage_trac/ticket/8495

 It doesn't support syntax such as \\, and it is rather cavalier (if
 there's no known Sage equivalent to a mma function, it just converts
 the function to lower case and hopes for the best!) but it might not
 be a bad start for a Mathematica parser.

 100% compatibility is an unrealistic goal (especially with things such
 as mathlink) since mma's language seems to be a moving target, but it
 should be possible to do the basics.

 Cheers,
 Felix

I'd agree the lanague is a bit of a moving target - 6.0, which was a
huge upgrade from 5.2,  introduced quite a few issues.

But actually using jmath

http://robotics.caltech.edu/~radford/jmath/

which has not been updated for 4 years seems to work quite well. It
gives one command line editing with readline - something I can never
understand why Wolfram Research did not implement. Perhaps they
consider 99% of people use the GUI.

jmath uses Mathlink, but it does not matter if the Mathematica
language changes, as the Mathlink protocol is only copying what one
types to the kernel and sends back what the kernel produces.

Wolfram Research could rename every single command - it would not matter.

What I was talking about would be more ambitious. I expect Richard
Fateman's Mathematica parser would be more prone to changes in the
language, as it is actually parsing the text.

100% comparability with what I am suggesting would never be possible,
as Sage only implements a subset (and to be fair a superset) of
Mathematica commands. But I believe it would be useful.

Dave

-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org

[sage-devel] Re: A Mathematica parser for Sage.

[rjf]

the mathematica syntax parser that I wrote appears to run inside
Maxima, so
you can, if you wish, feed such text to the mma-in-maxima system.

The intent in that project is mainly to take mma syntax for
expressions and map it into
maxima,  and not take the big step of having a more-or-less full
mathematica
evaluator in Maxima.

My original mockmma  (available free) also has a mathematica-style
evaluator using
matching and rules and such stuff,  though hardly complete.  There are
thousands of
procedures.  But the basic bones of the evaluator, defining functions
(really, rules),
is there.

So on the one hand it would be fairly trivial to get mathematica
syntax into Sage.

To get all mathematica semantics into Sage would be a pretty much
endless task,
unless WRI went out of business and made its code public. In which
case why bother.

And yes, the syntax has changed since Mma 3.0, but not so much that
the mockmma
parser could not be used as a basis for extension, in case some of
those extensions
turn out to be relevant somehow.



RJF




On Sep 1, 9:03 pm, David Kirkby david.kir...@onetel.net wrote:
 On 2 September 2010 04:01, Felix Lawrence fe...@physics.usyd.edu.au wrote:

  I think there's some confusion here.  kcrisman seems to be talking
  about allowing the Mathematica interface to parse mathematica output,
  importing it to Sage.  Dave seems to be proposing writing something
  that lets Sage run mathematica code natively, i.e. without calling
  Mathematica.

 Yes, I expect there's some confusion too.



  Incidentally, I think whuss's patch implemented parsing for symbolic
  variables and broke the existing functionality for parsing mma's
  output (trac seems to be down at the moment so I can't check).  A
  ticket is ready for review with a patch that improves the mathematica
  output parser: it reimplements the old functionality, keeps whuss's
  functionality and supports some new things.

  The ticket is available here (when trac recovers):
 http://trac.sagemath.org/sage_trac/ticket/8495

  It doesn't support syntax such as \\, and it is rather cavalier (if
  there's no known Sage equivalent to a mma function, it just converts
  the function to lower case and hopes for the best!) but it might not
  be a bad start for a Mathematica parser.

  100% compatibility is an unrealistic goal (especially with things such
  as mathlink) since mma's language seems to be a moving target, but it
  should be possible to do the basics.

  Cheers,
  Felix

 I'd agree the lanague is a bit of a moving target - 6.0, which was a
 huge upgrade from 5.2,  introduced quite a few issues.

 But actually using jmath

 http://robotics.caltech.edu/~radford/jmath/

 which has not been updated for 4 years seems to work quite well. It
 gives one command line editing with readline - something I can never
 understand why Wolfram Research did not implement. Perhaps they
 consider 99% of people use the GUI.


-- 
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to 
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org