Just to put this 5000 lines in context, here are counts of the number
of occurrences of mpz_add in all the Cython and Python files in Sage:
/sage/modular/modsym/heilbronn.pyx:1
/sage/ext/multi_modular.pyx:2
/sage/combinat/expnums.pyx:1
/sage/modules/vector_integer_dense.pyx:2
/sage/algebras/quatalg/quaternion_algebra_element.pyx:17
/sage/matrix/matrix_cyclo_dense.pyx:2
/sage/matrix/matrix_rational_dense.pyx:1
/sage/matrix/matrix_integer_2x2.pyx:9
/sage/matrix/matrix_integer_dense.pyx:10
/sage/rings/integer.pyx:4
/sage/rings/integer_ring.pyx:3
/sage/rings/integer_mod.pyx:6
/sage/rings/polynomial/real_roots.pyx:9
/sage/rings/padics/padic_fixed_mod_element.pyx:2
/sage/rings/padics/padic_capped_absolute_element.pyx:1
/sage/rings/padics/padic_ZZ_pX_CR_element.pyx:2
/sage/rings/padics/padic_base_generic_element.pyx:3
/sage/rings/padics/padic_ZZ_pX_CA_element.pyx:2
/sage/rings/padics/padic_capped_relative_element.pyx:6
/sage/rings/padics/pow_computer_ext.pyx:3
/sage/rings/real_mpfi.pyx:1
/sage/rings/real_mpfr.pyx:3
/sage/rings/number_field/number_field_element_quadratic.pyx:15
That's in a few hundred thousand lines of code. Notice, no lines in
python files whatsoever, only in cython (pyx) files.
Bill.
On 17 Aug, 03:42, Bill Hart <goodwillh...@googlemail.com> wrote:
> On 17 Aug, 02:28, rjf <fate...@gmail.com> wrote:
>
> > On Aug 16, 3:30 pm, Bill Hart <goodwillh...@googlemail.com> wrote:
> > ..snip...
>
> > > > {RJF] (mpfr::with-temps (/(- (* (- (* 2 i)1) x t1) (* (- i 1) t0))
> > > > i)))
>
> > > That's a very interesting example. Are you saying that Lisp
> > > automatically divines which MPFR functions to assign to those
> > > operators?
>
> > no, Lisp doesn't do that automatically. The macro program I wrote,
> > with-temps, in the mpfr package
> > does that (uh, automatically?).
> > It assumes that the manifest constants like 1, 2, are not mpfr objects
> > and so compiles
> > in conversions. It assumes that the variables represent mpfr
> > variables.
>
> Right, but you could write a similar macro in python. {Scratches
> head}.
>
> > > How would it cope with the new proposed addadd, subadd and sumdiff
> > > functions of MPIR?
>
> > I don't know what those functions do, but guessing that mpir_addadd
> > (target,a,b,c)
> > does target:=a+b+c,
>
> yep
>
> > I could change the with-temps macro to look for (+ a b c ...) and use
> > addadd to reduce the problem,
> > I suppose.
>
> > > I'm also curious as to why the need for temporaries in that macro. You
> > > can evaluate that expression with a mul, mul_ui, submul_ui and div_ui
> > > (or perhaps MPFR doesn't provide all those).
>
> > As I recall MPFR is all "state based" not "functional" in style (in
> > terms of its interface),
> > and I think GMP and therefore MPIR are similar.
> > That is, you cannot take
> > a:=b+c*d and convert (or compile it) into
> > set(a, plus(b,times(c,d))).
>
> OK, so a functional language like lisp is useless for interfacing to
> non-functional languages like C. Good point.
>
>
>
> > You have to do this:
> > allocate temp t1 (sometime, perhaps much earlier...)
>
> > times(t1,c,d) // t1:=c*d
> > plus(a,t1,b) // a := t1+b
>
> > there is a combined addmul, but you get the idea, I think. You need
> > some temps.
> > This kind of setup is equivalent to register allocation for compiling,
> > but you
> > get to allocate as many temp registers as you want, since they don't
> > correspond
> > to hardware registers anyway.
>
> > > > a more conventional version of this might be the "infix" version that
> > > > I thought
> > > > python / Sage programmers would use.
>
> > > > ((2*i-1)*x*t1) - (i-1)*t0))/i
>
> > > Hmm, my knowledge of python is pretty scant, but that's what I'd type.
>
> > OK, so why does Sage have 5000 lines with mpz_.... ?
>
> Because we either don't have your macro, which you admit is not part
> of Lisp, but something you wrote, or many, many people coding away on
> Sage don't know of the existence of such a thing if it exists.
>
> Or, just maybe, there are instances in the hundreds of thousands of
> lines of Sage python code where your macro wouldn't do the optimal
> thing, so someone wrote their own specialised code which does the
> correct thing.
>
> > If I wrote a whole suite of programs to use mpz_ or mpfr_ or
> > interval_ or .... programs
> > I would hardly ever use those programs explicitly. I would use use
> > ordinary operations
> > like +, *, etc. and have them magically macro-expanded to calls to
> > those library functions.
>
> So what do you suppose the other few hundred thousand lines of Sage
> python code do?
>
> I'd be really surprised if the number of lines of code in the examples
> you posted on your website is even remotely similar to the number of
> lines of code in Sage.
>
> When you have a really efficient computational algebra package written
> in Lisp with the depth of functionality of Sage posted on your
> website, then we'll do a comparison of the number of lines of
> mpfr_this and mpz_that.
>
>
>
> > > Does Lisp automatically deal with the case where the (2*i - 1)
> > > overflows?
>
> > there are two places for overflow, 2*i is the first one.
> > mpfr_mul_ui(target1,i,2) would not overflow, since the target is a
> > bigfloat. [oh, by the way, I think there
> > is also a 4th argument which is the rounding mode, usually
> > *nearest*],.
>
> That's terribly inefficient. It's much more efficient to multiply the
> 2*i in a machine word than to pull out an mpfr for the purpose.
> Possibly where some of those 5000 lines come from.
>
>
>
> > The next place is
> > mpfr_sub_ui(target2,target1,1).
> > That won't overflow either.
>
> > The chance that you are interested in evaluating a Legendre polynomial
> > of degree 2^32 or so is pretty remote, I think.
> > But this particular expression would be OK.
>
> > Of course Sage has to deal with that. But I am not seeing
>
> > > any reason why you couldn't code up a macro, similar to the one your
> > > Lisp gives you which allows evaluation of expressions such as these in
> > > an optimal manner.
>
> > I agree with you entirely, which is why I asked about the 5000 lines
> > with mpz.
> > One possible answer is, "#...@#$%$, Python doesn't have macros."
>
> But I bet cython does. It also depends what you want to call a macro.
>
> I'm wondering why you used postfix instead of infix? Could it be due
> to some limitation of lisp which makes it difficult to deal with infix
> expressions. Let me hazard a guess. You *could* write an infix version
> in Lisp, but it would be *much* more complicated. Whereas Python has
> lots of nice expression parsing libraries which do this trivially for
> you.
>
>
>
> > > Does Lisp also automatically do expression simplification for you so
> > > that if you typed i*(x+1) - i*x that it would just evaluate i, not all
> > > the rest of the gunk which cancels?
>
> > Such an optimization would certainly not be automatic. Maxima might do
> > that.
> > assuming that x and i are exact quantities.
>
> > In general, the answer is not, in fact, i.
> > Consider if x is a floating point number 1e30 or so. then x+1 is
> > equal to x.
> > Then the expression is not i, but 0.
>
> > > > [RJF] But if I understand you, the Sage programmer writing directly in
> > > > Python has a much worse option,
> > > > involving something like the 7 mpfr_add, mpfr_mul, ... functions
> > > > calls,
> > > > interspersed with establishing some temporaries for intermediate
> > > > results.
> > > > Maybe some of the 5,000 lines are really a failure of python to
> > > > provide a convenient
> > > > level upon which to write code that does some "other" kind of
> > > > arithmetic?
>
> > > That's pretty waffly. What do you mean by "other" kind of arithmetic.
>
> > By "other" I mean arithmetic that is different from what is already
> > built in.
>
> By and large Python is much easier to use than most other languages.
> So even if it were true that python didn't have a convenient level
> upon which to implement such a thing, the enormous number of other
> conveniences over a language like Lisp far outweigh this disadvantage.
> I couldn't help noticing the enormous number of parentheses in your
> postfix expression as compared to the much nicer python infix. I hope
> you have a good magnifying glass or a decent editor to see which ones
> match up with which.
>
> But your premise is flawed. Sage does just fine in terms of offering
> oodles of "other" arithmetics. That's kind of what Sage is. And it was
> written in Python.
>
> Yes, python doesn't have macros in the sense that you mean them. But
> Sage itself does have a preprocessor, which could be used in theory to
> provide the sorts of macros you are talking about. It has not been
> done because of a (valid) design decision to keep the syntax as close
> to python as possible (the Sage preprocessor does almost nothing).
> Having said that, making mpz's and the like, easy to use, is not
> really that hard without macros. And at the level William was talking
> about, you can use Cython anyhow, so what's the problem.
>
>
>
> > > Sage has a very powerful collection of "arithmetics". I mean you can
> > > multiply matrices, polynomials, group elements, all sorts of things,
> > > with a * operator.
>
> > OK, how hard is it to add another kind of arithmetic? (say, for
> > argument's sake
> > complex numbers are not built in. How hard would it be to add them?
> > Would you have
> > to write complex_add(a,b), complex_mul(c,d) etc etc?? I hope not.
> > But then why would you have to write mpz_..() all over the place?
>
> But the vast majority of things you are going to want to do are going
> to not be arithmetic things. E.g. with polynomials over Z, you have +,
> *, -, ^ and pretty much I've run out of symbols. But there's 139
> functions in FLINT of the form fmpz_poly_blah(). A very small number
> of them do +, *, -, ^.
>
> Here for example is the code which handles multiplication of
> polynomials, so that people can just type f*g:
>
> cpdef RingElement _mul_(self, RingElement right):
> r"""
> Returns self multiplied by right.
>
> EXAMPLES::
>
> sage: R.<x> = PolynomialRing(ZZ)
> sage: (x - 2)*(x^2 - 8*x + 16)
> x^3 - 10*x^2 + 32*x - 32
> """
> cdef Polynomial_integer_dense_flint x = self._new()
> _sig_on
> fmpz_poly_mul(x.__poly, self.__poly,
> (<Polynomial_integer_dense_flint>right).__poly)
> _sig_off
> return x
>
> It's written in cython of course, not python, as most of the stuff
> which uses C libraries should be. Sage handles the rest.
>
>
>
> > > The original discussion was about in_place operators, which Sage does
> > > allow. The example William gave was:
>
> > > sage: n = 15
> > > sage: n._iadd_(5)
> > > 20
> > > sage: n
> > > 20
>
> > > I don't see any lines of mpfr_add's there, do you?
>
> > No, Sage might have in_place operators, and MPIR might have in_place
> > operators, but that
> > doesn't mean that Sage can use the in_place operators in MPIR with the
> > above syntax.
>
> Of course it can. Here is one of your 5000 lines of mpz_add's:
>
> cpdef ModuleElement _iadd_(self, ModuleElement right):
> # self and right are guaranteed to be Integers, self safe to
> mutate
> mpz_add(self.value, self.value, (<Integer>right).value)
> return self
>
> It allows the user to do an inplace add for mpz's, using the syntax
> William gave. You can check it for yourself in the file sage/rings/
> integer.pyx
>
>
>
> > That might be another reason for 5000 mpz_ lines.
>
> Nope.
>
>
>
> > > As I said, I am very surprised to learn Lisp allows you to do such
> > > things without any mpfr_add's and mpfr_mul's, if that is what you
> > > intended.
>
> > Not only does Lisp allow it (if one has the appropriate macro code),
>
> > a sample package has been posted on my web page for a few years
> > along with papers describing it.
>
> You get to write papers on this stuff! I should change fields.
>
>
>
> > > Otherwise I don't have a clue what you are talking about.
> > > What "convenient level" does Lisp provide that python does not, and
> > > what ""other" kind or arithmetic" are you referring to?
>
> > I experimented with other kinds of generic arithmetic
> > supported in the same way, including, as I recall, exact rationals
> > with affine or projective
> > infinities, real intervals, complex numbers using (r, theta) notation,
> > polynomials in one
> > or more variables, quad-double floats, a kind of taylor-series
> > approximation to functions
> > that allows for automatic differentiation, etc. Programs like newton
> > iteration or
> > gaussian quadrature can be run in any of several different packages
> > without textual change,
> > with the macro expansion taking care of the details.
>
> Look, I agree that it would be a nice language feature. Even the C++
> preprocessor doesn't have enough macro expansion for my liking, and C+
> + is one of the languages used in Sage. This is definitely a nice
> feature in Lisp. But it isn't going to be much of a draw card given
> the other decisively off putting features of Lisp like difficult to
> use packages and modules and the lack of a decent compiler and the
> fact that there is very little community around it.
>
> It would be a much more productive use of time to implement macros in
> Python. Of course that is being done. There's a Google summer of code
> project for it.
>
>
>
> > If this kind of thing is not possible with Python, that is, in my
> > view, unfortunate.
>
> Well, macros in the sense that you mean it are not. The other stuff
> is, along with a hell of a lot that lisp can't easily do, due to the
> overwhelming number of freely available modules you can just download
> for python.
>
> > There are several papers with explanations, but perhaps the simplest
> > is
> > related to setup for QD (quad-double) arithmetic...
> > seehttp://www.cs.berkeley.edu/~fateman/papers/qd.pdf
>
> > which mentions MPFR towards the end.
>
> >http://www.cs.berkeley.edu/~fateman/papers/qd.pdf
>
> > there's a paper about automatic differentiation with AD in the title,
>
> > and one about quadraturehttp://www.cs.berkeley.edu/~fateman/papers/quad.pdf
>
> Does your citation rate go up when you post links to your papers all
> over newsgroups by the way?
>
> Anyhow, thanks for the link, but I'm not presently interested in Lisp
> or other language design features. I'll store it in my memory banks
> and come back to it later if I have a need for it.
>
> Bill.
--~--~---------~--~----~------------~-------~--~----~
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel-unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URLs: http://www.sagemath.org
-~----------~----~----~----~------~----~------~--~---
