There has been some previous discussion about this on sage-devel, I can't find exactly the thread I remember but here's a somewhat related one:
http://groups.google.com/group/sage-devel/browse_thread/thread/b91c51672ae0f475/ Personally I think it makes sense to put the most effort into getting sage to 100% coverage. Whenever possible when writing doctests the results should be checked. Perhaps in some test blocks it could be remarked that the result has been verified by another system. -Marshall On Mar 3, 1:54 am, Joshua Herman <zitterbeweg...@gmail.com> wrote: > Is there a mathematica test suite we could adapt or a standardized set > of tests we could use? Maybe we could take the 100 most often used > functions and make a test suite? > > ---- LOOK ITS A SIGNATURE CLICK IF YOU > DARE---http://www.google.com/profiles/zitterbewegung > > On Wed, Mar 3, 2010 at 12:04 AM, David Kirkby <david.kir...@onetel.net> wrote: > > Has anyone ever considered randomised testing of Sage against Mathematica? > > > As long as the result is either > > > a) True or False > > b) An integer > > > then comparison should be very easy. As a dead simple example, > > > 1) Generate a large random number n. > > 2) Use is_prime(n) in Sage to determine if n is prime or composite. > > 3) Use PrimeQ[n] in Mathematica to see if n is prime or composite. > > 4) If Sage and Mathematica disagree, write it to a log file. > > > Something a bit more complex. > > > 1) Generating random equation f(x) - something that one could integrate. > > 2) Generate generate random upper and lower limits, 'a' and 'b' > > 3) Perform a numerical integration of f(x) between between 'a' and 'b' in > > Sage > > 4) Perform a numerical integration of f(x) between between 'a' and 'b' > > in Mathematica > > 5) Compare the outputs of the Sage and Mathematica > > > A floating point number, would be more difficult to compare, as one > > would need to consider what is a reasonable level of difference. > > > Comparing symbolic results directly would be a much more difficult > > task, and probably impossible without a huge effort, since you can > > often write an equation in several different ways which are equal, but > > a computer program could not easily be programmed to determine if they > > are equal. > > > One could potentially let a computer crunch away all the time, looking > > for differences. Then when they are found, a human would had to > > investigate why the difference occurs. > > > One could then add a trac item for "Mathematica bugs" There was once a > > push for a public list of Mathematica bugs. I got involved a bit with > > that, but it died a death and I became more interested in Sage. > > > Some of you may know of Vladimir Bondarenko, who is a strange > > character who regularly used to publish Mathematica and Maple bugs he > > had found. In some discussions I've had with him, he was of the > > opinion that Wolfram Research took bug reports more seriously than > > Maplesoft. I've never worked out what technique he uses, but I believe > > is doing some randomised testing, though it is more sophisticated that > > what I'm suggesting above. > > > There must be a big range of problem types where this is practical - > > and a much larger range where it is not. > > > You could at the same also compare the time taken to execute the > > operation to find areas where Sage is much faster or slower than > > Mathematica. > > > Dave > > > -- > > To post to this group, send an email to sage-devel@googlegroups.com > > To unsubscribe from this group, send an email to > > sage-devel+unsubscr...@googlegroups.com > > For more options, visit this group > > athttp://groups.google.com/group/sage-devel > > URL:http://www.sagemath.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
