On Wed, Dec 1, 2010 at 2:36 PM, William Stein <wst...@gmail.com> wrote: > On Wed, Dec 1, 2010 at 2:25 PM, David Kirkby <david.kir...@onetel.net> wrote: >>> I do think it would be good to start using nosetest >>> (http://somethingaboutorange.com/mrl/projects/nose/0.11.2/) to >>> automatically run all functions that start with "test_" in all files, >> >> I suggested 'nose' was added a long time ago >> >> http://groups.google.com/group/sage-devel/browse_thread/thread/928632557a8a041c/f8bc25a249ea4483?hl=en&lnk=gst&q=nose#f8bc25a249ea4483 >> >> the only person to reply (Robert Bradshaw) disagreed.
I think there's a distinction between an spkg that people might find useful to use with Sage, and an spkg that's actually used in in Sage. For the former, if easy_install "just works," than it's not worth us creating and maintaining a separate spkg, but for the latter, we should ship it. The fact that an upstream package use nose in its tests did not seem like enough of a justification to create a whole new spkg, but if we want to write Sage tests with nose than I have no objection. I certainly think that there's a diminishing return on doctests once you reach a certain point (which we're probably not at yet). >>> But asking referees to check claimed examples -- >>> that makes sense! In particular, if I referee some code, and it >>> turns out somebody finds that the examples were just wrong, then I as >>> the referee will be pretty embarrassed. >> >> Yes, but using examples like >> >> sage: taylor(gamma(1/3+x),x,0,3) >> >> makes it almost impossible for a referee to check it, as the output is huge. What would make a better test in this case would be taking the resulting power series, perhaps to a higher degree of precision, and evaluating at 0.1, 0.5, and showing that the result is close to gamma(1/3 + 0.1), gamma(1/3 + 0.5). Or perhaps verifying that the 3rd coefficient is equal to the 3rd derivative / 6. > I totally agree, and I think that's a very valid criticism for you to > make as a referee. > > But let's not make a new policy out of this. +1. As more time is spent reading the code and tests rather than applying patches, we should be more critical of good vs. bad tests. This also goes with the ideal of making it really easy to edit a patch, perhaps even online. (Imagine if you could run some code and press a button to add that doctest to the library, pending refereeing of course...) >> In any case, you stated only a week or so ago that Magma 2.13 is now >> installed on sage.math >> >> http://groups.google.com/group/sage-devel/msg/8e473e24b0e48772?hl=en > > That is a post from 2006?!? > >> It's a shame the license of Wolfram Alpha does not allow for testing >> software like Sage. (This was debated some time ago on sage-devel). >> Otherwise that would give a nice easy way to verify *some* results. >> >> "is 100001 prime" >> >> http://www.wolframalpha.com/input/?i=is+100001+prime > > I'm not sure what you're talking about exactly at this point. > Referees can use wolfram alpha if they want to independently check > stuff... Do you mean adding doctests that call wolframalpha? That > would be weird. > >> I appreciate in many cases it's not going to be possible to verify by >> other means. One has to be extra careful about the code then. On the topic of verifying tests, I think internal consistency checks are much better, both pedagogically and for verifiability, than external checks against other (perhaps inaccessible) systems. For example, the statement above that checks a power series against its definition and properties, or (since you brought up the idea of factorial) factorial(10) == prod([1..10]), or taking the derivative to verify an integral. Especially in more advanced math there are so many wonderful connections, both theorems and conjectures, that can be verified with a good test. For example, computing all the BSD invariants of an elliptic curve and verifying that the BSD formula holds is a strong indicator that the invariants were computed correctly via their various algorithms. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
