On Wed, Nov 3, 2010 at 8:54 AM, Bill Hart <goodwillh...@googlemail.com> wrote:
>
>
> On 3 Nov, 14:46, William Stein <wst...@gmail.com> wrote:
>> On Wed, Nov 3, 2010 at 5:30 AM, Bill Hart <goodwillh...@googlemail.com>
>> wrote:
>> > Finally, testing each individual spkg (against its dependencies) on
>> > all supported platforms *before* having to download and build the
>> > whole of the latest Sage seems to me to be a logical first step. I'm
>> > not seeing even that happen at the moment. This again is a kind of
>> > modularity. If the new Pari doesn't even build on the Solaris, what is
>> > the point of spending a whole day building and testing the whole of
>> > Sage on Solaris? And if Pari doesn't even have a comprehensive test
>> > suite and a new stable release I'm not getting why we are even using
>> > it the way we are. We surely need to be much more sceptical about it
>> > and test the hell out of it before trying to put it into Sage. OK it's
>> > in now, but is it really worth doing it that way again in the future?
>>
>> Just for the record, for people reading this who might think Bill
>> speaks for the whole project,
>
> Wait, what!??? ...why on earth would anyone think that!!? Have I ever
> held some kind of executive capacity in the project?
I have no idea, but often people read some random posting of an idea
on a list, then contact me off list worried that there has been some
scary official statement that X will happen with Sage. It happens all
too often. It's not your fault...
> If we add Windows to the list of supported platforms eventually, will
> that get added to the list of platforms that will need to be tested on
> and fixed every other week when there is a new Sage release?
Yes.
> Did
> anyone do this with the Cygwin port to check for regressions/new
> doctest failures before 4.6 came out?
There still isn't a version on Cygwin that even *builds*
automatically, let alone works without further work. Tests still
fail, etc.
So this is different.
> At what point will the list of supported platforms just become too
> large to be managing simultaneous releases on all platforms
> simultaneously every other week?
There was a recent long discussion about supported platforms on
sage-devel. The suggestion was that we *define*
the set of supported platforms to be the ones where we do automated
testing (with buildbot, etc., and all tests pass),
and be sure to include platforms in that list that people funding sage
development care a lot about. I now think this is a
really good idea. Then the answer to your question is: "The list of
supported platforms is *exactly* the size that we
can clearly manage, since we are managing it."
> Another oft suggested strategy is to have a sage-stable and sage-
> devel. In some ways, purple-sage looks like it might become some
> people's sage-devel, I don't know.
That's a completely wrong way to think about what purple-sage is,
though I can see how the misconception could arise,
especially because initially I didn't know, and I thought it might be
that. But Purple sage is:
(1) a stripped down Sage, with a very small number of
modifications to the sage library so that this stripped down version
works, and
(2) a completely *new* library, 100% outside of the "sage"
namespace, of code.
As an example to make things more concrete, yesterday Fredrik
Johansson pointed me to a track ticket, and said "maybe you could
include this in psage", since he was viewing psage as "sage + any
patches william can think to apply to sage". I think that would be a
hellish nightmare to maintain... and that is not what psage is. The
code in psage is either completely new code, or code that might never
ever go into Sage, or if it does, it won't happen for a while. It's
by people doing research math, who don't want to worry about having
stable API's, 100% coverage, etc. Right now it's got code for
computing Maass forms, Siegel modular forms, Hilbert modular forms,
and L-functions of elliptic curves over function fields.
>> * Bill remarks his test suite for some project has about one line
>> devoted to testing for every line of actual code, and Sage should too.
>> I looked, and Sage probably has about 3 lines of code for every line
>> of testing, since there are 134,329 lines of input doctests, and
>> probably around 300,000-400,000 lines of actual code in the sage
>> library.
>
> The biggest problem I see here is each doctest only seems to do a
> single iteration of the function in question. I am sure there are
> exceptions to this. But I don't see how you hope to catch corner cases
> by testing fewer times than I have fingers and toes.
That's a matter of convention. To encourage people to write more
useful tests, here's one I wrote. Look at
sage/matrix/matrix_integer_dense_hnf.py at the function "def
sanity_checks(times=50, n=8, m=5, proof=True, stabilize=2,
check_using_magma = True)". It has doctests that do HNF using
several different systems on random input and compare the results.
It could be made to run longer than it does now though. Amusingly,
when I wrote that function, I immediately found a serious bug in
PARI... (which the pari devs immediately fixed, of course).
> Still, one line in four, if accurate, doesn't sound like a bad ratio.
> Writing test code in Python is easier than in some other languages, so
> I don't know what is a good ratio for python code.
Well the docstrings are mainly useful for users and people reading the
code. I really hope we also add tons of unit tests, especially when
the doctest coverage gets to 100%...
>> That said, the biggest obstruction to Sage on Windows has
>> always been lack of focused work on the port by people who knew what
>> they're doing. Sage only got ported to Solaris because David Kirkby
>> put in an _enormous_ amount of effort making that happen. Something
>> similar is needed for windows. Blair Sutton started in that
>> direction, but ran out of steam after about 4 months; David Kirkby was
>> much, much more longterm persistent.
>
> There is almost infinitely more work in a native Windows port than a
> Solaris port. Solaris is similar enough to other unixes to make a port
> doable by one person. I think it is completely unfair to say Blair
> failed because David was more persistent. Blair failed because it is a
> much, much harder job, and he got tired.
He got tired after about 1/10th the amount of time elapsed.
I agree that the "native windows port" will be harder though, because it will
involve fundamentally rewriting how some of Sage works. E.g.,
switching entirely to
libGAP (a C interface to GAP) would be a good step.
> One person will never finish the job on Windows. It needs a team of
> skilled individuals. But a smaller bite-sized useful chunk of Sage at
> a time is far more likely to be ported than the whole thing at once.
I'm sure one person could do it. I think I could do it. Or Mike
Hansen. Or Carl Witty. But I'm probably not going to, since I have
more important things to do.
I might come back to it in a few years though, if nobody else has done
it first.
>> * Bill suggests that the development of the core of Sage should be
>> broken up into 20 or so special groups, etc. I disagree with this;
>> personally I think specialized groups in different research areas
>> should instead develop packages *on top* of Sage, while *everybody*
>> contributes to making the core of Sage much more bugfree and stable
>> than it already is. But this shouldn't happen until we iron out the
>> bugs about how to best develop packages on top of Sage see remarks
>> above).
>
> I don't recall suggesting the "core of Sage" should be broken into 20
> pieces. That would be totally pointless.
Cool. I'm glad I misunderstood.
> Or are you suggesting that everything that is currently in that 300mb
> tarball should be considered "core Sage". I think that is a completely
> and utterly unsustainable model.
Yes, that's what I mean by "core sage".
> The core should be *much* smaller. Then communities should be building
> on top of that smaller core with packages relevant to their area.
> That's what I mean by modularisation.
>
> I cannot think of a single other project out there that has a single
> monolithic 300mb source tarball released every other week.
Sage is unfortunately not released every other week.
Sage isn't about copying other projects. Just because you don't know
of other projects that are like Sage, doesn't mean Sage must fail or
is "unsustainable".
> Anyhow, I
> don't think we disagree on the general principle that modularity is a
> good thing.
Yes. I also agree that growing the "core of Sage" whatever that
means too much is not the right next direction to go in. The next
direction should be to have a library version of Sage, root out bugs,
make the core Sage library even higher quality and harder to get code
into, make it really, really easy for people to build their own
packages on Sage (and host them on Pypi says), etc. The python
community has done tons for us already, and we should reap the
benefits of all that.
>> Michael even goes on to remark that "FLINT is a bloated pig...". :-)
>Yeah but it runs really fast, and he was right. Why do you think we
> rewrote it from scratch.
I had to include a funny mabshoff quote. I for one definitely
appreciate FLINT!
>> We switched to Pari SVN for several reasons including being told by
>> Karim that the time was right and that a stable release based on it
>> was "coming soon".
>
> Great! Git version of flint 1.6 is available here:
> http://selmer.warwick.ac.uk/FLINT.git
> branch test_code.
> The time is right and a stable release based on it is "coming soon".
OK, let me add that in addition to be told the time is right, Robert
Bradshaw, John Cremona, and I were all in the same place at Sage Days
looking for a challenging 3-day project. And John really wanted the
new PARI since it fixed some bugs. Also, he offered us a free
dinner...
-- William
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
