Re: [sage-devel] Regression testing

2010-10-29 Thread Robert Bradshaw
On Tue, Oct 26, 2010 at 12:57 AM, Dr. David Kirkby david.kir...@onetel.net wrote: On 10/25/10 07:06 PM, Robert Bradshaw wrote: On Mon, Oct 25, 2010 at 8:19 AM, David Kirkbydavid.kir...@onetel.net  wrote: On 21 October 2010 01:33, David Roer...@math.harvard.edu  wrote: There are a number of

Re: [sage-devel] Regression testing

2010-10-29 Thread Dr. David Kirkby
On 10/29/10 10:10 PM, Robert Bradshaw wrote: On Tue, Oct 26, 2010 at 12:57 AM, Dr. David Kirkby 'cksum' is a 32-bit checksum. Actually, if used all three sections of the output 1) Checksum 2) Length 3) Filename I feel that should be sufficiently relieable. The probability of a test having

Re: [sage-devel] Regression testing

2010-10-29 Thread Robert Bradshaw
On Fri, Oct 29, 2010 at 4:29 PM, Dr. David Kirkby david.kir...@onetel.net wrote: On 10/29/10 10:10 PM, Robert Bradshaw wrote: On Tue, Oct 26, 2010 at 12:57 AM, Dr. David Kirkby 'cksum' is a 32-bit checksum. Actually, if used all three sections of the output 1) Checksum 2) Length 3)

Re: [sage-devel] Regression testing

2010-10-26 Thread Dr. David Kirkby
On 10/25/10 07:06 PM, Robert Bradshaw wrote: On Mon, Oct 25, 2010 at 8:19 AM, David Kirkbydavid.kir...@onetel.net wrote: On 21 October 2010 01:33, David Roer...@math.harvard.edu wrote: There are a number of tickets in trac about performance regressions in Sage. I'm sure there are far more

Re: [sage-devel] Regression testing

2010-10-25 Thread David Kirkby
On 21 October 2010 01:33, David Roe r...@math.harvard.edu wrote: There are a number of tickets in trac about performance regressions in Sage.  I'm sure there are far more performance regressions which we don't know about because nobody noticed. I agree, and I've seen some comments from

Re: [sage-devel] Regression testing

2010-10-25 Thread Robert Bradshaw
On Mon, Oct 25, 2010 at 8:19 AM, David Kirkby david.kir...@onetel.net wrote: On 21 October 2010 01:33, David Roe r...@math.harvard.edu wrote: There are a number of tickets in trac about performance regressions in Sage.  I'm sure there are far more performance regressions which we don't know

Re: [sage-devel] Regression testing

2010-10-25 Thread William Stein
Also, I was talking to Craig Citro about this and he had the interesting idea of creating some kind of a test object which would be saved and then could be run into future versions of Sage and re-run in. The idea of saving the tests that are run, and then running the exact same tests (rather

Re: [sage-devel] Regression testing

2010-10-25 Thread Jeroen Demeyer
On 2010-10-25 20:06, Robert Bradshaw wrote: To be very useful, I think we need to be more granular than having per-file tests. Just think about the number of files that get touched, even a little bit, each release... Full doctest blocks should be independent (though of course when looking at a

Re: [sage-devel] Regression testing

2010-10-25 Thread Mitesh Patel
On 10/25/2010 01:54 PM, William Stein wrote: Also, I was talking to Craig Citro about this and he had the interesting idea of creating some kind of a test object which would be saved and then could be run into future versions of Sage and re-run in. The idea of saving the tests that are run,

Re: [sage-devel] Regression testing

2010-10-25 Thread William Stein
On Mon, Oct 25, 2010 at 4:23 PM, Mitesh Patel qed...@gmail.com wrote: On 10/25/2010 01:54 PM, William Stein wrote: Also, I was talking to Craig Citro about this and he had the interesting idea of creating some kind of a test object which would be saved and then could be run into future

Re: [sage-devel] Regression testing

2010-10-25 Thread Robert Bradshaw
On Mon, Oct 25, 2010 at 11:54 AM, William Stein wst...@gmail.com wrote: Also, I was talking to Craig Citro about this and he had the interesting idea of creating some kind of a test object which would be saved and then could be run into future versions of Sage and re-run in. The idea of saving

Re: [sage-devel] Regression testing

2010-10-25 Thread David Roe
I think if you set both number and repeat to 1 in sage.misc.sage_timeit, it will only run once (though I could be wrong). We should think about a way to automate uploading of timing data if someone doesn't have MongoDB installed. For example, we could have the test script which ran doctests have

Re: [sage-devel] Regression testing

2010-10-25 Thread Robert Bradshaw
On Mon, Oct 25, 2010 at 8:39 PM, David Roe r...@math.harvard.edu wrote: I think if you set both number and repeat to 1 in sage.misc.sage_timeit, it will only run once (though I could be wrong). Yes, though it'd probably be both cheap and valuable to run fast commands more than once (but less

[sage-devel] Regression testing

2010-10-20 Thread David Roe
There are a number of tickets in trac about performance regressions in Sage. I'm sure there are far more performance regressions which we don't know about because nobody noticed. As someone writing library code, it's generally not obvious that one is about to introduce a performance regression

Re: [sage-devel] Regression testing

2010-10-20 Thread Robert Bradshaw
On Wed, Oct 20, 2010 at 5:33 PM, David Roe r...@math.harvard.edu wrote: There are a number of tickets in trac about performance regressions in Sage.  I'm sure there are far more performance regressions which we don't know about because nobody noticed. As someone writing library code, it's