Re: [sage-devel] the attack of the (really) long doctests
2009/12/14 Alex Ghitza aghi...@gmail.com: OK, the first patch, dealing with the Bernoulli numbers tests in arith.py, is up at http://trac.sagemath.org/sage_trac/ticket/7678 I have made some comments there, and would welcome a second opinion! John -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] the attack of the (really) long doctests
On Dec 14, 2009, at 7:03 AM, John Cremona wrote: 2009/12/14 Alex Ghitza aghi...@gmail.com: OK, the first patch, dealing with the Bernoulli numbers tests in arith.py, is up at http://trac.sagemath.org/sage_trac/ticket/7678 I have made some comments there, and would welcome a second opinion! Looks good to me too. - Robert -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] the attack of the (really) long doctests
This all sounds very sensible. Another related point is that some individual files are very very long. For example, one of the 10 files in your list is sage/schemes/elliptic_curves/ell_rational_field.py which is 7514 lines long, more than double the next longest in that directory. (I have written some of the code in that file so am of course partly to blame here!). A check of the length of individual source code files revealed some other bad cases: 3165 rings/polynomial/multi_polynomial_ideal.py 3342 server/notebook/interact.py 3346 crypto/mq/sr.py 3716 schemes/elliptic_curves/ell_curve_isogeny.py 3734 modular/modsym/ambient.py 4194 server/notebook/worksheet.py 4688 modular/abvar/abvar.py 6158 combinat/words/word.py 7441 rings/number_field/number_field.py 7514 schemes/elliptic_curves/ell_rational_field.py Some of these could (should?) probably be split up a bit. John 2009/12/13 Alex Ghitza aghi...@gmail.com: Hi, The developer guide says that any single doctest shouldn't take more than around 30 seconds. This is of course highly machine-dependent, but I assume we can take sage.math as a reference point. There are a handful of files in the Sage library that account for a large amount of the time necessary to run long doctests: sage -t -long devel/sage/sage/graphs/graph.py [152.2 s] sage -t -long devel/sage/sage/rings/arith.py [165.8 s] sage -t -long devel/sage/sage/plot/plot.py [167.3 s] sage -t -long devel/sage/sage/schemes/elliptic_curves/sha_tate.py [186.4 s] sage -t -long devel/sage/sage/groups/perm_gps/partn_ref/refinement_graphs.pyx [213.6 s] sage -t -long devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py [224.7 s] sage -t -long devel/sage/sage/combinat/root_system/weyl_characters.py [255.1 s] sage -t -long devel/sage/sage/groups/matrix_gps/matrix_group.py [91.4 s] sage -t -long devel/sage/sage/graphs/graph_generators.py [111.7 s] sage -t -long devel/sage/sage/groups/perm_gps/partn_ref/refinement_binary.pyx [291.2 s] Together, these 10 files take about 26% of all doctesting time. In some of them, this is due to just one or two doctests that take a very long time (definitely more than 30 seconds). Take for instance rings/arith.py: -- | Sage Version 4.3.rc0, Release Date: 2009-12-10 | | Type notebook() for the GUI, and license() for information. | -- ** * * * Warning: this is a prerelease version, and it may be unstable. * * * ** sage: algs = ['gap', 'gp', 'pari', 'bernmm'] sage: time vals = [[bernoulli(i, algorithm = j) for j in algs] for i in range(2, 2255)] CPU times: user 24.40 s, sys: 1.13 s, total: 25.53 s Wall time: 41.37 s sage: algs = ['gp', 'pari', 'bernmm'] sage: time vals = [[bernoulli(i, algorithm = j) for j in algs] for i in range(2256, 5000)] CPU times: user 96.18 s, sys: 1.13 s, total: 97.31 s Wall time: 109.31 s Note that most of the this time is taken by the bernmm algorithm: sage: time vals = [bernoulli(i, algorithm = 'bernmm') for i in range(2256, 5000)] CPU times: user 87.33 s, sys: 0.00 s, total: 87.33 s Wall time: 87.33 s I understand that we want to test bernmm thoroughly. But can we do something less time-consuming? For instance, instead of checking it on range(2, 5000), can we pick 500 random integers in that range and check it there? Here's what that would look like on sage.math: sage: test_list = [ZZ.random_element(2, 5000) for _ in range(500)] sage: time vals = [bernoulli(i, algorithm = 'bernmm') for i in test_list] CPU times: user 9.38 s, sys: 0.00 s, total: 9.38 s Wall time: 9.37 s Another example is the randomized testing in groups/perm_gps/partn_ref/refinement_binary.pyx: sage: import sage.groups.perm_gps.partn_ref.refinement_binary sage: x = walltime() sage: set_random_seed(int(str(x-int(x))[2:])) sage: time sage.groups.perm_gps.partn_ref.refinement_binary.random_tests(50, 200, 10) All passed: 2000 random tests on 100 codes. CPU times: user 206.11 s, sys: 0.00 s, total: 206.11 s Wall time: 206.11 s Would it be ok to change this so it takes a little bit less time? There are other examples out there. I'm not advocating blindly shortening very long doctests -- some of them might be there for a very good reason. However, if it's possible to reduce at least some of them, that would help a lot with testing on machines that don't have the kind of resources that sage.math has. In the same vein, +1 to
Re: [sage-devel] the attack of the (really) long doctests
On Sun, Dec 13, 2009 at 3:00 AM, John Cremona john.crem...@gmail.com wrote: This all sounds very sensible. Another related point is that some individual files are very very long. For example, one of the 10 files in your list is sage/schemes/elliptic_curves/ell_rational_field.py which is 7514 lines long, more than double the next longest in that directory. (I have written some of the code in that file so am of course partly to blame here!). A check of the length of individual source code files revealed some other bad cases: 3165 rings/polynomial/multi_polynomial_ideal.py 3342 server/notebook/interact.py 3346 crypto/mq/sr.py 3716 schemes/elliptic_curves/ell_curve_isogeny.py 3734 modular/modsym/ambient.py 4194 server/notebook/worksheet.py 4688 modular/abvar/abvar.py 6158 combinat/words/word.py 7441 rings/number_field/number_field.py 7514 schemes/elliptic_curves/ell_rational_field.py Some of these could (should?) probably be split up a bit. +1 to shortening/marking #long those doctests Alex mentioned. Also, trac 6616 splits up ell_rational_field.py by removing all the Heegner point code. William -- William Stein Associate Professor of Mathematics University of Washington http://wstein.org -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] the attack of the (really) long doctests
2009/12/13 William Stein wst...@gmail.com: 7514 schemes/elliptic_curves/ell_rational_field.py Some of these could (should?) probably be split up a bit. +1 to shortening/marking #long those doctests Alex mentioned. Also, trac 6616 splits up ell_rational_field.py by removing all the Heegner point code. I take the hint -- I'll look at that one! John William -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] the attack of the (really) long doctests
On Sun, Dec 13, 2009 at 11:01:14AM -0800, William Stein wrote: +1 to shortening/marking #long those doctests Alex mentioned. OK, the first patch, dealing with the Bernoulli numbers tests in arith.py, is up at http://trac.sagemath.org/sage_trac/ticket/7678 I will do groups/perm_gps/partn_ref/ next. It's taking a bit longer because I think I ran into some issues with refinement_matrices.pyx. I'll try to confirm that. Best, Alex -- Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne -- Australia -- http://www.ms.unimelb.edu.au/~aghitza/ -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [sage-devel] the attack of the (really) long doctests
I will do groups/perm_gps/partn_ref/ next. It's taking a bit longer because I think I ran into some issues with refinement_matrices.pyx. What are the issues? I can take care of partn_ref if you want... -- Robert L. Miller http://www.rlmiller.org/ -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
[sage-devel] the attack of the (really) long doctests
Hi, The developer guide says that any single doctest shouldn't take more than around 30 seconds. This is of course highly machine-dependent, but I assume we can take sage.math as a reference point. There are a handful of files in the Sage library that account for a large amount of the time necessary to run long doctests: sage -t -long devel/sage/sage/graphs/graph.py [152.2 s] sage -t -long devel/sage/sage/rings/arith.py [165.8 s] sage -t -long devel/sage/sage/plot/plot.py [167.3 s] sage -t -long devel/sage/sage/schemes/elliptic_curves/sha_tate.py [186.4 s] sage -t -long devel/sage/sage/groups/perm_gps/partn_ref/refinement_graphs.pyx [213.6 s] sage -t -long devel/sage/sage/schemes/elliptic_curves/ell_rational_field.py [224.7 s] sage -t -long devel/sage/sage/combinat/root_system/weyl_characters.py [255.1 s] sage -t -long devel/sage/sage/groups/matrix_gps/matrix_group.py [91.4 s] sage -t -long devel/sage/sage/graphs/graph_generators.py [111.7 s] sage -t -long devel/sage/sage/groups/perm_gps/partn_ref/refinement_binary.pyx [291.2 s] Together, these 10 files take about 26% of all doctesting time. In some of them, this is due to just one or two doctests that take a very long time (definitely more than 30 seconds). Take for instance rings/arith.py: -- | Sage Version 4.3.rc0, Release Date: 2009-12-10 | | Type notebook() for the GUI, and license() for information.| -- ** ** * Warning: this is a prerelease version, and it may be unstable. * ** ** sage: algs = ['gap', 'gp', 'pari', 'bernmm'] sage: time vals = [[bernoulli(i, algorithm = j) for j in algs] for i in range(2, 2255)] CPU times: user 24.40 s, sys: 1.13 s, total: 25.53 s Wall time: 41.37 s sage: algs = ['gp', 'pari', 'bernmm'] sage: time vals = [[bernoulli(i, algorithm = j) for j in algs] for i in range(2256, 5000)] CPU times: user 96.18 s, sys: 1.13 s, total: 97.31 s Wall time: 109.31 s Note that most of the this time is taken by the bernmm algorithm: sage: time vals = [bernoulli(i, algorithm = 'bernmm') for i in range(2256, 5000)] CPU times: user 87.33 s, sys: 0.00 s, total: 87.33 s Wall time: 87.33 s I understand that we want to test bernmm thoroughly. But can we do something less time-consuming? For instance, instead of checking it on range(2, 5000), can we pick 500 random integers in that range and check it there? Here's what that would look like on sage.math: sage: test_list = [ZZ.random_element(2, 5000) for _ in range(500)] sage: time vals = [bernoulli(i, algorithm = 'bernmm') for i in test_list] CPU times: user 9.38 s, sys: 0.00 s, total: 9.38 s Wall time: 9.37 s Another example is the randomized testing in groups/perm_gps/partn_ref/refinement_binary.pyx: sage: import sage.groups.perm_gps.partn_ref.refinement_binary sage: x = walltime() sage: set_random_seed(int(str(x-int(x))[2:])) sage: time sage.groups.perm_gps.partn_ref.refinement_binary.random_tests(50, 200, 10) All passed: 2000 random tests on 100 codes. CPU times: user 206.11 s, sys: 0.00 s, total: 206.11 s Wall time: 206.11 s Would it be ok to change this so it takes a little bit less time? There are other examples out there. I'm not advocating blindly shortening very long doctests -- some of them might be there for a very good reason. However, if it's possible to reduce at least some of them, that would help a lot with testing on machines that don't have the kind of resources that sage.math has. In the same vein, +1 to Nicolas' suggestion at http://trac.sagemath.org/sage_trac/ticket/7493 This could be something that patch authors and ticket reviewers run to help catch doctests that are long, or too long. Best, Alex -- Alex Ghitza -- Lecturer in Mathematics -- The University of Melbourne -- Australia -- http://www.ms.unimelb.edu.au/~aghitza/ -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
