Hi Nasser Nice work! Thanks and best regards Kurt p.s. I'm quite surprised by the Sympy performance. Not bad (IMO) for a pure Python system :)
On 21.08.2022 08:15, 'Nasser M. Abbasi' via FriCAS - computer algebra system wrote: > FYI, > > (Posted at https://groups.google.com/g/fricas-devel and > https://groups.google.com/g/sci.math.symbolic) > > Independent CAS integration tests summer 2022 edition is now complete. > (It took about 5 months since I initially started working on it!) > > <https://www.12000.org/my_notes/CAS_integration_tests/reports/summer_2022/index.htm> > > The following is summary of changes in this version > compared to last year's edition > ======================== > 1. Number of integrals increased to 85,479 from 71,994. > > The integrals are made of the following 210 files: > > a. Rubi test suite. Files 1 to 208. Thanks to Alert Rich. > Downloaded from <https://github.com/RuleBasedIntegration> > b. IntegrateAlgebraic test file. File 209. Thanks to Sam Blake. > c. Fricas test file. File 210. Thanks to Waldek Hebisch. > > 2. CAS version changes: > Maple 2021.1 -> 2022.1 > Mathematica 12.3 -> 13.01 > Giac 1.7.0 -> 1.9.0-13 > Sympy 1.8/python 3.8.8 -> 1.10.1/Python 3.10.4 > Maxima 5.44 -> 5.46 > FriCAS 1.3.7 -> 1.3.8 > Sagemath 9.3 -> 9.6 > > No version change for Mupad at Matlab 2021a and Rubi at 4.16.1 > but Rubi was run again on the current Mathematica 13.1 version. > > 3. General improvements to Latex and other formatting. > > 4. Flow chart of the test program at > <https://www.12000.org/my_notes/CAS_integration_tests/images/design.svg> > > 5. Links to download all the 85,483 integrals in plain text in > Mathematica, Rubi, Maple, Mupad, Sympy, and Sagemath (Fricas, Maxima, Giac) > formats are at introduction page of the main page. > > 6. Regression reports comparing results in 2022 edition to 2021 edition > are also now available. These reports show integrals that failed > in current versions compared to last versions per CAS. > > <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/index.htm> > > Regression per CAS > ------------------ > This table below gives the number of integrals that failed in > this test (could be due to timeout, exception or not solved) but > are solved in summer 2021 for each CAS. > > Number of regression per CAS is (less is better) > > Fricas: 16 > Maxima: 17 > Mathematica: 98 > Maple: 114 > Giac: 363 > Sympy: 379 > > Tables give details about each integral that failed in regression. > > 7. Sympy, Sagemath (Fricas, Maxima, Giac) were all run on Linux > inside Oracle Virtual Box. Maple, Rubi, Mupad and Mathematica were > run on windows 10. All on same PC with 128 GB RAM with > Intel Core i9-12900K 3.20 GHz. > > RESULTS > ======= > > Overall solved percentage in summer 2022 edition > ========================================= > 1. Mathematica 13.1 97.99 % > 2. Rubi 4.16.1 94.21 % > 3. Maple 2022.1 84.53 % > 4. Fricas 1.3.8 79.33 % > 5. Giac 1.9.0-13 58.37 % > 6. Maxima 5.46 56.86 % > 7. Mupad 2021a 56.1 % > 8. Sympy 1.10.1 41.89 % > > Grading results > ================= > Note that mupad is not graded. Default grade of B is given for > any passed integral. > > 1. Rubi 91.50 % > 2. Mathematica 78.58 % > 3. Maple 56.34 % > 4. Fricas 53.78 % > 5. Maxima 43.57 % > 6. Giac 42.36 % > 7. Sympy 28.49 % > 8. Mupad* N/A > > Time and leaf size Performance > ============================== > Sorted by time performance in average time used per integrals (seconds). > > Note for Maple(*) timing: the option method=_RETURNVERBOSE was used > which made Maple run through all integration methods increasing > average time used. For example in summer 2021 edition which did > not use this option, Maple had 0.79 seconds average time on > a slower PC also than the one used for the current tests. > > 1. Rubi 0.3 (sec) > 2. Maxima 0.49 (sec) > 3. Giac 0.75 (sec) > 4. Fricas 1.17 (sec) > 5. Mathematica 2.38 (sec) > 6. Mupad 2.75 (sec) > 7. Maple* 3.33 (sec) > 8. Sympy 5.24 (sec) > > General notes > ============== > 1) Maxima, Fricas and Giac were called via Sagemath. Some integrals > could have failed due to interface issues. These will have F(-2) > error code in the section "Detailed conclusion table per each > integral for all CAS systems" per each report and also in the actual > integral page with the actual exception name shown. > > This allows one to see if the cause of the failure is due to > interface issue or internal error from the CAS itself. > > For an example, the integral on > > <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection69.htm#53> > > Has F(-2) on Fricas output, but looking at the exception > > integrate(x*sin(a+b*log(c*x^n))^(1/2),x, algorithm="fricas") > > "Exception raised: TypeError >> Error detected within library code: > integrate: implementation incomplete (has polynomial part)" > > So this is not interface issue with Sagemath. The exception comes > from Fricas itself. > > But this integral > > <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection57.htm#41> > > Has F(-2) on Giac result, and looking at the exception it says > > integrate(x^2*sin(a+log(c*x^n)*(-1/n^2)^(1/2))^3,x, algorithm="giac") > > "Exception raised: NotImplementedError >> > Unable to parse Giac output: > ((-9*i)*sageVARn^4*sageVARx^3*exp((-3*i) *sageVARa)* > exp((3*sageVARn*abs(sageVARn)*ln(sageVARx)+3*abs(sageVARn)* > ln(sageVARc))/sageVARn^2)+27*i*sageVARn^4*sageVARx^3*exp((-i)" > > Which indicates it is an interface issue this time. This is > verified by running this integral directly in giac and it is > solved with no error. > > CAS integration test program currently does not distinguish > between an exception generated due to a sagemath interface problem > or an exception generated from the CAS itself. I am not > sure if there a robust way to do this for Maxima, Fricas > and Giac. If a way can be found, will add it. > > But there seems to be less of these interface problems in the > current Sagemath version than the last one used. > > 2) The section "List of integrals sorted by grade for each CAS" in each > report shows list of integrals for each grade per CAS. This allows > one to quickly find integrals with specific grade. > > 3) Also as mentioned before there are a number of integrals for Maxima > which fail when it asks a question in order to complete the > integration as this is not supported by Sagemath or by the > CAS integration test program. If an option can be added to > turn this feature off that will be a good thing and will improve > Maxima score. > > 4) Any bugs found or problems please let me know and will try my best > to fix them. > > --Nasser > -- You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group. To unsubscribe from this group and stop receiving emails from it, send an email to fricas-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/1eca04c2-ac1a-9da6-8788-e9b166c48108%40gmail.com.
