I was recently checking https://github.com/sympy/sympy_benchmarks and had some questions:
1. Are slow_benchmarks are also run by asv? 2. If yes, why it's in a seperate folder? 3. If No, why it's there? On Tuesday, July 16, 2024 at 12:00:44 AM UTC+5:30 asme...@gmail.com wrote: > Looking at your linked GitHub action, you run the benchmark in the > last 5 commits. SymPy's CI for PRs only runs the benchmarks for the PR > and master. > > Aside from this, it's dependent on how long your benchmarks take to > run. Someone should probably write a guide on how to make a good > benchmark. Ideally a benchmark shouldn't take more than a second to > run, and that's at the very high end (10 or 100 ms is better), because > it needs to be run multiple times to get an average runtime. > Benchmarks that run too quickly are also bad because you are basically > benchmarking noise. > > Also, if TARDIS has to be recompiled every time in order to install it > into the virtual environment, that could be a factor (SymPy is pure > Python, so installation time is not an issue). If this is an issue you > might consider caching nightly wheels for recent commits somewhere. > > Aaron Meurer > > On Mon, Jul 15, 2024 at 1:14 AM Asish Kumar > <officiala...@gmail.com> wrote: > > > > Hi there, > > > > I'm a GSoC student with TARDIS-SN this summer and I had a issue with the > long time it takes to run benchmarks. Currently my benchmarks are getting > run for 1.5 hrs (for only a single commit). I was wondering how sympy does > it? > > > > I checked the benchmark actions of sympy and was pretty surprised to see > that it only takes 20 mins to run the whole benchmark. > > > > Thank you > > Asish Kumar > > > > -- > > You received this message because you are subscribed to the Google > Groups "sympy" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to sympy+un...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/1ff21b7f-382e-4d73-90fd-85316b9ca5bdn%40googlegroups.com > . > -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/34cb7637-4acd-4ade-8b14-b684edaf31d2n%40googlegroups.com.
