Looking at your linked GitHub action, you run the benchmark in the last 5 commits. SymPy's CI for PRs only runs the benchmarks for the PR and master.
Aside from this, it's dependent on how long your benchmarks take to run. Someone should probably write a guide on how to make a good benchmark. Ideally a benchmark shouldn't take more than a second to run, and that's at the very high end (10 or 100 ms is better), because it needs to be run multiple times to get an average runtime. Benchmarks that run too quickly are also bad because you are basically benchmarking noise. Also, if TARDIS has to be recompiled every time in order to install it into the virtual environment, that could be a factor (SymPy is pure Python, so installation time is not an issue). If this is an issue you might consider caching nightly wheels for recent commits somewhere. Aaron Meurer On Mon, Jul 15, 2024 at 1:14 AM Asish Kumar <officialasishku...@gmail.com> wrote: > > Hi there, > > I'm a GSoC student with TARDIS-SN this summer and I had a issue with the long > time it takes to run benchmarks. Currently my benchmarks are getting run for > 1.5 hrs (for only a single commit). I was wondering how sympy does it? > > I checked the benchmark actions of sympy and was pretty surprised to see that > it only takes 20 mins to run the whole benchmark. > > Thank you > Asish Kumar > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/1ff21b7f-382e-4d73-90fd-85316b9ca5bdn%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6K5sTb-C9VhSRA02ZAugpKGo0oJ97NqaHne2K11HcW4Cw%40mail.gmail.com.
