On Sun, Nov 17, 2019 at 4:07 AM Oscar Benjamin <oscar.j.benja...@gmail.com> wrote: > > On Sun, 17 Nov 2019 at 05:01, Aaron Meurer <asmeu...@gmail.com> wrote: > > > > Thinking more about the automation, I think we need to have the > > release be 100% automated. Right now it is 90% automated (not counting > > the problem of fixing blocker issues). > > > > Oscar, it will be a good idea to make a note of anything in the > > release process that isn't automated that should be. > > I will make notes and rewrite the release/README afterwards. To be > honest though the reason it has taken 5 days to get to the point of > having a beta release is actually not because of the *lack* of > automation. It has taken that long because I needed to figure out what > was going on in the release script which is very slow to run and the > only way to try and fix anything is to go through a PR with full > Travis tests etc. > > I am in the process of getting the right permissions to be able to use > docker at work which will speed things up but for now I am running > this on a laptop at home with a relatively slow internet connection. > The first time the release script runs it needs to get the docker > containers. Each time the release script runs (and multiple times > within each run) it needs to download a lot of files from conda. It > also runs the whole sympy test suite in between doing the other tasks. > > For me right now it takes around 2 hours to run the release script. > Once the script fails it takes about 5 minutes to write the code that > fixes it but that needs to be sent back to github and it takes 1.5 > hours to run the tests before they can be merged and the release > script can start again. That means that any issue in the release > script has a 4 hour turn around time to detect the problem and attempt > something to fix it.
Once we have this on CI it won't be an issue. The biggest reason it is slow is that it runs the full test suite. We could remove that once it's on CI because the other CI already runs the tests (although that would make it not be able to catch bugs like https://github.com/sympy/sympy/issues/17884 so maybe it isn't a good idea). The docs also take a little while to build but not an hour. rever locally also is able to cache results so it doesn't have to rerun things that haven't changed (the docker script reruns from scratch for reproducibility). So we just need to clarify the process of how to test the release vs. how to do the final release once everything is working. > > What would make all of this faster is if the elements of the release > script were split out as separate scripts that could be tested locally > without going through a PR and docker etc. This is the case for the > mailmap script but not for others like testing the tarball whitelist. You should be able to run the rever commands manually. We do the official release in a docker container so that it is isolated and reproducible, but for testing you can call rever directly. > > > One thing I forgot about is the release notes. It looks like you already > > added the > > deprecations. We should figure out how to automate that. Maybe the bot > > can do it. > > I've added deprecations that I knew about that weren't listed and I > used git grep to find any deprecation warnings with the 1.5 version. > That can be automated but that only catches things that are > deprecated. The backward compatibility breaks that don't/can't have a > deprecation warning are the more important ones to document here and I > don't see any way to pick up on those automatically. > > Perhaps changing the PR template so that there is an explicit backward > compatibility release notes section would help. > > > The highlights can't be automated because they are human > > curated, so I think that means those should just be removed. Unless > > there is some way we can do it with the bot as well (maybe by letting > > people tag a PRs notes as a "highlighted note"?). > > > > The AUTHORS/.mailmap isn't automated either. I think they only way > > around that would be to run the authors/mailmap scripts as part of the > > Travis tests, so that we require any PR with a new new author or new > > git metadata to update those files. The main problem with that is that > > it puts an extra burden on new contributors, by definition. So I don't > > know if there's a better way to do it, or if it's worth doing at all. > > It isn't difficult to do and it also can be done at any time, not just > > during a release. Maybe we could add it to the CI but as an allowed > > failure, that way it isn't a requirement to merge, but it will alert > > people when it gets out of date. > > Fixing the mailmap isn't a big problem. If we had more regular > releases there wouldn't be so much to update there. It wold be good to > split out the part of the release script that shows the authors though > because that only happens at the end whereas the update_mailmap.py > script runs at the beginning. You should be able to do this with rever. rever -e print_authors or something like that. The only problem is the dependencies make it so that it requires you to run the tests first. So we should modify the dependency tree so the tests aren't required for anything except for the full release process. But ultimately we should have the bot add the authors to the release notes. I just haven't worked out all the details and implemented yet it. See https://github.com/sympy/sympy-bot/issues/44. The bot also needs to be made so it can create the new release notes file https://github.com/sympy/sympy-bot/issues/26. Both things are doable. We just need to implement them. > > > The only other thing I can remember that isn't automated is various > > updates that you have to do to the release script, like updating the > > tarball whitelist. But if we run the script on CI that will not be a > > problem because it will be updated when it needs to be, instead of > > when the release happens. > > This is a straight-forward thing to fix but it would be quicker and > easier to do if it could be checked before running the full release > script. > > Full automation is a possibility here but I also think that it can be > made a lot more efficient as a mostly automated process. > > Oscar > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sympy+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/sympy/CAHVvXxS9XUS_unw844q4C-jC%2B-4EL6Hd9Gqnj%2BU8rBNUWkRK5A%40mail.gmail.com. -- You received this message because you are subscribed to the Google Groups "sympy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sympy+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6KC4s%2BWm9YDqBiWpqBZmpWw_23JtojTh1CFA3nyAruMew%40mail.gmail.com.
