Hello! Beam has a possibility to "Phrase trigger" Jenkins' jobs from Github. If a job has a special phrase defined in it, a contributor can type the phrase in a PR comment and the Job will run on Jenkins. A very popular usage is to run PreCommits this way (eg. by typing "Run JavaPreCommit" in a PR comment), but almost all jobs seem to have some phrase defined.
For jobs that exist on the master branch (in .test_infra/jenkins), it is enough to just type the phrase. But if we want to run a job that is freshly defined in our pull request, we need to run the Seed job first to overwrite existing Jenkins configuration (by typing "Run seed job" in the comment). This feature, however super useful and easy to use, to me has some disadvantages: - we overwrite the whole Jenkins' configuration. This means that when two developers run seed job at the same time they interrupt each other's work without even noticing, - there is no way to "clean up" after such phrase triggering. Job definitions from the PR will remain on Jenkins until Seed job is run again from master (this happens every 6 hours at least). This, in turn, means that if we define a job that is also time-triggered, it will start again but using the source from master branch. It will fail if the code invoked by the job is not on master yet. This is, in fact, a very common scenario when adding new Performance tests. Do you think we can improve this a little bit? I realize that it can be hard not to overwrite all Jenkins' definitions (or maybe I'm wrong?), but maybe we can add some little improvements, like: - define a special seed(ish) job that only adds new jobs and does not overwrite the others. It would reduce the risk of interrupting each other's work, - define special phrase trigger which will run the seed job on master (not on the PR branch) on demand. It would allow doing the "clean up". WDYT? Do you have other ideas/suggestions? Did you face the above-mentioned issues before? Best regards, Łukasz
