Re: [TC Bot] Proposal of improvement
Hi Igniters, I'm glad to announce that one more change is done related to PR easy navigation. Now TC bot detects if there was a runAll and shows button <> only if there is some completed build. It is not a final-state of UI, but currently, I think https://issues.apache.org/jira/browse/IGNITE-9800 is done. In pair with the contribution done by Nikolay Kulagin ( https://issues.apache.org/jira/browse/IGNITE-9770 ), it allows navigating from the main screen to getting bot Visa without tons of data to be entered. I've documented flow as graph here: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+Teamcity+Bot#ApacheIgniteTeamcityBot-HowtocheckaPRwiththeTCBot - this to be enhanced by automatic scenarios once it is available in simple navigation mode. Thanks to everyone involved. I hope I will have spare cycles to continue to contribute. Sincerely, Dmitriy Pavlov пн, 8 окт. 2018 г. в 15:39, Dmitriy Pavlov : > Hi Nikolay, Igniters, > > Please check the newest version of the Bot. > > It contains huge UI refactoring and simplified navigation to open PR and > its test results. Just couple seconds is needed to find a PR now. > > Thanks to Dmitrii Ryabov, who developed initial GitHub integration. It is > more or less reused, and PRs are cached in the Ignite instance. > > As always, I appreciate the feedback. > > Sincerely, > Dmitriy Pavlov > > вт, 25 сент. 2018 г. в 13:00, Dmitriy Pavlov : > >> JIRA ticket is some elementary change that needs to be reviewed. We don't >> review the patch, we review ticket (with motivation, implementation >> details, history of discussion), so reviewer will look at a ticket first. >> >> PR does not have a mark, that it is ready to be merged. Some PRs are >> created just for research, but Patch Available ticket is something that is >> ready to be in the product. >> >> So if we concentrate on a ticket, from the point of view of a new >> contributor, >> - he or she creates a branch, PR and sets ticket to PA, >> - and the bot will do all necessary mechanic work. >> >> No issues with asking newcomers to run (proper) tests. A newbie needs >> only to follow the first steps in How To Contribute. A reviewer will see a >> ticket with the bot Visa after 2-3 hours after setting of PA state. >> >> But only one concern I have here, I'm not sure I have spare cycles to do >> this project. I'd rather move towards it in step-by-step mode, doing small >> changes in each version. Any assistance is appreciated here. >> >> вт, 25 сент. 2018 г. в 11:25, Nikolay Izhikov : >> >>> Hello, Dmitriy >>> >>> > What about the case when committer creates ignite-9679 branch and >>> tests it> without PR? >>> >>> It means, committer is experienced enough to run tests via Team City >>> interface :). >>> >>> > So scanning seems to be possible only in JIRA >>> >>> I don't understand you here. >>> You can retrieve comments filtered by *date*. >>> You don't have to scan all 1000 PR's one by one. >>> Anyway 1000 PR doesn't sound like big issue for me. >>> >>> My vote goes strong to GiHub user interface. >>> I think we should have closer integration with GitHub, not Jira. >>> >>> Jira is about tickets and project management. >>> GitHub is about code, commits and patches. >>> We test patch, not ticket. >>> >>> >>> В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет: >>> > Hi Nikolay, >>> > >>> > What about the case when committer creates ignite-9679 branch and >>> tests it >>> > without PR? >>> > >>> > We have 1100+ open PRs and less than 100 open tickets. So scanning >>> seems to >>> > be possible only in JIRA. Mention probably will work for GitHub, but it >>> > needs to be researched. >>> > >>> > Two open PRs is not a valid situation in the majority of cases and How >>> To >>> > Contribute asks to avoid it. The bot can ignore closed PRs and the bot >>> can >>> > expect there is only one open PR per ticket. >>> > >>> > Sincerely, >>> > Dmitriy Pavlov >>> > >>> > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov : >>> > >>> > > Hello, Dmitriy. >>> > > >>> > > > But it could be a lot of work to handle mentions (probably from the >>> > > >>> > > email> account and inbox). >>> > > >>> > > Actually, it can be done via GitHub REST API [1]. >>> > > It has 'since' param, so getting new GitHub comments is a very basic >>> task. >>> > > >>> > > > Patch available ticket >>> > > >>> > > I think we shouldn't take a ticket as an entity that should be >>> tested. >>> > > For me, it's a PR. >>> > > >>> > > Moreover, it's a common case when we have several PR in a ticket. >>> > > And it's a common case when both of them has to be tested. >>> > > >>> > > My vote goes to the closer integration with GitHub. >>> > > >>> > > [1] >>> > > >>> https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository >>> > > >>> > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: >>> > > > Hi Nikolay, >>> > > > >>> > > > The idea makes perfect sense for me, and we should definitely take >>> the >>> > > >>> > > best >>> > >
Re: [TC Bot] Proposal of improvement
JIRA ticket is some elementary change that needs to be reviewed. We don't review the patch, we review ticket (with motivation, implementation details, history of discussion), so reviewer will look at a ticket first. PR does not have a mark, that it is ready to be merged. Some PRs are created just for research, but Patch Available ticket is something that is ready to be in the product. So if we concentrate on a ticket, from the point of view of a new contributor, - he or she creates a branch, PR and sets ticket to PA, - and the bot will do all necessary mechanic work. No issues with asking newcomers to run (proper) tests. A newbie needs only to follow the first steps in How To Contribute. A reviewer will see a ticket with the bot Visa after 2-3 hours after setting of PA state. But only one concern I have here, I'm not sure I have spare cycles to do this project. I'd rather move towards it in step-by-step mode, doing small changes in each version. Any assistance is appreciated here. вт, 25 сент. 2018 г. в 11:25, Nikolay Izhikov : > Hello, Dmitriy > > > What about the case when committer creates ignite-9679 branch and tests > it> without PR? > > It means, committer is experienced enough to run tests via Team City > interface :). > > > So scanning seems to be possible only in JIRA > > I don't understand you here. > You can retrieve comments filtered by *date*. > You don't have to scan all 1000 PR's one by one. > Anyway 1000 PR doesn't sound like big issue for me. > > My vote goes strong to GiHub user interface. > I think we should have closer integration with GitHub, not Jira. > > Jira is about tickets and project management. > GitHub is about code, commits and patches. > We test patch, not ticket. > > > В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет: > > Hi Nikolay, > > > > What about the case when committer creates ignite-9679 branch and tests > it > > without PR? > > > > We have 1100+ open PRs and less than 100 open tickets. So scanning seems > to > > be possible only in JIRA. Mention probably will work for GitHub, but it > > needs to be researched. > > > > Two open PRs is not a valid situation in the majority of cases and How To > > Contribute asks to avoid it. The bot can ignore closed PRs and the bot > can > > expect there is only one open PR per ticket. > > > > Sincerely, > > Dmitriy Pavlov > > > > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov : > > > > > Hello, Dmitriy. > > > > > > > But it could be a lot of work to handle mentions (probably from the > > > > > > email> account and inbox). > > > > > > Actually, it can be done via GitHub REST API [1]. > > > It has 'since' param, so getting new GitHub comments is a very basic > task. > > > > > > > Patch available ticket > > > > > > I think we shouldn't take a ticket as an entity that should be tested. > > > For me, it's a PR. > > > > > > Moreover, it's a common case when we have several PR in a ticket. > > > And it's a common case when both of them has to be tested. > > > > > > My vote goes to the closer integration with GitHub. > > > > > > [1] > > > > https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository > > > > > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: > > > > Hi Nikolay, > > > > > > > > The idea makes perfect sense for me, and we should definitely take > the > > > > > > best > > > > practices from other big Apache projects. > > > > > > > > But it could be a lot of work to handle mentions (probably from the > email > > > > account and inbox). > > > > > > > > I would like to suggest the following algorithm: > > > > > > > > Patch available ticket, which was never checked by the bot will be > > > > processed in the following steps: > > > > 1. check existing run all (by PR or by branch name), if found go to > the > > > > step 3 > > > > 2. run-all to be triggered by PR > > > > 3. results should be analyzed for the presence of possible blockers. > If > > > > there is no blockers go to step 5. > > > > 4. re-run of particular suites containing possible blockers should be > > > > applied to try to get success for very rare flaky failures (<1%). Go > to 3 > > > > (this go to should be done only once). > > > > 5. comment should be added to JIRA ticket containing information > about > > > > results. > > > > > > > > If a ticket was processed by bot early (probably author added some > fixes) > > > > but still in PA state, the bot will check comments list and find > possible > > > > new mentions (made after the previous build complete date). If it > finds > > > > such comments it goes to step 1 (trying to find only new builds > > > > > > available). > > > > > > > > What do you think? > > > > > > > > Sincerely, > > > > Dmitriy Pavlov > > > > > > > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov : > > > > > > > > > Hello, Igniters. > > > > > > > > > > I propose to implement following behaviour: > > > > > > > > > > 1. Execute "Run all" suite for specific PR when the author of PR > makes > > > > > > a > > > > > comment
Re: [TC Bot] Proposal of improvement
Hello, Dmitriy > What about the case when committer creates ignite-9679 branch and tests it> > without PR? It means, committer is experienced enough to run tests via Team City interface :). > So scanning seems to be possible only in JIRA I don't understand you here. You can retrieve comments filtered by *date*. You don't have to scan all 1000 PR's one by one. Anyway 1000 PR doesn't sound like big issue for me. My vote goes strong to GiHub user interface. I think we should have closer integration with GitHub, not Jira. Jira is about tickets and project management. GitHub is about code, commits and patches. We test patch, not ticket. В Вт, 25/09/2018 в 00:06 +0300, Dmitriy Pavlov пишет: > Hi Nikolay, > > What about the case when committer creates ignite-9679 branch and tests it > without PR? > > We have 1100+ open PRs and less than 100 open tickets. So scanning seems to > be possible only in JIRA. Mention probably will work for GitHub, but it > needs to be researched. > > Two open PRs is not a valid situation in the majority of cases and How To > Contribute asks to avoid it. The bot can ignore closed PRs and the bot can > expect there is only one open PR per ticket. > > Sincerely, > Dmitriy Pavlov > > пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov : > > > Hello, Dmitriy. > > > > > But it could be a lot of work to handle mentions (probably from the > > > > email> account and inbox). > > > > Actually, it can be done via GitHub REST API [1]. > > It has 'since' param, so getting new GitHub comments is a very basic task. > > > > > Patch available ticket > > > > I think we shouldn't take a ticket as an entity that should be tested. > > For me, it's a PR. > > > > Moreover, it's a common case when we have several PR in a ticket. > > And it's a common case when both of them has to be tested. > > > > My vote goes to the closer integration with GitHub. > > > > [1] > > https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository > > > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: > > > Hi Nikolay, > > > > > > The idea makes perfect sense for me, and we should definitely take the > > > > best > > > practices from other big Apache projects. > > > > > > But it could be a lot of work to handle mentions (probably from the email > > > account and inbox). > > > > > > I would like to suggest the following algorithm: > > > > > > Patch available ticket, which was never checked by the bot will be > > > processed in the following steps: > > > 1. check existing run all (by PR or by branch name), if found go to the > > > step 3 > > > 2. run-all to be triggered by PR > > > 3. results should be analyzed for the presence of possible blockers. If > > > there is no blockers go to step 5. > > > 4. re-run of particular suites containing possible blockers should be > > > applied to try to get success for very rare flaky failures (<1%). Go to 3 > > > (this go to should be done only once). > > > 5. comment should be added to JIRA ticket containing information about > > > results. > > > > > > If a ticket was processed by bot early (probably author added some fixes) > > > but still in PA state, the bot will check comments list and find possible > > > new mentions (made after the previous build complete date). If it finds > > > such comments it goes to step 1 (trying to find only new builds > > > > available). > > > > > > What do you think? > > > > > > Sincerely, > > > Dmitriy Pavlov > > > > > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov : > > > > > > > Hello, Igniters. > > > > > > > > I propose to implement following behaviour: > > > > > > > > 1. Execute "Run all" suite for specific PR when the author of PR makes > > > > a > > > > comment > > > > "@mtcga.bot Run Tests!" in GitHub comments. > > > > > > > > 2. Send a comment with "Run All" results both: to a Jira ticket and > > > > GitHub > > > > comment. > > > > > > > > 3. Label PR based on "Run All" results like it done in Apache Kafka [1] > > > > > > > > I've create ticket for this proposal [2] > > > > > > > > Thoughts? > > > > > > > > [1] https://github.com/apache/kafka/pulls > > > > [2] https://issues.apache.org/jira/browse/IGNITE-9678 signature.asc Description: This is a digitally signed message part
Re: [TC Bot] Proposal of improvement
Hi Nikolay, What about the case when committer creates ignite-9679 branch and tests it without PR? We have 1100+ open PRs and less than 100 open tickets. So scanning seems to be possible only in JIRA. Mention probably will work for GitHub, but it needs to be researched. Two open PRs is not a valid situation in the majority of cases and How To Contribute asks to avoid it. The bot can ignore closed PRs and the bot can expect there is only one open PR per ticket. Sincerely, Dmitriy Pavlov пн, 24 сент. 2018 г. в 23:41, Nikolay Izhikov : > Hello, Dmitriy. > > > But it could be a lot of work to handle mentions (probably from the > email> account and inbox). > > Actually, it can be done via GitHub REST API [1]. > It has 'since' param, so getting new GitHub comments is a very basic task. > > > Patch available ticket > > I think we shouldn't take a ticket as an entity that should be tested. > For me, it's a PR. > > Moreover, it's a common case when we have several PR in a ticket. > And it's a common case when both of them has to be tested. > > My vote goes to the closer integration with GitHub. > > [1] > https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository > > В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: > > Hi Nikolay, > > > > The idea makes perfect sense for me, and we should definitely take the > best > > practices from other big Apache projects. > > > > But it could be a lot of work to handle mentions (probably from the email > > account and inbox). > > > > I would like to suggest the following algorithm: > > > > Patch available ticket, which was never checked by the bot will be > > processed in the following steps: > > 1. check existing run all (by PR or by branch name), if found go to the > > step 3 > > 2. run-all to be triggered by PR > > 3. results should be analyzed for the presence of possible blockers. If > > there is no blockers go to step 5. > > 4. re-run of particular suites containing possible blockers should be > > applied to try to get success for very rare flaky failures (<1%). Go to 3 > > (this go to should be done only once). > > 5. comment should be added to JIRA ticket containing information about > > results. > > > > If a ticket was processed by bot early (probably author added some fixes) > > but still in PA state, the bot will check comments list and find possible > > new mentions (made after the previous build complete date). If it finds > > such comments it goes to step 1 (trying to find only new builds > available). > > > > What do you think? > > > > Sincerely, > > Dmitriy Pavlov > > > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov : > > > > > Hello, Igniters. > > > > > > I propose to implement following behaviour: > > > > > > 1. Execute "Run all" suite for specific PR when the author of PR makes > a > > > comment > > > "@mtcga.bot Run Tests!" in GitHub comments. > > > > > > 2. Send a comment with "Run All" results both: to a Jira ticket and > GitHub > > > comment. > > > > > > 3. Label PR based on "Run All" results like it done in Apache Kafka [1] > > > > > > I've create ticket for this proposal [2] > > > > > > Thoughts? > > > > > > [1] https://github.com/apache/kafka/pulls > > > [2] https://issues.apache.org/jira/browse/IGNITE-9678
Re: [TC Bot] Proposal of improvement
Hello, Dmitriy. > But it could be a lot of work to handle mentions (probably from the email> > account and inbox). Actually, it can be done via GitHub REST API [1]. It has 'since' param, so getting new GitHub comments is a very basic task. > Patch available ticket I think we shouldn't take a ticket as an entity that should be tested. For me, it's a PR. Moreover, it's a common case when we have several PR in a ticket. And it's a common case when both of them has to be tested. My vote goes to the closer integration with GitHub. [1] https://developer.github.com/v3/pulls/comments/#list-comments-in-a-repository В Пн, 24/09/2018 в 22:36 +0300, Dmitriy Pavlov пишет: > Hi Nikolay, > > The idea makes perfect sense for me, and we should definitely take the best > practices from other big Apache projects. > > But it could be a lot of work to handle mentions (probably from the email > account and inbox). > > I would like to suggest the following algorithm: > > Patch available ticket, which was never checked by the bot will be > processed in the following steps: > 1. check existing run all (by PR or by branch name), if found go to the > step 3 > 2. run-all to be triggered by PR > 3. results should be analyzed for the presence of possible blockers. If > there is no blockers go to step 5. > 4. re-run of particular suites containing possible blockers should be > applied to try to get success for very rare flaky failures (<1%). Go to 3 > (this go to should be done only once). > 5. comment should be added to JIRA ticket containing information about > results. > > If a ticket was processed by bot early (probably author added some fixes) > but still in PA state, the bot will check comments list and find possible > new mentions (made after the previous build complete date). If it finds > such comments it goes to step 1 (trying to find only new builds available). > > What do you think? > > Sincerely, > Dmitriy Pavlov > > пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov : > > > Hello, Igniters. > > > > I propose to implement following behaviour: > > > > 1. Execute "Run all" suite for specific PR when the author of PR makes a > > comment > > "@mtcga.bot Run Tests!" in GitHub comments. > > > > 2. Send a comment with "Run All" results both: to a Jira ticket and GitHub > > comment. > > > > 3. Label PR based on "Run All" results like it done in Apache Kafka [1] > > > > I've create ticket for this proposal [2] > > > > Thoughts? > > > > [1] https://github.com/apache/kafka/pulls > > [2] https://issues.apache.org/jira/browse/IGNITE-9678 signature.asc Description: This is a digitally signed message part
Re: [TC Bot] Proposal of improvement
Hi Nikolay, The idea makes perfect sense for me, and we should definitely take the best practices from other big Apache projects. But it could be a lot of work to handle mentions (probably from the email account and inbox). I would like to suggest the following algorithm: Patch available ticket, which was never checked by the bot will be processed in the following steps: 1. check existing run all (by PR or by branch name), if found go to the step 3 2. run-all to be triggered by PR 3. results should be analyzed for the presence of possible blockers. If there is no blockers go to step 5. 4. re-run of particular suites containing possible blockers should be applied to try to get success for very rare flaky failures (<1%). Go to 3 (this go to should be done only once). 5. comment should be added to JIRA ticket containing information about results. If a ticket was processed by bot early (probably author added some fixes) but still in PA state, the bot will check comments list and find possible new mentions (made after the previous build complete date). If it finds such comments it goes to step 1 (trying to find only new builds available). What do you think? Sincerely, Dmitriy Pavlov пн, 24 сент. 2018 г. в 21:43, Nikolay Izhikov : > Hello, Igniters. > > I propose to implement following behaviour: > > 1. Execute "Run all" suite for specific PR when the author of PR makes a > comment > "@mtcga.bot Run Tests!" in GitHub comments. > > 2. Send a comment with "Run All" results both: to a Jira ticket and GitHub > comment. > > 3. Label PR based on "Run All" results like it done in Apache Kafka [1] > > I've create ticket for this proposal [2] > > Thoughts? > > [1] https://github.com/apache/kafka/pulls > [2] https://issues.apache.org/jira/browse/IGNITE-9678
