[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=383458&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-383458 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Feb/20 09:52 Start Date: 07/Feb/20 09:52 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-583315750 @markflyhigh Thanks for this in-depth explanation. Right now I'm distracted by other stuff too, but I'm really interested in helping to unify the way we construct those tests in Python SDK. Keep me posted about the progress, if there would be any. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 383458) Time Spent: 12h (was: 11h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 12h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=383022&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-383022 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 18:28 Start Date: 06/Feb/20 18:28 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 383022) Time Spent: 11h 50m (was: 11h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382918&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382918 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 14:29 Start Date: 06/Feb/20 14:29 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-582931086 R: @markflyhigh Please take a look at my answer. It'd be good if we could move forward with the PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382918) Time Spent: 11h 40m (was: 11.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382903&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382903 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 13:44 Start Date: 06/Feb/20 13:44 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-582911745 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382903) Time Spent: 11.5h (was: 11h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382890&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382890 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 13:29 Start Date: 06/Feb/20 13:29 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-582905935 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382890) Time Spent: 11h 20m (was: 11h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382889&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382889 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 13:28 Start Date: 06/Feb/20 13:28 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-582905303 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382889) Time Spent: 11h 10m (was: 11h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382885&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382885 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 13:20 Start Date: 06/Feb/20 13:20 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-58290 retest this poelase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382885) Time Spent: 10h 50m (was: 10h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=382886&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-382886 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Feb/20 13:20 Start Date: 06/Feb/20 13:20 Worklog Time Spent: 10m Work Description: mwalenia commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-58290 retest this poelase This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 382886) Time Spent: 11h (was: 10h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 11h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=377564&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377564 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 27/Jan/20 10:51 Start Date: 27/Jan/20 10:51 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-578693235 Thanks @markflyhigh. I've already pointed out the benefits in one of the comments above, let me do this once again here. What do we gain from moving load tests out of python test: - Without nose, it is difficult to pass additional command-line arguments to the test. Since load tests require additional parameters, this looks like a blocker for us. - As long as load tests are based on python tests, they are discovered by nose/pytest and included in test suites running as a part of PreCommit. This is undesirable. Load tests require additional parameters and they were never intented to be run as a part of PreCommit. In order to prevent this behaviour, we had developed some kind of guard[1] which skips test if certain condition is met. If we move out of python test, this guard will not be needed anymore. In general, this does not affect test management that much. We use gradle task[2] to run load tests and this gradle task is the only place affected by my changes. Let me know if you find this answer helpful. [1] https://github.com/apache/beam/blob/31a5a33bd268eb93aa5eb8d1c31f5043b85b1511/sdks/python/apache_beam/testing/load_tests/pardo_test.py#L134-L139 [2] https://github.com/apache/beam/blob/master/sdks/python/apache_beam/testing/load_tests/build.gradle This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377564) Time Spent: 10h 40m (was: 10.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=377012&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-377012 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 24/Jan/20 19:46 Start Date: 24/Jan/20 19:46 Worklog Time Spent: 10m Work Description: markflyhigh commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-578274409 Hi Kamil, sorry for the delay. I'm occupied by oncall stuff this week and plan to get back to this PR on Tuesday next week. I understand the idea of removing nose from load test. However, this change seems more than that, which is moving load test out of python test, and creating a divergence between general integration test. Can you provide more information for why we want to do that? What's the benefit? How does it affect the test management? Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 377012) Time Spent: 10.5h (was: 10h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=376941&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376941 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 24/Jan/20 17:39 Start Date: 24/Jan/20 17:39 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-578228395 Sorry for the delay and if I wasn't explicit enough - I'd like @markflyhigh to take a first pass on these changes. I'll touch bases with Mark to see if he is available to review. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 376941) Time Spent: 10h 20m (was: 10h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=376758&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-376758 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 24/Jan/20 09:00 Start Date: 24/Jan/20 09:00 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-578046335 R: @tvalentyn @markflyhigh Kindly asking for a review once again This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 376758) Time Spent: 10h 10m (was: 10h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373585&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373585 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 12:57 Start Date: 17/Jan/20 12:57 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575615328 @tvalentyn @markflyhigh PTAL This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373585) Time Spent: 10h (was: 9h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 10h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373567&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373567 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 12:14 Start Date: 17/Jan/20 12:14 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575602959 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373567) Time Spent: 9h 50m (was: 9h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373548&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373548 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 11:15 Start Date: 17/Jan/20 11:15 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575585096 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373548) Time Spent: 9h 40m (was: 9.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373534&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373534 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 10:59 Start Date: 17/Jan/20 10:59 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575579400 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373534) Time Spent: 9.5h (was: 9h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373533 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 10:59 Start Date: 17/Jan/20 10:59 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575579345 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373533) Time Spent: 9h 20m (was: 9h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373500&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373500 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 09:46 Start Date: 17/Jan/20 09:46 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575553136 Merge conflicts resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373500) Time Spent: 9h 10m (was: 9h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373499&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373499 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 09:46 Start Date: 17/Jan/20 09:46 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575553014 @tvalentyn Thanks, I'm aware of this. Right now I'm struggling to make all checks green again This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373499) Time Spent: 9h (was: 8h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 9h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373498&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373498 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 17/Jan/20 09:43 Start Date: 17/Jan/20 09:43 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575551992 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373498) Time Spent: 8h 50m (was: 8h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373169&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373169 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 17:44 Start Date: 16/Jan/20 17:44 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575265944 @kamilwu, once this is ready for the next step review iteration please say PTAL @reviewer_name(,s) or something to make sure reviewers are aware that you are waiting on their input. Thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373169) Time Spent: 8h 40m (was: 8.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373066&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373066 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 15:37 Start Date: 16/Jan/20 15:37 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575208342 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373066) Time Spent: 8.5h (was: 8h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=373034&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-373034 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 14:32 Start Date: 16/Jan/20 14:32 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575177940 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 373034) Time Spent: 8h 20m (was: 8h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372941&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372941 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 11:05 Start Date: 16/Jan/20 11:05 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575101160 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372941) Time Spent: 8h 10m (was: 8h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372932&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372932 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 10:26 Start Date: 16/Jan/20 10:26 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575086095 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372932) Time Spent: 8h (was: 7h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 8h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372883&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372883 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 09:04 Start Date: 16/Jan/20 09:04 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575053178 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372883) Time Spent: 7h 50m (was: 7h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372871&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372871 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 08:50 Start Date: 16/Jan/20 08:50 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575048065 run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372871) Time Spent: 7h 40m (was: 7.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372869&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372869 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 16/Jan/20 08:48 Start Date: 16/Jan/20 08:48 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-575047266 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 372869) Time Spent: 7.5h (was: 7h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=372866&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-372866
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 16/Jan/20 08:43
Start Date: 16/Jan/20 08:43
Worklog Time Spent: 10m
Work Description: kamilwu commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r367291831
##
File path: sdks/python/apache_beam/testing/load_tests/load_test.py
##
@@ -31,8 +31,8 @@ def __init__(self):
self.project_id = self.pipeline.get_option('project')
self.metrics_namespace = self.pipeline.get_option('metrics_table')
self._metrics_monitor = MetricsReader(
-publish_to_bq=self.pipeline.get_option('publish_to_big_query') ==
-'true',
+publish_to_bq=self.pipeline.get_option('publish_to_big_query')
+.lower() == 'true',
project_name=self.project_id,
Review comment:
I created `LoadTestOptions` class with common parameters that load tests can
take. `--publish_to_big_query` is one of them. I hope this will make those
options better organised.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 372866)
Time Spent: 7h 20m (was: 7h 10m)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=370727&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370727 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 13/Jan/20 12:00 Start Date: 13/Jan/20 12:00 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r365767789 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -18,18 +18,65 @@ import json import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): Review comment: No particular benefit. Because `LoadTest` is no longer a subclass of `unittest.TestCase`, I had to move this code somewhere. And the reasons why I stopped using `unittest.TestCase` are: - We had to prevent load tests from being included in other test suites and that's why `LOAD_TEST_ENABLED` guard was used. Now the guard is no longer needed, - `tearDown()` was actually used to run a pipeline, which was misleading, as tearDown is supposed only to clean up resources after test execution, - Without nose or pytest, it is difficult to pass extra command-line arguments. To sum up, I think `unittest` doesn't suit our needs here and can be easily replaced with plain Python classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370727) Time Spent: 7h 10m (was: 7h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=370120&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370120 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 10/Jan/20 22:53 Start Date: 10/Jan/20 22:53 Worklog Time Spent: 10m Work Description: markflyhigh commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r365441572 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -18,18 +18,65 @@ import json import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): Review comment: What's the benefit to move `setUp` to `__init__`? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370120) Time Spent: 7h (was: 6h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=370122&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370122 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 10/Jan/20 22:53 Start Date: 10/Jan/20 22:53 Worklog Time Spent: 10m Work Description: markflyhigh commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r365443222 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: I would suggest to keep `--test-pipeline-options` (either with `TestPipeline` or other parse approach), which provide great flexibility of passing pipeline options from command line. With the plan of moving `nose` to `pytest`, `TestPipeline` will be adopted to `pytest` eventually. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370122) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 7h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=370121&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370121
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 10/Jan/20 22:53
Start Date: 10/Jan/20 22:53
Worklog Time Spent: 10m
Work Description: markflyhigh commented on pull request #10435:
[BEAM-8951] Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r365451894
##
File path: sdks/python/apache_beam/testing/load_tests/load_test.py
##
@@ -31,8 +31,8 @@ def __init__(self):
self.project_id = self.pipeline.get_option('project')
self.metrics_namespace = self.pipeline.get_option('metrics_table')
self._metrics_monitor = MetricsReader(
-publish_to_bq=self.pipeline.get_option('publish_to_big_query') ==
-'true',
+publish_to_bq=self.pipeline.get_option('publish_to_big_query')
+.lower() == 'true',
project_name=self.project_id,
Review comment:
Another way to make those options organized is to register them in
`TestPipelineOption` (recommended if commonly used in performance testing),
which works the same way as parser does.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 370121)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=369089&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369089 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 15:04 Start Date: 09/Jan/20 15:04 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572601311 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369089) Time Spent: 6h 50m (was: 6h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=369025&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369025 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 13:38 Start Date: 09/Jan/20 13:38 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572565159 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369025) Time Spent: 6h 40m (was: 6.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=369024&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369024 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 13:38 Start Date: 09/Jan/20 13:38 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572564810 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369024) Time Spent: 6.5h (was: 6h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=369023&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369023 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 13:35 Start Date: 09/Jan/20 13:35 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572563558 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369023) Time Spent: 6h 20m (was: 6h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=369022&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369022 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 13:33 Start Date: 09/Jan/20 13:33 Worklog Time Spent: 10m Work Description: kkucharc commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572563052 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369022) Time Spent: 6h 10m (was: 6h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368527&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368527 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 09/Jan/20 00:35 Start Date: 09/Jan/20 00:35 Worklog Time Spent: 10m Work Description: Ardagan commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572325201 Run Load Tests Java Combine Portable Flink Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368527) Time Spent: 6h (was: 5h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 6h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368407&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368407 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 21:23 Start Date: 08/Jan/20 21:23 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572262861 ignore this comment This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368407) Time Spent: 5h 50m (was: 5h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368405&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368405 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 21:23 Start Date: 08/Jan/20 21:23 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572262861 ignore this comment This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368405) Time Spent: 5.5h (was: 5h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 5.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368406&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368406 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 21:23 Start Date: 08/Jan/20 21:23 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572262961 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368406) Time Spent: 5h 40m (was: 5.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368201&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368201 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 15:00 Start Date: 08/Jan/20 15:00 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572107908 @tvalentyn Could you trigger the Python 3.7 test once again, please? It should work now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368201) Time Spent: 5h 20m (was: 5h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368200&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368200 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 14:58 Start Date: 08/Jan/20 14:58 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-572107072 Rebased the branch with the latest master. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368200) Time Spent: 5h 10m (was: 5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 5h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368199&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368199
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 08/Jan/20 14:57
Start Date: 08/Jan/20 14:57
Worklog Time Spent: 10m
Work Description: kamilwu commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r364273079
##
File path: sdks/python/apache_beam/testing/load_tests/load_test.py
##
@@ -31,8 +31,8 @@ def __init__(self):
self.project_id = self.pipeline.get_option('project')
self.metrics_namespace = self.pipeline.get_option('metrics_table')
self._metrics_monitor = MetricsReader(
-publish_to_bq=self.pipeline.get_option('publish_to_big_query') ==
-'true',
+publish_to_bq=self.pipeline.get_option('publish_to_big_query')
+.lower() == 'true',
project_name=self.project_id,
Review comment:
I suppose I hadn't understood your suggestion fully, my apologies. It's
definitely worth implementing. I also added try/except block to make an
exception a little more informative for users.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 368199)
Time Spent: 5h (was: 4h 50m)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=368194&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368194 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 08/Jan/20 14:45 Start Date: 08/Jan/20 14:45 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r364266252 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: In this case, I'd rather stick with `--test-pipeline-options` for now. Thanks for the link. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368194) Time Spent: 4h 50m (was: 4h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 4h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367803&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367803 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Jan/20 22:01 Start Date: 07/Jan/20 22:01 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r363956004 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: I spoke with @udim - if we migrate these tests to use pytest later, sticking with `--test-pipeline-options` may ease this migration. we haven't fully paved the path to switch to pytest, but @udim has an (old) in-progress PR https://github.com/apache/beam/pull/7949 in case you are curious to give pytest a try for loadtest purposes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367803) Time Spent: 4h 40m (was: 4.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 4h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367769&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367769 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Jan/20 21:20 Start Date: 07/Jan/20 21:20 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r363956004 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: I spoke with @udim - if we migrate these tests to use pytest later, sticking with `--test-pipeline-options` may ease this migration. we haven't fully paved the path to switch to pytest, but @udim may have some in-progress PRs in case you are curious to give pytest a try for loadtest purposes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367769) Time Spent: 4.5h (was: 4h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 4.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367650&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367650
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 07/Jan/20 18:41
Start Date: 07/Jan/20 18:41
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r363892264
##
File path: sdks/python/apache_beam/testing/load_tests/load_test.py
##
@@ -31,8 +31,8 @@ def __init__(self):
self.project_id = self.pipeline.get_option('project')
self.metrics_namespace = self.pipeline.get_option('metrics_table')
self._metrics_monitor = MetricsReader(
-publish_to_bq=self.pipeline.get_option('publish_to_big_query') ==
-'true',
+publish_to_bq=self.pipeline.get_option('publish_to_big_query')
+.lower() == 'true',
project_name=self.project_id,
Review comment:
My earlier suggestion would also help catch typos, since this flag is not
declared a boolean.
If we end up removing test pipeline options, you could also use something
like:
```
parser.add_argument('--publish_to_big_query', default=False,
type=lambda s: bool(['false',
'true'].index(s.lower())),
```
It is easy to pass an incorrect value of a boolean flag without noticing the
error...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 367650)
Time Spent: 4h 20m (was: 4h 10m)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367646&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367646 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Jan/20 18:37 Start Date: 07/Jan/20 18:37 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r363890457 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: I'd let @markflyhigh comment on which way is more idiomatic/preferable going forward for adding new tests. Without test-pipeline-options we will be declaring options explicitly, it may be easier to see which flags need to be passed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367646) Time Spent: 4h 10m (was: 4h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367644&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367644 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Jan/20 18:35 Start Date: 07/Jan/20 18:35 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-571714311 Run Python 3.7 Load Tests ParDo Dataflow Batch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367644) Time Spent: 4h (was: 3h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 4h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367403&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367403 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 07/Jan/20 10:31 Start Date: 07/Jan/20 10:31 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r363685198 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: It's just a matter of passing pipeline options in a different way (without `--test-pipeline-options` command line argument). Do you think it might be beneficial to use regular pipeline instead of TestPipeline? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367403) Time Spent: 3h 50m (was: 3h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 3h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=367401&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367401
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 07/Jan/20 10:23
Start Date: 07/Jan/20 10:23
Worklog Time Spent: 10m
Work Description: kamilwu commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r363681664
##
File path:
sdks/python/apache_beam/testing/load_tests/streaming/group_by_key_streaming_test.py
##
@@ -62,12 +56,11 @@ def _add_argparse_args(cls, parser):
parser.add_argument('--test-pipeline-options')
-@unittest.skipIf(not load_test_enabled, 'Enabled only for phase triggering.')
class GroupByKeyStreamingTest(LoadTest):
ID_LABEL = 'id'
- def setUp(self):
-super(GroupByKeyStreamingTest, self).setUp()
+ def __init__(self):
+super(GroupByKeyStreamingTest, self).__init__()
Review comment:
No, `save_main_session` is not required in any of these tests.
We have one ParDo test running on Python 3 on Dataflow that we can run to
make sure everything works with these changes. Could you run it by typing `Run
Python 3.7 Load Tests ParDo Dataflow Batch`? I'm afraid that I lost an ability
to run Jenkins tests as a non-commiter.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 367401)
Time Spent: 3h 40m (was: 3.5h)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=366924&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-366924 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 06/Jan/20 20:09 Start Date: 06/Jan/20 20:09 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#discussion_r363461573 ## File path: sdks/python/apache_beam/testing/load_tests/load_test.py ## @@ -17,19 +17,50 @@ from __future__ import absolute_import import json -import logging -import unittest from apache_beam.metrics import MetricsFilter from apache_beam.testing.load_tests.load_test_metrics_utils import MetricsReader from apache_beam.testing.test_pipeline import TestPipeline -class LoadTest(unittest.TestCase): - def parseTestPipelineOptions(self, options=None): +class LoadTest(object): + def __init__(self): +self.pipeline = TestPipeline(is_integration_test=True) Review comment: If we proceed with this change, do we still need to use TestPipeline as opposed to regular pipeline? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 366924) Time Spent: 3.5h (was: 3h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 3.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=366922&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-366922
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 06/Jan/20 20:05
Start Date: 06/Jan/20 20:05
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r363452643
##
File path:
sdks/python/apache_beam/testing/load_tests/streaming/group_by_key_streaming_test.py
##
@@ -62,12 +56,11 @@ def _add_argparse_args(cls, parser):
parser.add_argument('--test-pipeline-options')
-@unittest.skipIf(not load_test_enabled, 'Enabled only for phase triggering.')
class GroupByKeyStreamingTest(LoadTest):
ID_LABEL = 'id'
- def setUp(self):
-super(GroupByKeyStreamingTest, self).setUp()
+ def __init__(self):
+super(GroupByKeyStreamingTest, self).__init__()
Review comment:
Does running these tests on Dataflow runner require --save_main_session?
Heads-up that there is https://issues.apache.org/jira/browse/BEAM-6158 which
may affect the execution on Python 3. Please check that we can run these tests
on Python 3 on Dataflow (you can check on any test that is supported by
Dataflow Runner).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 366922)
Time Spent: 3h 20m (was: 3h 10m)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=366923&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-366923
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 06/Jan/20 20:05
Start Date: 06/Jan/20 20:05
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r363434067
##
File path: sdks/python/apache_beam/testing/load_tests/combine_test.py
##
@@ -59,62 +55,10 @@
--input_options=\'
{"num_records": 1,
"key_size": 1,
- "value_size":1,
- "bundle_size_distribution_type": "const",
- "bundle_size_distribution_param": 1,
- "force_initial_num_bundles": 1}\'
---runner=DirectRunner
---fanout=1
---top_count=1000' \
+ "value_size": 1}\'
Review comment:
The gradle command is not copy-pasteable. Python command is. Did not check
commands in other files.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 366923)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[
https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=366921&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-366921
]
ASF GitHub Bot logged work on BEAM-8951:
Author: ASF GitHub Bot
Created on: 06/Jan/20 20:05
Start Date: 06/Jan/20 20:05
Worklog Time Spent: 10m
Work Description: tvalentyn commented on pull request #10435: [BEAM-8951]
Stop using nose in load tests
URL: https://github.com/apache/beam/pull/10435#discussion_r363449250
##
File path: sdks/python/apache_beam/testing/load_tests/load_test.py
##
@@ -17,19 +17,50 @@
from __future__ import absolute_import
import json
-import logging
-import unittest
from apache_beam.metrics import MetricsFilter
from apache_beam.testing.load_tests.load_test_metrics_utils import
MetricsReader
from apache_beam.testing.test_pipeline import TestPipeline
-class LoadTest(unittest.TestCase):
- def parseTestPipelineOptions(self, options=None):
+class LoadTest(object):
+ def __init__(self):
+self.pipeline = TestPipeline(is_integration_test=True)
+self.input_options = json.loads(self.pipeline.get_option('input_options')
+or '{}')
+self.project_id = self.pipeline.get_option('project')
+self.metrics_namespace = self.pipeline.get_option('metrics_table')
+self._metrics_monitor = MetricsReader(
+publish_to_bq=self.pipeline.get_option('publish_to_big_query') ==
+'true',
Review comment:
Let's strengthen this to check for possible misspellings. `publish_to_bq =
bool(['false',
'true'].index(self.pipeline.get_option('publish_to_big_query').lower()))`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 366921)
Time Spent: 3h 20m (was: 3h 10m)
> Stop using nose in load tests
> -
>
> Key: BEAM-8951
> URL: https://issues.apache.org/jira/browse/BEAM-8951
> Project: Beam
> Issue Type: Sub-task
> Components: testing
>Reporter: Kamil Wasilewski
>Assignee: Kamil Wasilewski
>Priority: Major
> Fix For: Not applicable
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The community is considering moving away from nose to pytest:
> https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of
> running Python load tests: instead of being subclasses of
> `unittest.TestCase`, they could be plain Python scripts, just like wordcount
> examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard
> will be no longer needed and could be safely removed
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365824&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365824 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 03/Jan/20 12:16 Start Date: 03/Jan/20 12:16 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570556994 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365824) Time Spent: 3h 10m (was: 3h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 3h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365731&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365731 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 03/Jan/20 08:53 Start Date: 03/Jan/20 08:53 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570507181 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365731) Time Spent: 3h (was: 2h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 3h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365330&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365330 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 11:01 Start Date: 02/Jan/20 11:01 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570176111 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365330) Time Spent: 2h 50m (was: 2h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365321&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365321 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:40 Start Date: 02/Jan/20 10:40 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570171685 > With nose, I think we had to configure output collectors via xml files I think it doesn't really matter in this case since we don't run load tests in parallel. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365321) Time Spent: 2h 40m (was: 2.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365317&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365317 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:30 Start Date: 02/Jan/20 10:30 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570169366 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365317) Time Spent: 2h 10m (was: 2h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365319&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365319 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:30 Start Date: 02/Jan/20 10:30 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570167238 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365319) Time Spent: 2.5h (was: 2h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365318&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365318 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:30 Start Date: 02/Jan/20 10:30 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570165541 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365318) Time Spent: 2h 20m (was: 2h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365313&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365313 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:21 Start Date: 02/Jan/20 10:21 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570167238 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365313) Time Spent: 2h (was: 1h 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 2h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=365310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-365310 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 02/Jan/20 10:13 Start Date: 02/Jan/20 10:13 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-570165541 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 365310) Time Spent: 1h 50m (was: 1h 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=362017&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-362017 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 21/Dec/19 03:13 Start Date: 21/Dec/19 03:13 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-568147273 Didn't have time to take a look today and I am planning to be out next week. @udim have converted several suites to pytest recently and may have some feedback here. With nose, I think we had to configure output collectors via xml files, see: https://github.com/apache/beam/blob/754b64b4a59f717d84032570acb8ed4cad87b227/sdks/python/scripts/run_integration_test.sh#L248 , I have not yet learned how change output collection with pytest. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 362017) Time Spent: 1h 40m (was: 1.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361691&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361691 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 15:28 Start Date: 20/Dec/19 15:28 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567963470 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361691) Time Spent: 1.5h (was: 1h 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361686&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361686 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 15:09 Start Date: 20/Dec/19 15:09 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567957020 R: @tvalentyn Can I ask you for a review? Without nose, it was much easier to include logging records to console output (https://issues.apache.org/jira/browse/BEAM-8997). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361686) Time Spent: 1h 20m (was: 1h 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361681 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 14:53 Start Date: 20/Dec/19 14:53 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567951637 Run Python Load Tests Smoke This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361681) Time Spent: 1h (was: 50m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361682&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361682 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 14:53 Start Date: 20/Dec/19 14:53 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567950999 Run Python Load Tests Smoke This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361682) Time Spent: 1h 10m (was: 1h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 10m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361680&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361680 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 14:51 Start Date: 20/Dec/19 14:51 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567950999 Run Python Load Tests Smoke This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361680) Time Spent: 50m (was: 40m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 50m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361672&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361672 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 14:31 Start Date: 20/Dec/19 14:31 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567944308 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361672) Time Spent: 40m (was: 0.5h) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 40m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361669&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361669 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 14:21 Start Date: 20/Dec/19 14:21 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567940740 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361669) Time Spent: 0.5h (was: 20m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 0.5h > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361660&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361660 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 13:59 Start Date: 20/Dec/19 13:59 Worklog Time Spent: 10m Work Description: kamilwu commented on issue #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435#issuecomment-567933566 Run Python Load Tests Smoke This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 361660) Time Spent: 20m (was: 10m) > Stop using nose in load tests > - > > Key: BEAM-8951 > URL: https://issues.apache.org/jira/browse/BEAM-8951 > Project: Beam > Issue Type: Sub-task > Components: testing >Reporter: Kamil Wasilewski >Assignee: Kamil Wasilewski >Priority: Major > Fix For: Not applicable > > Time Spent: 20m > Remaining Estimate: 0h > > The community is considering moving away from nose to pytest: > https://issues.apache.org/jira/browse/BEAM-3713. We should change the way of > running Python load tests: instead of being subclasses of > `unittest.TestCase`, they could be plain Python scripts, just like wordcount > examples. This will bring one additional benefit: _LOAD_TEST_ENABLED_ guard > will be no longer needed and could be safely removed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8951) Stop using nose in load tests
[ https://issues.apache.org/jira/browse/BEAM-8951?focusedWorklogId=361659&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-361659 ] ASF GitHub Bot logged work on BEAM-8951: Author: ASF GitHub Bot Created on: 20/Dec/19 13:55 Start Date: 20/Dec/19 13:55 Worklog Time Spent: 10m Work Description: kamilwu commented on pull request #10435: [BEAM-8951] Stop using nose in load tests URL: https://github.com/apache/beam/pull/10435 Beam is moving away from nose. We could change the way of running Python load tests: instead of being subclasses of `unittest.TestCase`, they could be plain Python scripts, just like wordcount examples. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) Python | [