Hello. I fixed two system tests that fails in trunk, also.
streams_upgrade_test.py::StreamsUpgradeTest.test_version_probing_upgrade streams_static_membership_test.py Please, take a look at my PR [1] [1] https://github.com/apache/kafka/pull/9312 > 20 сент. 2020 г., в 06:11, Guozhang Wang <wangg...@gmail.com> написал(а): > > I've triggered a system test on top of your branch. > > Maybe you could also re-run the jenkins unit tests since currently all of > them fails but you've only touched on system tests, so I'd like to confirm > at least one successful run. > > On Wed, Sep 16, 2020 at 3:37 AM Nikolay Izhikov <nizhi...@apache.org> wrote: > >> Hello, Guozhang. >> >>> I can help run the test suite once your PR is cleanly rebased to verify >> the whole suite works >> >> Thank you for joining to the review. >> >> 1. PR rebased on the current trunk. >> >> 2. I triggered all tests in my private environment to verify them after >> rebase. >> Will inform you once tests passed on my environment. >> >> 3. We need a new ducktape release [1] to be able to merge PR [2]. >> For now, PR based on the ducktape trunk branch [3], not some >> specific release. >> If ducktape team need any help with the release, please, let me >> know. >> >> [1] https://github.com/confluentinc/ducktape/issues/245 >> [2] https://github.com/apache/kafka/pull/9196 >> [3] >> https://github.com/apache/kafka/pull/9196/files#diff-9235a7bdb1ca9268681c0e56f3f3609bR39 >> >>> 16 сент. 2020 г., в 07:32, Guozhang Wang <wangg...@gmail.com> >> написал(а): >>> >>> Hello Nikolay, >>> >>> I can help run the test suite once your PR is cleanly rebased to verify >> the >>> whole suite works and then I can merge (I'm trusting Ivan and Magnus here >>> for their reviews :) >>> >>> Guozhang >>> >>> On Mon, Sep 14, 2020 at 3:56 AM Nikolay Izhikov <nizhi...@apache.org> >> wrote: >>> >>>> Hello! >>>> >>>> I got 2 approvals from Ivan Daschinskiy and Magnus Edenhill. >>>> Committers, please, join the review. >>>> >>>>> 3 сент. 2020 г., в 11:06, Nikolay Izhikov <nizhikov....@gmail.com> >>>> написал(а): >>>>> >>>>> Hello! >>>>> >>>>> Just a friendly reminder. >>>>> >>>>> Patch to resolve some kind of technical debt - python2 in system tests >>>> is ready! >>>>> Can someone, please, take a look? >>>>> >>>>> https://github.com/apache/kafka/pull/9196 >>>>> >>>>>> 28 авг. 2020 г., в 11:19, Nikolay Izhikov <nizhikov....@gmail.com> >>>> написал(а): >>>>>> >>>>>> Hello! >>>>>> >>>>>> Any feedback on this? >>>>>> What I should additionally do to prepare system tests migration? >>>>>> >>>>>>> 24 авг. 2020 г., в 11:17, Nikolay Izhikov <nizhikov....@gmail.com> >>>> написал(а): >>>>>>> >>>>>>> Hello. >>>>>>> >>>>>>> PR [1] is ready. >>>>>>> Please, review. >>>>>>> >>>>>>> But, I need help with the two following questions: >>>>>>> >>>>>>> 1. We need a new release of ducktape which includes fixes [2], [3] >> for >>>> python3. >>>>>>> I created the issue in ducktape repo [4]. >>>>>>> Can someone help me with the release? >>>>>>> >>>>>>> 2. I know that some companies run system tests for the trunk on a >>>> regular bases. >>>>>>> Can someone show me some results of these runs? >>>>>>> So, I can compare failures in my PR and in the trunk. >>>>>>> >>>>>>> Results [5] of run all for my PR available in the ticket [6] >>>>>>> >>>>>>> ``` >>>>>>> SESSION REPORT (ALL TESTS) >>>>>>> ducktape version: 0.8.0 >>>>>>> session_id: 2020-08-23--002 >>>>>>> run time: 1010 minutes 46.483 seconds >>>>>>> tests run: 684 >>>>>>> passed: 505 >>>>>>> failed: 9 >>>>>>> ignored: 170 >>>>>>> ``` >>>>>>> >>>>>>> [1] https://github.com/apache/kafka/pull/9196 >>>>>>> [2] >>>> >> https://github.com/confluentinc/ducktape/commit/23bd5ab53802e3a1e1da1ddf3630934f33b02305 >>>>>>> [3] >>>> >> https://github.com/confluentinc/ducktape/commit/bfe53712f83b025832d29a43cde3de3d7803106f >>>>>>> [4] https://github.com/confluentinc/ducktape/issues/245 >>>>>>> [5] >>>> https://issues.apache.org/jira/secure/attachment/13010366/report.txt >>>>>>> [6] https://issues.apache.org/jira/browse/KAFKA-10402 >>>>>>> >>>>>>>> 14 авг. 2020 г., в 21:26, Ismael Juma <ism...@juma.me.uk> >> написал(а): >>>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> On Fri, Aug 14, 2020 at 7:42 AM John Roesler <vvcep...@apache.org> >>>> wrote: >>>>>>>> >>>>>>>>> Thanks Nikolay, >>>>>>>>> >>>>>>>>> No objection. This would be very nice to have. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> John >>>>>>>>> >>>>>>>>> On Fri, Aug 14, 2020, at 09:18, Nikolay Izhikov wrote: >>>>>>>>>> Hello. >>>>>>>>>> >>>>>>>>>>> If anyone's interested in porting it to Python 3 it would be a >> good >>>>>>>>> change. >>>>>>>>>> >>>>>>>>>> I’ve created a ticket [1] to upgrade system tests to python3. >>>>>>>>>> Does someone have any additional inputs or objections for this >>>> change? >>>>>>>>>> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/KAFKA-10402 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> 1 июля 2020 г., в 00:26, Gokul Ramanan Subramanian < >>>>>>>>> gokul24...@gmail.com> написал(а): >>>>>>>>>>> >>>>>>>>>>> Thanks Colin. >>>>>>>>>>> >>>>>>>>>>> While at the subject of system tests, there are a few times I see >>>> tests >>>>>>>>>>> timed out (even on a large machine such as m5.4xlarge EC2 with >>>> Linux). >>>>>>>>> Are >>>>>>>>>>> there any knobs that system tests provide to control timeouts / >>>>>>>>> throughputs >>>>>>>>>>> across all tests? >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> On Tue, Jun 30, 2020 at 6:32 PM Colin McCabe <cmcc...@apache.org >>> >>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Ducktape runs on Python 2. You can't use it with Python 3, as >>>> you are >>>>>>>>>>>> trying to do here. >>>>>>>>>>>> >>>>>>>>>>>> If anyone's interested in porting it to Python 3 it would be a >>>> good >>>>>>>>> change. >>>>>>>>>>>> >>>>>>>>>>>> Otherwise, using docker as suggested here seems to be the best >>>> way to >>>>>>>>> go. >>>>>>>>>>>> >>>>>>>>>>>> best, >>>>>>>>>>>> Colin >>>>>>>>>>>> >>>>>>>>>>>> On Mon, Jun 29, 2020, at 02:14, Gokul Ramanan Subramanian wrote: >>>>>>>>>>>>> Hi. >>>>>>>>>>>>> >>>>>>>>>>>>> Has anyone had luck running Kafka system tests on a Mac. I >> have a >>>>>>>>> MacOS >>>>>>>>>>>>> Mojave 10.14.6. I got Python 3.6.9 using pyenv. However, the >>>> command >>>>>>>>>>>>> *ducktape tests/kafkatest/tests* yields the following error, >>>> making >>>>>>>>> it >>>>>>>>>>>> look >>>>>>>>>>>>> like some Python incompatibility issue. >>>>>>>>>>>>> >>>>>>>>>>>>> $ ducktape tests/kafkatest/tests >>>>>>>>>>>>> Traceback (most recent call last): >>>>>>>>>>>>> File "/Users/gokusubr/.pyenv/versions/3.6.9/bin/ducktape", line >>>> 11, >>>>>>>>> in >>>>>>>>>>>>> <module> >>>>>>>>>>>>> load_entry_point('ducktape', 'console_scripts', 'ducktape')() >>>>>>>>>>>>> File >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>> line 487, in load_entry_point >>>>>>>>>>>>> return get_distribution(dist).load_entry_point(group, name) >>>>>>>>>>>>> File >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>> line 2728, in load_entry_point >>>>>>>>>>>>> return ep.load() >>>>>>>>>>>>> File >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>> line 2346, in load >>>>>>>>>>>>> return self.resolve() >>>>>>>>>>>>> File >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/pkg_resources/__init__.py", >>>>>>>>>>>>> line 2352, in resolve >>>>>>>>>>>>> module = __import__(self.module_name, fromlist=['__name__'], >>>>>>>>>>>>> level=0) >>>>>>>>>>>>> File >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>> >>>> >> "/Users/gokusubr/.pyenv/versions/3.6.9/lib/python3.6/site-packages/ducktape-0.7.6-py3.6.egg/ducktape/command_line/main.py", >>>>>>>>>>>>> line 127 >>>>>>>>>>>>> print "parameters are not valid json: " + str(e.message) >>>>>>>>>>>>> ^ >>>>>>>>>>>>> SyntaxError: invalid syntax >>>>>>>>>>>>> >>>>>>>>>>>>> I followed the instructions in tests/README.md to setup a >>>> cluster of >>>>>>>>> 9 >>>>>>>>>>>>> worker machines. That worked well. When I ran *python setup.py >>>>>>>>> develop* >>>>>>>>>>>> to >>>>>>>>>>>>> install the necessary dependencies (including ducktape), I got >>>>>>>>> similar >>>>>>>>>>>>> errors to above, but the overall command completed >> successfully. >>>>>>>>>>>>> >>>>>>>>>>>>> Any help appreciated. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks. >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> -- >>> -- Guozhang >> >> > > -- > -- Guozhang
