Something about this, yes. Agree, the tests implementation should be changed. So I d say we should disable this write test and fix on separate jira.
Currently assuming this commit [1] to have introduced the additional serde, which reveals that issue. [1] https://github.com/apache/beam/pull/7635 On Fri, Feb 22, 2019 at 7:11 PM Reuven Lax <re...@google.com> wrote: > So this test is assuming that a ParDo is only deserialized once? This is > not a safe assumption, and might be broken by any number of changes. Is > there any other way we could structure this test? > > On Fri, Feb 22, 2019 at 9:47 AM Michael Luckey <adude3...@gmail.com> > wrote: > >> Sorry for spamming... >> >> referring to this id here >> >> >> https://github.com/apache/beam/blob/master/sdks/java/io/kudu/src/test/java/org/apache/beam/sdk/io/kudu/KuduIOTest.java#L206 >> >> On Fri, Feb 22, 2019 at 6:41 PM Michael Luckey <adude3...@gmail.com> >> wrote: >> >>> Might have to do with latest schema changes >>> >>> at >>> org.apache.beam.sdk.util.SerializableUtils.deserializeFromByteArray(SerializableUtils.java:71) >>> at >>> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.doFnWithExecutionInformationFromProto(ParDoTranslation.java:581) >>> at >>> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:311) >>> at >>> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:296) >>> at >>> org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication(ParDoEvaluatorFactory.java:86) >>> at >>> org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication(TransformEvaluatorRegistry.java:169) >>> at >>> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:117) >>> >>> This seems to do serialisations, which increments that private static id >>> field before actually deserialising the instance used within test >>> >>> On Fri, Feb 22, 2019 at 6:36 PM Michael Luckey <adude3...@gmail.com> >>> wrote: >>> >>>> It does not look flaky to me. It seems to be failing all the time. Did >>>> anything change with serialisation? Because test seems to expect entities >>>> with id = '1', whereas the actual used instance within pipeline starts with >>>> an id of 11 on my machine... >>>> >>>> >>>> https://github.com/apache/beam/blob/master/sdks/java/io/kudu/src/test/java/org/apache/beam/sdk/io/kudu/KuduIOTest.java#L148 >>>> >>>> On Fri, Feb 22, 2019 at 6:26 PM Reuven Lax <re...@google.com> wrote: >>>> >>>>> It's a precommit test. Should we disable this test until it can be >>>>> diagnosed? >>>>> >>>>> On Thu, Feb 21, 2019 at 2:25 PM Kenneth Knowles <k...@google.com> >>>>> wrote: >>>>> >>>>>> I don't see it in the postcommit history: >>>>>> https://builds.apache.org/job/beam_PreCommit_Java_Cron/977/testReport/org.apache.beam.sdk.io.kudu/KuduIOTest/history/ >>>>>> >>>>>> (URL assembled by trial and error and from Mikhail's prior sharing) >>>>>> >>>>>> Kenn >>>>>> >>>>>> On Thu, Feb 21, 2019 at 12:59 PM Reuven Lax <re...@google.com> wrote: >>>>>> >>>>>>> I'm finding the KuduIO tests to be extremely flaky. I've just ran >>>>>>> Java Presubmit three times in a row, and each time the KuduIO tests >>>>>>> failed. >>>>>>> How can we improve this situation? >>>>>>> >>>>>>> Example failure: >>>>>>> >>>>>>> >>>>>>> https://builds.apache.org/job/beam_PreCommit_Java_Phrase/754/testReport/junit/org.apache.beam.sdk.io.kudu/KuduIOTest/testWrite/ >>>>>>> >>>>>>
