So this test is assuming that a ParDo is only deserialized once? This is not a safe assumption, and might be broken by any number of changes. Is there any other way we could structure this test?
On Fri, Feb 22, 2019 at 9:47 AM Michael Luckey <adude3...@gmail.com> wrote: > Sorry for spamming... > > referring to this id here > > > https://github.com/apache/beam/blob/master/sdks/java/io/kudu/src/test/java/org/apache/beam/sdk/io/kudu/KuduIOTest.java#L206 > > On Fri, Feb 22, 2019 at 6:41 PM Michael Luckey <adude3...@gmail.com> > wrote: > >> Might have to do with latest schema changes >> >> at >> org.apache.beam.sdk.util.SerializableUtils.deserializeFromByteArray(SerializableUtils.java:71) >> at >> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.doFnWithExecutionInformationFromProto(ParDoTranslation.java:581) >> at >> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:311) >> at >> org.apache.beam.repackaged.beam_runners_direct_java.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:296) >> at >> org.apache.beam.runners.direct.ParDoEvaluatorFactory.forApplication(ParDoEvaluatorFactory.java:86) >> at >> org.apache.beam.runners.direct.TransformEvaluatorRegistry.forApplication(TransformEvaluatorRegistry.java:169) >> at >> org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:117) >> >> This seems to do serialisations, which increments that private static id >> field before actually deserialising the instance used within test >> >> On Fri, Feb 22, 2019 at 6:36 PM Michael Luckey <adude3...@gmail.com> >> wrote: >> >>> It does not look flaky to me. It seems to be failing all the time. Did >>> anything change with serialisation? Because test seems to expect entities >>> with id = '1', whereas the actual used instance within pipeline starts with >>> an id of 11 on my machine... >>> >>> >>> https://github.com/apache/beam/blob/master/sdks/java/io/kudu/src/test/java/org/apache/beam/sdk/io/kudu/KuduIOTest.java#L148 >>> >>> On Fri, Feb 22, 2019 at 6:26 PM Reuven Lax <re...@google.com> wrote: >>> >>>> It's a precommit test. Should we disable this test until it can be >>>> diagnosed? >>>> >>>> On Thu, Feb 21, 2019 at 2:25 PM Kenneth Knowles <k...@google.com> wrote: >>>> >>>>> I don't see it in the postcommit history: >>>>> https://builds.apache.org/job/beam_PreCommit_Java_Cron/977/testReport/org.apache.beam.sdk.io.kudu/KuduIOTest/history/ >>>>> >>>>> (URL assembled by trial and error and from Mikhail's prior sharing) >>>>> >>>>> Kenn >>>>> >>>>> On Thu, Feb 21, 2019 at 12:59 PM Reuven Lax <re...@google.com> wrote: >>>>> >>>>>> I'm finding the KuduIO tests to be extremely flaky. I've just ran >>>>>> Java Presubmit three times in a row, and each time the KuduIO tests >>>>>> failed. >>>>>> How can we improve this situation? >>>>>> >>>>>> Example failure: >>>>>> >>>>>> >>>>>> https://builds.apache.org/job/beam_PreCommit_Java_Phrase/754/testReport/junit/org.apache.beam.sdk.io.kudu/KuduIOTest/testWrite/ >>>>>> >>>>>
