[jira] [Commented] (BEAM-3028) DatastoreIO integration tests failing on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197796#comment-16197796
]
Chamikara Jayalath commented on BEAM-3028:
--
Fix is in. Seems like post commits are failing currently for unrelated reasons.
But this test passed for me locally with the fix. So closing this JIRA.
> DatastoreIO integration tests failing on Dataflow
> -
>
> Key: BEAM-3028
> URL: https://issues.apache.org/jira/browse/BEAM-3028
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
>Affects Versions: 2.2.0
>Reporter: Kenneth Knowles
>Assignee: Chamikara Jayalath
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Our testing signal has been bad enough that we seemed to have ignored or
> missed this when it happened.
> From these builds: (while the logs last, you can check them)
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1Read/
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1ReadWithGQLQueryWithNoLimit/
> Quoting:
> {code}
> java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
> Expected: <1000L>
> but: was <12000L>
> at
> org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:165)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:397)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:387)
> at
> org.apache.beam.sdk.io.gcp.datastore.V1ReadIT.testE2EV1Read(V1ReadIT.java:95)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-3028) DatastoreIO integration tests failing on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16197693#comment-16197693
]
ASF GitHub Bot commented on BEAM-3028:
--
Github user asfgit closed the pull request at:
https://github.com/apache/beam/pull/3962
> DatastoreIO integration tests failing on Dataflow
> -
>
> Key: BEAM-3028
> URL: https://issues.apache.org/jira/browse/BEAM-3028
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
>Affects Versions: 2.2.0
>Reporter: Kenneth Knowles
>Assignee: Chamikara Jayalath
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Our testing signal has been bad enough that we seemed to have ignored or
> missed this when it happened.
> From these builds: (while the logs last, you can check them)
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1Read/
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1ReadWithGQLQueryWithNoLimit/
> Quoting:
> {code}
> java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
> Expected: <1000L>
> but: was <12000L>
> at
> org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:165)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:397)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:387)
> at
> org.apache.beam.sdk.io.gcp.datastore.V1ReadIT.testE2EV1Read(V1ReadIT.java:95)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-3028) DatastoreIO integration tests failing on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196381#comment-16196381
]
Chamikara Jayalath commented on BEAM-3028:
--
https://github.com/apache/beam/pull/3962
> DatastoreIO integration tests failing on Dataflow
> -
>
> Key: BEAM-3028
> URL: https://issues.apache.org/jira/browse/BEAM-3028
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
>Affects Versions: 2.2.0
>Reporter: Kenneth Knowles
>Assignee: Chamikara Jayalath
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Our testing signal has been bad enough that we seemed to have ignored or
> missed this when it happened.
> From these builds: (while the logs last, you can check them)
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1Read/
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1ReadWithGQLQueryWithNoLimit/
> Quoting:
> {code}
> java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
> Expected: <1000L>
> but: was <12000L>
> at
> org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:165)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:397)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:387)
> at
> org.apache.beam.sdk.io.gcp.datastore.V1ReadIT.testE2EV1Read(V1ReadIT.java:95)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-3028) DatastoreIO integration tests failing on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16196374#comment-16196374
]
Chamikara Jayalath commented on BEAM-3028:
--
Issues is at following location.
https://github.com/apache/beam/blob/master/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/datastore/DatastoreV1.java#L810
When splitting the query we output the original query instead of the
sub-queries.
Will send out a PR.
> DatastoreIO integration tests failing on Dataflow
> -
>
> Key: BEAM-3028
> URL: https://issues.apache.org/jira/browse/BEAM-3028
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
>Affects Versions: 2.2.0
>Reporter: Kenneth Knowles
>Assignee: Chamikara Jayalath
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Our testing signal has been bad enough that we seemed to have ignored or
> missed this when it happened.
> From these builds: (while the logs last, you can check them)
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1Read/
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1ReadWithGQLQueryWithNoLimit/
> Quoting:
> {code}
> java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
> Expected: <1000L>
> but: was <12000L>
> at
> org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:165)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:397)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:387)
> at
> org.apache.beam.sdk.io.gcp.datastore.V1ReadIT.testE2EV1Read(V1ReadIT.java:95)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (BEAM-3028) DatastoreIO integration tests failing on Dataflow
[
https://issues.apache.org/jira/browse/BEAM-3028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16195434#comment-16195434
]
Chamikara Jayalath commented on BEAM-3028:
--
I manually tried running this test several times and each time the actual
number of entities written was different but was a multiple of the actual
number of entities expected (1000).
Expected: <1000L>
but: was <1L>
at
org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:344)
Expected: <1000L>
but: was <4000L>
at
org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:344)
java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
Expected: <1000L>
but: was <6000L>
at
org.apache.beam.runners.direct.DirectRunner$DirectPipelineResult.waitUntilFinish(DirectRunner.java:344)
CCing some folks who updated Datastore IT recently.[~vikasrk] [~cphipps] [~jkff]
> DatastoreIO integration tests failing on Dataflow
> -
>
> Key: BEAM-3028
> URL: https://issues.apache.org/jira/browse/BEAM-3028
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
>Affects Versions: 2.2.0
>Reporter: Kenneth Knowles
>Assignee: Chamikara Jayalath
>Priority: Blocker
> Fix For: 2.2.0
>
>
> Our testing signal has been bad enough that we seemed to have ignored or
> missed this when it happened.
> From these builds: (while the logs last, you can check them)
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1Read/
> https://builds.apache.org/job/beam_PostCommit_Java_MavenInstall/4963/org.apache.beam$beam-runners-google-cloud-dataflow-java/testReport/junit/org.apache.beam.sdk.io.gcp.datastore/V1ReadIT/testE2EV1ReadWithGQLQueryWithNoLimit/
> Quoting:
> {code}
> java.lang.AssertionError: Combine.globally(Count)/Flatten.PCollections.out:
> Expected: <1000L>
> but: was <12000L>
> at
> org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:165)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:397)
> at org.apache.beam.sdk.testing.PAssert.thatSingleton(PAssert.java:387)
> at
> org.apache.beam.sdk.io.gcp.datastore.V1ReadIT.testE2EV1Read(V1ReadIT.java:95)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
