See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1445/display/redirect?page=changes>
Changes: [markliu] [BEAM-5953] Fix DataflowRunner in Python 3 - type errors [github] [BEAM-5446] SplittableDoFn: Remove "internal" methods for public API [lcwik] [BEAM-6037] Make Spark runner pipeline translation based on URNs (#7005) [lcwik] [BEAM-3608] Vendor guava 20.0 (#6809) ------------------------------------------ [...truncated 19.43 MB...] Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView as step s9 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Create123/Read(CreateSource) as step s10 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding OutputSideInputs as step s11 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/Window.Into()/Window.Assign as step s12 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) as step s13 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map as step s14 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign as step s15 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey as step s16 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map as step s17 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/RewindowActuals/Window.Assign as step s18 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map as step s19 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/RemoveActualsTriggering/Flatten.PCollections as step s20 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/Create.Values/Read(CreateSource) as step s21 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign as step s22 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/RemoveDummyTriggering/Flatten.PCollections as step s23 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/FlattenDummyAndContents as step s24 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/NeverTrigger/Flatten.PCollections as step s25 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/GroupDummyAndContents as step s26 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/Values/Values/Map as step s27 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GroupGlobally/ParDo(Concat) as step s28 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/GetPane/Map as step s29 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/RunChecks as step s30 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding PAssert$33/VerifyAssertions/ParDo(DefaultConclude) as step s31 Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-1113023406-60141c80/output/results/staging/ Nov 13, 2018 2:34:12 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <70577 bytes, hash xitLlFoHkVj_Ko_aNww-2g> to gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-1113023406-60141c80/output/results/staging/pipeline-xitLlFoHkVj_Ko_aNww-2g.pb org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT Dataflow SDK version: 2.9.0-SNAPSHOT org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR Nov 13, 2018 2:34:13 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-12_18_34_12-17741057379330836219?project=apache-beam-testing org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT Submitted job: 2018-11-12_18_34_12-17741057379330836219 org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR Nov 13, 2018 2:34:13 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-11-12_18_34_12-17741057379330836219 Nov 13, 2018 2:34:13 AM org.apache.beam.runners.dataflow.TestDataflowRunner run INFO: Running Dataflow job 2018-11-12_18_34_12-17741057379330836219 with 1 expected assertions. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:12.737Z: Autoscaling is enabled for job 2018-11-12_18_34_12-17741057379330836219. The number of workers will be between 1 and 1000. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:12.775Z: Autoscaling was automatically enabled for job 2018-11-12_18_34_12-17741057379330836219. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:15.450Z: Checking permissions granted to controller Service Account. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:22.378Z: Worker configuration: n1-standard-1 in us-central1-b. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:22.892Z: Expanding CoGroupByKey operations into optimizable parts. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.044Z: Expanding GroupByKey operations into optimizable parts. Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.086Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.260Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.301Z: Elided trivial flatten Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.338Z: Elided trivial flatten Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.375Z: Elided trivial flatten Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.416Z: Unzipping flatten s24 for input s19.org.apache.beam.sdk.values.PCollection.<init>:402#f234eb3f2c0e3f86 Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.458Z: Fusing unzipped copy of PAssert$33/GroupGlobally/GroupDummyAndContents/Reify, through flatten PAssert$33/GroupGlobally/FlattenDummyAndContents, into producer PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.496Z: Fusing consumer PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow into PAssert$33/GroupGlobally/GroupDummyAndContents/Read Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.530Z: Fusing consumer PAssert$33/GroupGlobally/ParDo(Concat) into PAssert$33/GroupGlobally/Values/Values/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.569Z: Fusing consumer PAssert$33/GetPane/Map into PAssert$33/GroupGlobally/ParDo(Concat) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.606Z: Fusing consumer PAssert$33/GroupGlobally/Values/Values/Map into PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.639Z: Fusing consumer PAssert$33/RunChecks into PAssert$33/GetPane/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.678Z: Fusing consumer PAssert$33/VerifyAssertions/ParDo(DefaultConclude) into PAssert$33/RunChecks Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.717Z: Unzipping flatten s24-u45 for input s26-reify-value18-c43 Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.759Z: Fusing unzipped copy of PAssert$33/GroupGlobally/GroupDummyAndContents/Write, through flatten s24-u45, into producer PAssert$33/GroupGlobally/GroupDummyAndContents/Reify Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.791Z: Fusing consumer PAssert$33/GroupGlobally/GroupDummyAndContents/Reify into PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.816Z: Fusing consumer PAssert$33/GroupGlobally/GroupDummyAndContents/Write into PAssert$33/GroupGlobally/GroupDummyAndContents/Reify Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.851Z: Fusing consumer PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign into PAssert$33/GroupGlobally/Create.Values/Read(CreateSource) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.878Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.900Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write into PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.937Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into PAssert$33/GroupGlobally/Window.Into()/Window.Assign Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:23.970Z: Fusing consumer OutputSideInputs into Create123/Read(CreateSource) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.009Z: Fusing consumer PAssert$33/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.046Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map into PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.086Z: Fusing consumer PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map into PAssert$33/GroupGlobally/RewindowActuals/Window.Assign Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.123Z: Fusing consumer PAssert$33/GroupGlobally/RewindowActuals/Window.Assign into PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.158Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.198Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.238Z: Fusing consumer PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.270Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.311Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.350Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.385Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Create47/Read(CreateSource) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.425Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.463Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.492Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.532Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.566Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.607Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:24.996Z: Executing operation PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Create Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.034Z: Executing operation PAssert$33/GroupGlobally/GroupDummyAndContents/Create Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.075Z: Starting 1 workers in us-central1-b... Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.075Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.115Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.336Z: Executing operation PAssert$33/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write Nov 13, 2018 2:34:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:25.375Z: Executing operation Create47/Read(CreateSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write Nov 13, 2018 2:34:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:34:37.363Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Nov 13, 2018 2:35:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:11.682Z: Workers have started successfully. Nov 13, 2018 2:35:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:21.507Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Nov 13, 2018 2:35:21 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:21.545Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 13, 2018 2:35:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:28.299Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close Nov 13, 2018 2:35:29 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:28.401Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write Nov 13, 2018 2:35:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:34.162Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close Nov 13, 2018 2:35:36 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:34.243Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) Nov 13, 2018 2:35:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:42.926Z: Workers have started successfully. Nov 13, 2018 2:35:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:47.337Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView Nov 13, 2018 2:35:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:47.511Z: Executing operation Create123/Read(CreateSource)+OutputSideInputs+PAssert$33/GroupGlobally/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write Nov 13, 2018 2:35:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:53.550Z: Executing operation PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Close Nov 13, 2018 2:35:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:35:53.654Z: Executing operation PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$33/GroupGlobally/RewindowActuals/Window.Assign+PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write Nov 13, 2018 2:36:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:36:00.056Z: Executing operation PAssert$33/GroupGlobally/GroupDummyAndContents/Close Nov 13, 2018 2:36:02 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:36:00.135Z: Executing operation PAssert$33/GroupGlobally/GroupDummyAndContents/Read+PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$33/GroupGlobally/Values/Values/Map+PAssert$33/GroupGlobally/ParDo(Concat)+PAssert$33/GetPane/Map+PAssert$33/RunChecks+PAssert$33/VerifyAssertions/ParDo(DefaultConclude) Nov 13, 2018 2:36:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:36:05.785Z: Cleaning up. Nov 13, 2018 2:36:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:36:05.884Z: Stopping worker pool... Nov 13, 2018 2:38:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:38:14.989Z: Autoscaling: Resized worker pool from 1 to 0. Nov 13, 2018 2:38:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:38:15.026Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 13, 2018 2:38:16 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-13T02:38:15.081Z: Worker pool stopped. Nov 13, 2018 2:38:22 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-11-12_18_34_12-17741057379330836219 finished with status DONE. Nov 13, 2018 2:38:22 AM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-11-12_18_34_12-17741057379330836219. Found 1 success, 0 failures out of 1 expected assertions. Gradle Test Executor 94 finished executing tests. > Task :beam-runners-google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest > FAILED 217 tests completed, 10 failed, 2 skipped Finished generating test XML results (0.45 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerLegacyWorkerTest> Generating HTML test report... Finished generating test html results (0.283 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerLegacyWorkerTest> :beam-runners-google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest (Thread[Task worker for ':' Thread 111,5,main]) completed. Took 2 hrs 30 mins 55.032 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest'. > There were failing tests. See the report at: > file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerLegacyWorkerTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2h 34m 20s 59 actionable tasks: 55 executed, 4 from cache Publishing build scan... https://gradle.com/s/b2awnyttrgwbc Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org