See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/220/display/redirect?page=changes>
Changes:
[mairbek] Introduced SpannerWriteResult that
[mairbek] Addressed comments
[mairbek] Happy checkstyle
[apilloud] Enable githubCommitNotifier for post commits
------------------------------------------
[...truncated 10.43 MB...]
INFO: Adding
PAssert$33/GroupGlobally/RemoveActualsTriggering/Flatten.PCollections as step
s20
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/Create.Values/Read(CreateSource) as
step s21
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign as step
s22
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$33/GroupGlobally/RemoveDummyTriggering/Flatten.PCollections as step s23
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/FlattenDummyAndContents as step s24
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/NeverTrigger/Flatten.PCollections as
step s25
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/GroupDummyAndContents as step s26
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/Values/Values/Map as step s27
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/ParDo(Concat) as step s28
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GetPane/Map as step s29
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/RunChecks as step s30
May 09, 2018 12:59:03 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/VerifyAssertions/ParDo(DefaultConclude) as step s31
May 09, 2018 12:59:03 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-0509005858-24ad89f/output/results/staging/
May 09, 2018 12:59:04 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <71843 bytes, hash 7a5i5-wbZklIqIHX39HCNw> to
gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-0509005858-24ad89f/output/results/staging/pipeline-7a5i5-wbZklIqIHX39HCNw.pb
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT
Dataflow SDK version: 2.5.0-SNAPSHOT
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR
May 09, 2018 12:59:05 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-08_17_59_04-614006630417702889?project=apache-beam-testing
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT
Submitted job: 2018-05-08_17_59_04-614006630417702889
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR
May 09, 2018 12:59:05 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2018-05-08_17_59_04-614006630417702889
May 09, 2018 12:59:05 AM
org.apache.beam.runners.dataflow.TestDataflowRunner run
INFO: Running Dataflow job 2018-05-08_17_59_04-614006630417702889 with 1
expected assertions.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:04.512Z: Autoscaling is enabled for job
2018-05-08_17_59_04-614006630417702889. The number of workers will be between 1
and 1000.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:04.545Z: Autoscaling was automatically enabled for
job 2018-05-08_17_59_04-614006630417702889.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:07.253Z: Checking required Cloud APIs are enabled.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:07.451Z: Checking permissions granted to controller
Service Account.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:13.198Z: Worker configuration: n1-standard-1 in
us-central1-b.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:13.675Z: Expanding CoGroupByKey operations into
optimizable parts.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:13.882Z: Expanding GroupByKey operations into
optimizable parts.
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:13.929Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.211Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.265Z: Elided trivial flatten
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.326Z: Elided trivial flatten
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.388Z: Elided trivial flatten
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.440Z: Unzipping flatten s24 for input
s19.org.apache.beam.sdk.values.PCollection.<init>:329#8fd0f908e3ea90e3
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.492Z: Fusing unzipped copy of
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$33/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.545Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$33/GroupGlobally/GroupDummyAndContents/Read
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.600Z: Fusing consumer
PAssert$33/GroupGlobally/ParDo(Concat) into
PAssert$33/GroupGlobally/Values/Values/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.644Z: Fusing consumer PAssert$33/GetPane/Map into
PAssert$33/GroupGlobally/ParDo(Concat)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.687Z: Fusing consumer
PAssert$33/GroupGlobally/Values/Values/Map into
PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.738Z: Fusing consumer PAssert$33/RunChecks into
PAssert$33/GetPane/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.796Z: Fusing consumer
PAssert$33/VerifyAssertions/ParDo(DefaultConclude) into PAssert$33/RunChecks
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.860Z: Unzipping flatten s24-u45 for input
s26-reify-value18-c43
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.923Z: Fusing unzipped copy of
PAssert$33/GroupGlobally/GroupDummyAndContents/Write, through flatten s24-u45,
into producer PAssert$33/GroupGlobally/GroupDummyAndContents/Reify
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:14.986Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.043Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/Write into
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.099Z: Fusing consumer
PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$33/GroupGlobally/Create.Values/Read(CreateSource)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.142Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.198Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.256Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$33/GroupGlobally/Window.Into()/Window.Assign
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.307Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.372Z: Fusing consumer
PAssert$33/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.424Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.470Z: Fusing consumer
PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$33/GroupGlobally/RewindowActuals/Window.Assign
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.531Z: Fusing consumer
PAssert$33/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.591Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.639Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.685Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.734Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.790Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.842Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.902Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.954Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:15.999Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.055Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.114Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.178Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.239Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.751Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Create
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.793Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Create
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.830Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.864Z: Starting 1 workers in us-central1-b...
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:16.865Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:17.143Z: Executing operation
PAssert$33/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write
May 09, 2018 12:59:21 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:17.193Z: Executing operation
Create47/Read(CreateSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 09, 2018 12:59:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:28.301Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
May 09, 2018 12:59:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:39.017Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
May 09, 2018 12:59:40 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:39.063Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
May 09, 2018 12:59:58 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T00:59:56.190Z: Workers have started successfully.
May 09, 2018 1:00:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:20.841Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 09, 2018 1:00:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:20.957Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 09, 2018 1:00:28 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:27.679Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 09, 2018 1:00:28 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:27.784Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 09, 2018 1:00:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:39.689Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 09, 2018 1:00:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:39.920Z: Executing operation
Create123/Read(CreateSource)+OutputSideInputs+PAssert$33/GroupGlobally/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write
May 09, 2018 1:00:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:46.865Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Close
May 09, 2018 1:00:47 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:46.983Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$33/GroupGlobally/RewindowActuals/Window.Assign+PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write
May 09, 2018 1:00:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:47.487Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Close
May 09, 2018 1:00:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:47.610Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Read+PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$33/GroupGlobally/Values/Values/Map+PAssert$33/GroupGlobally/ParDo(Concat)+PAssert$33/GetPane/Map+PAssert$33/RunChecks+PAssert$33/VerifyAssertions/ParDo(DefaultConclude)
May 09, 2018 1:00:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:49.455Z: Cleaning up.
May 09, 2018 1:00:50 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:00:49.616Z: Stopping worker pool...
May 09, 2018 1:02:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:02:24.608Z: Autoscaling: Resized worker pool from 1 to
0.
May 09, 2018 1:02:27 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-09T01:02:24.655Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
May 09, 2018 1:02:33 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-05-08_17_59_04-614006630417702889 finished with status DONE.
May 09, 2018 1:02:33 AM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-05-08_17_59_04-614006630417702889. Found 1 success, 0 failures out of 1
expected assertions.
Gradle Test Executor 14 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerTest
Finished generating test XML results (0.304 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerTest>
Generating HTML test report...
Finished generating test html results (0.308 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerTest>
Packing task ':beam-runners-google-cloud-dataflow-java:validatesRunnerTest'
:beam-runners-google-cloud-dataflow-java:validatesRunnerTest (Thread[Task
worker for ':' Thread 9,5,main]) completed. Took 2 hrs 12 mins 58.929 secs.
:beam-runners-google-cloud-dataflow-java:validatesRunner (Thread[Task worker
for ':' Thread 9,5,main]) started.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunner
Skipping task ':beam-runners-google-cloud-dataflow-java:validatesRunner' as it
has no actions.
:beam-runners-google-cloud-dataflow-java:validatesRunner (Thread[Task worker
for ':' Thread 9,5,main]) completed. Took 0.0 secs.
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
See
https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 2h 14m 2s
38 actionable tasks: 34 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/kmme4tf7js3dq
Build step 'Invoke Gradle script' changed build result to SUCCESS
Recording test results
[Set GitHub commit status (universal)] SUCCESS on repos
[GHRepository@54af4bb0[description=Apache
Beam,homepage=,name=beam,fork=false,size=58753,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1
200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag,
Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining,
X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval],
Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip],
Content-Security-Policy=[default-src 'none'], Content-Type=[application/json;
charset=utf-8], Date=[Wed, 09 May 2018 01:02:38 GMT],
ETag=[W/"a78804f2fd4af0457a68fcd00bcae057"], Last-Modified=[Wed, 09 May 2018
00:26:13 GMT], OkHttp-Received-Millis=[1525827758889],
OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1],
OkHttp-Sent-Millis=[1525827758737], Referrer-Policy=[origin-when-cross-origin,
strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK],
Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload],
Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie,
X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo],
X-Content-Type-Options=[nosniff], X-Frame-Options=[deny],
X-GitHub-Media-Type=[github.v3; format=json],
X-GitHub-Request-Id=[AB48:4F41:A9D030:1735E5C:5AF2488C],
X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000],
X-RateLimit-Remaining=[4949], X-RateLimit-Reset=[1525828827],
X-Runtime-rack=[0.068826], X-XSS-Protection=[1;
mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]]
(sha:e7a3578) with context:beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle
Setting commit status on GitHub for
https://github.com/apache/beam/commit/e7a3578abb919021be95c9c7148fd2d4fc6331fd
ERROR: Build step failed with exception
java.io.FileNotFoundException:
https://api.github.com/repos/apache/beam/statuses/e7a3578abb919021be95c9c7148fd2d4fc6331fd
at
com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
at
com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
at
com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
at org.kohsuke.github.Requester.parse(Requester.java:612)
at org.kohsuke.github.Requester.parse(Requester.java:594)
at org.kohsuke.github.Requester._to(Requester.java:272)
Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not
Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"}
at org.kohsuke.github.Requester.handleApiError(Requester.java:686)
at org.kohsuke.github.Requester._to(Requester.java:293)
at org.kohsuke.github.Requester.to(Requester.java:234)
at
org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075)
at
org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160)
Caused:
org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException
at
org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74)
at
org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164)
at
com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151)
at
hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at
hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
at hudson.model.Build$BuildExecution.post2(Build.java:186)
at
hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
at hudson.model.Run.execute(Run.java:1749)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Build step 'Set build status on GitHub commit [deprecated]' marked build as
failure
Not sending mail to unregistered user apill...@google.com
Not sending mail to unregistered user mair...@google.com
