See
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/324/display/redirect?page=changes>
Changes:
[ankurgoenka] Adding ArtifactStagingFileHandler to push files to staging server
[ankurgoenka] Adding filesystem based artifact service to ULR job service
[ankurgoenka] Adding context manager to ArtifactStagingFileHandler
[ankurgoenka] Converting ArtifactServiceFileHandler to PortableStager based on
[ankurgoenka] Simplifying numeric constants
[ankurgoenka] Logging and naming changes
[ankurgoenka] Enhancing test cases
[ankurgoenka] Updating the dostring
------------------------------------------
[...truncated 18.03 MB...]
INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
as step s14
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign as step
s15
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey as step
s16
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map as
step s17
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/RewindowActuals/Window.Assign as step
s18
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map as step s19
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$33/GroupGlobally/RemoveActualsTriggering/Flatten.PCollections as step
s20
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/Create.Values/Read(CreateSource) as
step s21
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign as step
s22
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$33/GroupGlobally/RemoveDummyTriggering/Flatten.PCollections as step s23
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/FlattenDummyAndContents as step s24
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/NeverTrigger/Flatten.PCollections as
step s25
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/GroupDummyAndContents as step s26
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/Values/Values/Map as step s27
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GroupGlobally/ParDo(Concat) as step s28
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/GetPane/Map as step s29
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/RunChecks as step s30
May 23, 2018 3:40:52 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$33/VerifyAssertions/ParDo(DefaultConclude) as step s31
May 23, 2018 3:40:52 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-0523034046-cdd8ef59/output/results/staging/
May 23, 2018 3:40:52 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <71120 bytes, hash _HN0Fa2MkFbUAiDaG-EYsA> to
gs://temp-storage-for-validates-runner-tests//viewtest0testsingletonsideinput-jenkins-0523034046-cdd8ef59/output/results/staging/pipeline-_HN0Fa2MkFbUAiDaG-EYsA.pb
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT
Dataflow SDK version: 2.5.0-SNAPSHOT
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR
May 23, 2018 3:40:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-22_20_40_53-10774031641738260836?project=apache-beam-testing
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_OUT
Submitted job: 2018-05-22_20_40_53-10774031641738260836
org.apache.beam.sdk.transforms.ViewTest > testSingletonSideInput STANDARD_ERROR
May 23, 2018 3:40:54 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2018-05-22_20_40_53-10774031641738260836
May 23, 2018 3:40:54 AM org.apache.beam.runners.dataflow.TestDataflowRunner
run
INFO: Running Dataflow job 2018-05-22_20_40_53-10774031641738260836 with 1
expected assertions.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:40:53.327Z: Autoscaling is enabled for job
2018-05-22_20_40_53-10774031641738260836. The number of workers will be between
1 and 1000.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:40:53.371Z: Autoscaling was automatically enabled for
job 2018-05-22_20_40_53-10774031641738260836.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:40:56.109Z: Checking required Cloud APIs are enabled.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:40:56.308Z: Checking permissions granted to controller
Service Account.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:02.745Z: Worker configuration: n1-standard-1 in
us-central1-b.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.301Z: Expanding CoGroupByKey operations into
optimizable parts.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.460Z: Expanding GroupByKey operations into
optimizable parts.
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.509Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.732Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.775Z: Elided trivial flatten
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.809Z: Elided trivial flatten
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.850Z: Elided trivial flatten
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.893Z: Unzipping flatten s24 for input
s19.org.apache.beam.sdk.values.PCollection.<init>:329#8fd0f908e3ea90e3
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.934Z: Fusing unzipped copy of
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$33/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:03.980Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$33/GroupGlobally/GroupDummyAndContents/Read
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.027Z: Fusing consumer
PAssert$33/GroupGlobally/ParDo(Concat) into
PAssert$33/GroupGlobally/Values/Values/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.063Z: Fusing consumer PAssert$33/GetPane/Map into
PAssert$33/GroupGlobally/ParDo(Concat)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.104Z: Fusing consumer
PAssert$33/GroupGlobally/Values/Values/Map into
PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.143Z: Fusing consumer PAssert$33/RunChecks into
PAssert$33/GetPane/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.188Z: Fusing consumer
PAssert$33/VerifyAssertions/ParDo(DefaultConclude) into PAssert$33/RunChecks
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.224Z: Unzipping flatten s24-u45 for input
s26-reify-value18-c43
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.270Z: Fusing unzipped copy of
PAssert$33/GroupGlobally/GroupDummyAndContents/Write, through flatten s24-u45,
into producer PAssert$33/GroupGlobally/GroupDummyAndContents/Reify
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.315Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.352Z: Fusing consumer
PAssert$33/GroupGlobally/GroupDummyAndContents/Write into
PAssert$33/GroupGlobally/GroupDummyAndContents/Reify
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.400Z: Fusing consumer
PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$33/GroupGlobally/Create.Values/Read(CreateSource)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.436Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.469Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.498Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$33/GroupGlobally/Window.Into()/Window.Assign
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.542Z: Fusing consumer OutputSideInputs into
Create123/Read(CreateSource)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.578Z: Fusing consumer
PAssert$33/GroupGlobally/Window.Into()/Window.Assign into OutputSideInputs
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.610Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.647Z: Fusing consumer
PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$33/GroupGlobally/RewindowActuals/Window.Assign
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.703Z: Fusing consumer
PAssert$33/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.747Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.789Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.836Z: Fusing consumer
PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.872Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.919Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:04.967Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.019Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Create47/Read(CreateSource)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.064Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.109Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.149Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
into
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.193Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.241Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.279Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.661Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Create
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.693Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Create
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.721Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.748Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.759Z: Starting 1 workers in us-central1-b...
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:05.976Z: Executing operation
PAssert$33/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$33/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write
May 23, 2018 3:41:08 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:06.011Z: Executing operation
Create47/Read(CreateSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write
May 23, 2018 3:41:14 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:13.197Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
May 23, 2018 3:41:25 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:23.753Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
May 23, 2018 3:41:25 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:23.782Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
May 23, 2018 3:41:46 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:41:44.267Z: Workers have started successfully.
May 23, 2018 3:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:07.243Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close
May 23, 2018 3:42:07 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:07.322Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write
May 23, 2018 3:42:17 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:16.421Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close
May 23, 2018 3:42:17 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:16.487Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+View.AsSingleton/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow)
May 23, 2018 3:42:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:30.077Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/CreateDataflowView
May 23, 2018 3:42:30 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:30.252Z: Executing operation
Create123/Read(CreateSource)+OutputSideInputs+PAssert$33/GroupGlobally/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$33/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$33/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Write
May 23, 2018 3:42:38 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:36.033Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Close
May 23, 2018 3:42:38 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:36.121Z: Executing operation
PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$33/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$33/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$33/GroupGlobally/RewindowActuals/Window.Assign+PAssert$33/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$33/GroupGlobally/GroupDummyAndContents/Reify+PAssert$33/GroupGlobally/GroupDummyAndContents/Write
May 23, 2018 3:42:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:39.838Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Close
May 23, 2018 3:42:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:39.943Z: Executing operation
PAssert$33/GroupGlobally/GroupDummyAndContents/Read+PAssert$33/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$33/GroupGlobally/Values/Values/Map+PAssert$33/GroupGlobally/ParDo(Concat)+PAssert$33/GetPane/Map+PAssert$33/RunChecks+PAssert$33/VerifyAssertions/ParDo(DefaultConclude)
May 23, 2018 3:42:45 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:43.991Z: Cleaning up.
May 23, 2018 3:42:45 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:42:44.112Z: Stopping worker pool...
May 23, 2018 3:44:19 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:44:17.234Z: Autoscaling: Resized worker pool from 1 to
0.
May 23, 2018 3:44:19 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:44:17.261Z: Autoscaling: Would further reduce the
number of workers but reached the minimum number allowed for the job.
May 23, 2018 3:44:19 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-05-23T03:44:17.316Z: Worker pool stopped.
May 23, 2018 3:44:28 AM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-05-22_20_40_53-10774031641738260836 finished with status
DONE.
May 23, 2018 3:44:28 AM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Success result for Dataflow job
2018-05-22_20_40_53-10774031641738260836. Found 1 success, 0 failures out of 1
expected assertions.
Gradle Test Executor 36 finished executing tests.
> Task :beam-runners-google-cloud-dataflow-java:validatesRunnerTest FAILED
209 tests completed, 34 failed, 2 skipped
Finished generating test XML results (0.536 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerTest>
Generating HTML test report...
Finished generating test html results (0.511 secs) into:
<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerTest>
:beam-runners-google-cloud-dataflow-java:validatesRunnerTest (Thread[Task
worker for ':' Thread 4,5,main]) completed. Took 2 hrs 1 mins 11.139 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-runners-google-cloud-dataflow-java:validatesRunnerTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
See
https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2h 2m 13s
38 actionable tasks: 34 executed, 4 from cache
Publishing build scan...
https://gradle.com/s/tiluvmzqoqb6q
Build cache (/home/jenkins/.gradle/caches/build-cache-1) remove files older
than Wed May 16 01:42:19 UTC 2018.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing 6910 cache
entries (695 MB reclaimed).
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.496
secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Recording test results
Not sending mail to unregistered user ekirpic...@gmail.com
Not sending mail to unregistered user robbe.sneyd...@ml6.eu
Not sending mail to unregistered user git...@alasdairhodge.co.uk
Not sending mail to unregistered user szewi...@gmail.com
Not sending mail to unregistered user timrobertson...@gmail.com
Not sending mail to unregistered user
yifan...@yifanzou-linuxworkstation.sea.corp.google.com
Not sending mail to unregistered user ankurgoe...@gmail.com
Not sending mail to unregistered user sid...@google.com
