See <https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/12/display/redirect?page=changes>
Changes: [herohde] [BEAM-4034] Fix hooks panic with Go on Flink ------------------------------------------ [...truncated 16.16 MB...] Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/Flatten.PCollections as step s25 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/CreateDataflowView as step s26 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Partition input as step s27 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition as step s28 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Batch mutations together as step s29 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding SpannerIO.Write/Write mutations to Cloud Spanner/Write mutations to Spanner as step s30 Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-end-to-end-tests/spannerwriteit0testwrite-jenkins-0410012453-3a21c811/output/results/staging/ Apr 10, 2018 1:24:57 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <80747 bytes, hash h_UtfCnQ4kHmHU9ZFLWNMQ> to gs://temp-storage-for-end-to-end-tests/spannerwriteit0testwrite-jenkins-0410012453-3a21c811/output/results/staging/pipeline-h_UtfCnQ4kHmHU9ZFLWNMQ.pb org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT > testWrite STANDARD_OUT Dataflow SDK version: 2.5.0-SNAPSHOT org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR Apr 10, 2018 1:24:59 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-04-09_18_22_00-16544945643193247653 finished with status DONE. Apr 10, 2018 1:24:59 AM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-04-09_18_22_00-16544945643193247653. Found 0 success, 0 failures out of 0 expected assertions. org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT > testWrite STANDARD_ERROR Apr 10, 2018 1:24:59 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-04-09_18_24_58-17834478248174554934?project=apache-beam-testing org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT > testWrite STANDARD_OUT Submitted job: 2018-04-09_18_24_58-17834478248174554934 org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT > testWrite STANDARD_ERROR Apr 10, 2018 1:24:59 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2018-04-09_18_24_58-17834478248174554934 Apr 10, 2018 1:24:59 AM org.apache.beam.runners.dataflow.TestDataflowRunner run INFO: Running Dataflow job 2018-04-09_18_24_58-17834478248174554934 with 0 expected assertions. org.apache.beam.sdk.io.gcp.datastore.V1WriteIT > testE2EV1Write STANDARD_ERROR Apr 10, 2018 1:25:00 AM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch INFO: Writing batch of 500 entities Apr 10, 2018 1:25:00 AM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch INFO: Successfully wrote 500 entities Apr 10, 2018 1:25:01 AM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch INFO: Writing batch of 500 entities Apr 10, 2018 1:25:01 AM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil$V1TestWriter flushBatch INFO: Successfully wrote 500 entities Apr 10, 2018 1:25:01 AM org.apache.beam.sdk.io.gcp.datastore.V1TestUtil deleteAllEntities INFO: Successfully deleted 1000 entities Gradle Test Executor 126 finished executing tests. org.apache.beam.sdk.io.gcp.spanner.SpannerWriteIT > testWrite STANDARD_ERROR Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:24:58.425Z: Autoscaling is enabled for job 2018-04-09_18_24_58-17834478248174554934. The number of workers will be between 1 and 1000. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:24:58.455Z: Autoscaling was automatically enabled for job 2018-04-09_18_24_58-17834478248174554934. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:01.066Z: Checking required Cloud APIs are enabled. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:01.168Z: Checking permissions granted to controller Service Account. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:04.643Z: Expanding CoGroupByKey operations into optimizable parts. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:04.839Z: Expanding GroupByKey operations into optimizable parts. Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:04.857Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.018Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.041Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Read information schema into SpannerIO.Write/Write mutations to Cloud Spanner/Create.Values/Read(CreateSource) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.071Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.095Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.126Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.148Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into SpannerIO.Write/Write mutations to Cloud Spanner/Read information schema Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.169Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.199Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.224Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.254Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.276Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.305Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.338Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Extract keys into SpannerIO.Write/Write mutations to Cloud Spanner/Serialize mutations Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.349Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Serialize mutations into SpannerIO.Write/To mutation group Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.372Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParDo(ToIsmMetadataRecordForKey) into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.396Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Write mutations to Spanner into SpannerIO.Write/Write mutations to Cloud Spanner/Batch mutations together Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.422Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParMultiDo(ToIsmRecordForMapLike) into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.440Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParMultiDo(ToIsmRecordForMapLike) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.455Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) into SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Extract Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.486Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Reify into SpannerIO.Write/Write mutations to Cloud Spanner/Partition input Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.513Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues into SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.539Z: Fusing consumer SpannerIO.Write/To mutation group into ParDo(GenerateMutations) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.563Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParDo(ToIsmMetadataRecordForSize) into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.583Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/GroupByWindow into SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Read Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.608Z: Fusing consumer ParDo(GenerateMutations) into GenerateSequence/Read(BoundedCountingSource) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.636Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParMultiDo(ToIsmRecordForMapLike) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.661Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Batch mutations together into SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/GroupByWindow Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.683Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Extract into SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.710Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Reify into SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Partial Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.735Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Partial into SpannerIO.Write/Write mutations to Cloud Spanner/Extract keys Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.765Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow) Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.790Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Reify Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:05.821Z: Fusing consumer SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Write into SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Reify Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.169Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.188Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.201Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.225Z: Starting 1 workers in us-central1-f... Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.225Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.252Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.275Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Create Apr 10, 2018 1:25:07 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:06.306Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create Apr 10, 2018 1:25:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:15.651Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Apr 10, 2018 1:25:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:16.263Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Create.Values/Read(CreateSource)+SpannerIO.Write/Write mutations to Cloud Spanner/Read information schema+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write Apr 10, 2018 1:25:24 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:21.406Z: Autoscaling: Resizing worker pool from 1 to 7. Apr 10, 2018 1:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:33.426Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Apr 10, 2018 1:25:34 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:33.454Z: Resized worker pool to 1, though goal was 7. This could be a quota issue. Apr 10, 2018 1:25:44 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:44.494Z: Autoscaling: Raised the number of workers to 6 based on the rate of progress in the currently running step(s). Apr 10, 2018 1:25:45 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:44.511Z: Resized worker pool to 6, though goal was 7. This could be a quota issue. Apr 10, 2018 1:25:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:25:47.770Z: Workers have started successfully. Apr 10, 2018 1:26:01 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:00.349Z: Autoscaling: Raised the number of workers to 7 based on the rate of progress in the currently running step(s). Apr 10, 2018 1:26:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:18.840Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close Apr 10, 2018 1:26:18 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:18.945Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write Apr 10, 2018 1:26:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:23.216Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close Apr 10, 2018 1:26:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:23.298Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) Apr 10, 2018 1:26:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:32.797Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Schema View/Combine.GloballyAsSingletonView/CreateDataflowView Apr 10, 2018 1:26:33 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:33.070Z: Executing operation GenerateSequence/Read(BoundedCountingSource)+ParDo(GenerateMutations)+SpannerIO.Write/To mutation group+SpannerIO.Write/Write mutations to Cloud Spanner/Serialize mutations+SpannerIO.Write/Write mutations to Cloud Spanner/Extract keys+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Partial+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Reify+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Write Apr 10, 2018 1:26:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:38.873Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Close Apr 10, 2018 1:26:39 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:38.962Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/GroupByKey/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues+SpannerIO.Write/Write mutations to Cloud Spanner/Sample keys/Combine.GroupedValues/Extract+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/ParDo(GroupByKeyHashAndSortByKeyAndWindow)+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write Apr 10, 2018 1:26:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:46.786Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close Apr 10, 2018 1:26:49 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:46.871Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForData/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParMultiDo(ToIsmRecordForMapLike)+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Write+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Write Apr 10, 2018 1:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:51.199Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Close Apr 10, 2018 1:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:51.232Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Close Apr 10, 2018 1:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:51.285Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForSize/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParDo(ToIsmMetadataRecordForSize) Apr 10, 2018 1:26:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:51.316Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/GBKaSVForKeys/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/ParDo(ToIsmMetadataRecordForKey) Apr 10, 2018 1:26:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:54.681Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/Flatten.PCollections Apr 10, 2018 1:26:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:55.144Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Keys sample as view/CreateDataflowView Apr 10, 2018 1:26:55 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:55.402Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Partition input+SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Reify+SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Write Apr 10, 2018 1:26:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:58.053Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Close Apr 10, 2018 1:26:58 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:26:58.143Z: Executing operation SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/Read+SpannerIO.Write/Write mutations to Cloud Spanner/Group by partition/GroupByWindow+SpannerIO.Write/Write mutations to Cloud Spanner/Batch mutations together+SpannerIO.Write/Write mutations to Cloud Spanner/Write mutations to Spanner Apr 10, 2018 1:27:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:27:06.841Z: Cleaning up. Apr 10, 2018 1:27:09 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:27:07.258Z: Stopping worker pool... Apr 10, 2018 1:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:28:26.083Z: Autoscaling: Resized worker pool from 7 to 0. Apr 10, 2018 1:28:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T01:28:26.120Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Apr 10, 2018 1:28:34 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-04-09_18_24_58-17834478248174554934 finished with status DONE. Apr 10, 2018 1:28:34 AM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-04-09_18_24_58-17834478248174554934. Found 0 success, 0 failures out of 0 expected assertions. Gradle Test Executor 128 finished executing tests. 12 tests completed, 1 failed Finished generating test XML results (0.013 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/ws/src/runners/google-cloud-dataflow-java/build/test-results/googleCloudPlatformIntegrationTest> Generating HTML test report... Finished generating test html results (0.02 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/googleCloudPlatformIntegrationTest> :beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest FAILED :beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 16 mins 21.496 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:googleCloudPlatformIntegrationTest'. > There were failing tests. See the report at: > file://<https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/googleCloudPlatformIntegrationTest/index.html> * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 22m 12s 760 actionable tasks: 760 executed Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Recording test results Not sending mail to unregistered user eh...@google.com Not sending mail to unregistered user daniel.o.program...@gmail.com Not sending mail to unregistered user hero...@google.com