See <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/100/display/redirect?page=changes>
Changes: [lukasz.gajowy] [BEAM-6011] Create builder for phrase triggered only jobs [lukasz.gajowy] [BEAM-6011] Create builder for NEXMark Jobs and Suites [lukasz.gajowy] [BEAM-6011] Add NEXMark jobs that can be phrase-triggered ------------------------------------------ [...truncated 18.26 MB...] INFO: 2018-11-15T12:39:29.451Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.502Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.552Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.592Z: Fusing consumer PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.631Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.680Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.730Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.782Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.828Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.878Z: Fusing consumer OutputSideInputs into Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.924Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:29.960Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.007Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource) into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.057Z: Fusing consumer PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map into PAssert$27/GroupGlobally/RewindowActuals/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.103Z: Fusing consumer PAssert$27/GroupGlobally/RewindowActuals/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.154Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.198Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.241Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.308Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.355Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.398Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.441Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into PAssert$27/GroupGlobally/Window.Into()/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.481Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.539Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.587Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.635Z: Fusing consumer Create47/Read(CreateSource)/ParDo(SplitBoundedSource) into Create47/Read(CreateSource)/Impulse Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.684Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.729Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.773Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.815Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:30.856Z: Fusing consumer Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create123/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.058Z: Fusing consumer PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.104Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.151Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write into View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.196Z: Fusing consumer PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.244Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.292Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.337Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.380Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.430Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.474Z: Fusing consumer Create123/Read(CreateSource)/ParDo(SplitBoundedSource) into Create123/Read(CreateSource)/Impulse Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.523Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map into Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.561Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey into View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.598Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.647Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key into Create47/Read(CreateSource)/ParDo(SplitBoundedSource) Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.690Z: Fusing consumer View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial into View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.737Z: Fusing consumer Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:31.789Z: Fusing consumer Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource) into Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.322Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.372Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.422Z: Starting 1 workers in us-central1-b... Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.422Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.475Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.526Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.582Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.631Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:32.993Z: Executing operation Create123/Read(CreateSource)/Impulse+Create123/Read(CreateSource)/ParDo(SplitBoundedSource)+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:33.040Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Impulse+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(SplitBoundedSource)+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 15, 2018 12:39:34 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:33.090Z: Executing operation Create47/Read(CreateSource)/Impulse+Create47/Read(CreateSource)/ParDo(SplitBoundedSource)+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Pair with random key+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Nov 15, 2018 12:39:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:39:44.505Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:40:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:13.056Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:40:13 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:13.094Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 15, 2018 12:40:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:16.380Z: Workers have started successfully. Nov 15, 2018 12:40:16 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:16.428Z: Workers have started successfully. Nov 15, 2018 12:40:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:42.417Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 15, 2018 12:40:44 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:42.530Z: Executing operation PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+PAssert$27/GroupGlobally/Create.Values/Read(CreateSource)/ParDo(ReadFromBoundedSource)+PAssert$27/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 15, 2018 12:40:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:47.313Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 15, 2018 12:40:49 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:47.412Z: Executing operation Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create47/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create47/Read(CreateSource)/ParDo(ReadFromBoundedSource)+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Reify+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Write Nov 15, 2018 12:40:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:55.507Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Close Nov 15, 2018 12:40:57 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:55.604Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Read+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/GroupByWindow+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Extract+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map+View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization)+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateIsmShardKeyAndSortKey+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Write Nov 15, 2018 12:41:00 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:40:59.937Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Nov 15, 2018 12:41:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:03.316Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Close Nov 15, 2018 12:41:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:03.414Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/GroupByKeyHashAndSortByKeyAndWindow/Read+View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ToIsmRecordForMultimap Nov 15, 2018 12:41:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:12.625Z: Executing operation View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Materialize Nov 15, 2018 12:41:14 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:12.855Z: Executing operation Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Create123/Read(CreateSource)/Reshuffle.ViaRandomKey/Values/Values/Map+Create123/Read(CreateSource)/ParDo(ReadFromBoundedSource)+OutputSideInputs+PAssert$27/GroupGlobally/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$27/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$27/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Write Nov 15, 2018 12:41:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:16.330Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Close Nov 15, 2018 12:41:17 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:16.429Z: Executing operation PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$27/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$27/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$27/GroupGlobally/RewindowActuals/Window.Assign+PAssert$27/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$27/GroupGlobally/GroupDummyAndContents/Reify+PAssert$27/GroupGlobally/GroupDummyAndContents/Write Nov 15, 2018 12:41:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:19.104Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Close Nov 15, 2018 12:41:21 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:19.202Z: Executing operation PAssert$27/GroupGlobally/GroupDummyAndContents/Read+PAssert$27/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$27/GroupGlobally/Values/Values/Map+PAssert$27/GroupGlobally/ParDo(Concat)+PAssert$27/GetPane/Map+PAssert$27/RunChecks+PAssert$27/VerifyAssertions/ParDo(DefaultConclude) Nov 15, 2018 12:41:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:25.956Z: Cleaning up. Nov 15, 2018 12:41:27 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:41:26.085Z: Stopping worker pool... Nov 15, 2018 12:43:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:43:29.305Z: Autoscaling: Resized worker pool from 1 to 0. Nov 15, 2018 12:43:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:43:29.354Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Nov 15, 2018 12:43:30 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:43:29.413Z: Worker pool stopped. Nov 15, 2018 12:43:37 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-11-15_04_39_20-14840848994362655486 finished with status DONE. Nov 15, 2018 12:43:37 PM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess INFO: Success result for Dataflow job 2018-11-15_04_39_20-14840848994362655486. Found 0 success, 0 failures out of 0 expected assertions. Gradle Test Executor 97 finished executing tests. > Task :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest Finished generating test XML results (0.52 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/test-results/validatesRunnerFnApiWorkerTest> Generating HTML test report... Finished generating test html results (0.66 secs) into: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesRunnerFnApiWorkerTest> Packing task ':beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest' Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin :beam-runners-google-cloud-dataflow-java:validatesRunnerFnApiWorkerTest (Thread[Task worker for ':' Thread 26,5,main]) completed. Took 2 hrs 10 mins 25.286 secs. :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task worker for ':' Thread 26,5,main]) started. > Task :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages Caching disabled for task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages': Caching has not been enabled for the task Task ':beam-runners-google-cloud-dataflow-java:cleanUpDockerImages' is not up-to-date because: Task has not declared any outputs despite executing actions. Starting process 'command 'docker''. Working directory: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java> Command: docker rmi us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 Successfully started process 'command 'docker'' Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3f28f06986c73b623644a4705b6c86570017a6b5456eee7f9b1c08a39953e74f Deleted: sha256:dfb3e010ce8bae292a7f7f861cde3582fffe4d4dd33b933db9070c661e14fccf Deleted: sha256:9c68332bb7a7855d9cdb498a5810c33e0165758c07489ac93e5827fff614860d Deleted: sha256:a12874a95009ad3ab9fcfcd30078c959e07963f2a8d4acd56cb43165ee5d67dc Deleted: sha256:8dca30c28f6737db134124ceee456271e559f24a4f7f2a3b4a472b88b73d0b01 Deleted: sha256:e44ba4d213efda083926478d949152286d1511d1746e118664c6f6b55be1da38 Deleted: sha256:f1028cb74a3602b46636a9085a773df2158a4eeb5c7d18919e104f1c066b26b3 Deleted: sha256:d23d52aa87a30d1a3070494df48acfb0d42760b578cf1b224f69a646b115d969 Deleted: sha256:0d5ba6b4c11a5cbb55527118c780cc68482e1cdd3e2f706e31c01f7da0a6d394 Deleted: sha256:bb6bf7c9671f84f247f1db7057c1a0762cddc9b43abbd334901e9dcf2f290970 Deleted: sha256:3ef29e7732f7aecfeed1d11b2c36b75d9790c39f19a3c468e2c9129fe20974a3 Starting process 'command 'gcloud''. Working directory: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java> Command: gcloud --quiet container images delete --force-delete-tags us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 Successfully started process 'command 'gcloud'' Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3f28f06986c73b623644a4705b6c86570017a6b5456eee7f9b1c08a39953e74f Associated tags: - 20181115102920 Tags: - us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920]. Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:3f28f06986c73b623644a4705b6c86570017a6b5456eee7f9b1c08a39953e74f]. :beam-runners-google-cloud-dataflow-java:cleanUpDockerImages (Thread[Task worker for ':' Thread 26,5,main]) completed. Took 2.295 secs. :beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi (Thread[Task worker for ':' Thread 26,5,main]) started. > Task :beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi > FAILED Caching disabled for task ':beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi': Caching has not been enabled for the task Task ':beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi' is not up-to-date because: Task has not declared any outputs despite executing actions. Starting process 'command 'docker''. Working directory: <https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java> Command: docker rmi us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 Successfully started process 'command 'docker'' Error response from daemon: No such image: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20181115102920 :beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi (Thread[Task worker for ':' Thread 26,5,main]) completed. Took 0.039 secs. FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_PortabilityApi_Dataflow_Gradle/ws/src/runners/google-cloud-dataflow-java/build.gradle'> line: 250 * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:validatesRunnerPortabilityApi'. > Process 'command 'docker'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 2h 14m 34s 77 actionable tasks: 71 executed, 5 from cache, 1 up-to-date Publishing build scan... https://gradle.com/s/pp6327borct7s Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org