See <https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/181/display/redirect?page=changes>
Changes: [ccy] Real-time timers shouldn't stall completed step in DirectRunner [yifanzou] return stdout text from command execution and stop using var.last_text ------------------------------------------ [...truncated 975.02 KB...] INFO: 19 sending EndOfStream Apr 10, 2018 3:55:04 AM com.datatorrent.stram.engine.Node emitEndStream INFO: 20 sending EndOfStream Apr 10, 2018 3:55:04 AM com.datatorrent.stram.engine.Node emitEndStream INFO: 21 sending EndOfStream Apr 10, 2018 3:55:05 AM com.datatorrent.stram.Journal write WARNING: Journal output stream is null. Skipping write to the WAL. Apr 10, 2018 3:55:05 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Undeploy request: [16] Apr 10, 2018 3:55:05 AM com.datatorrent.stram.engine.StreamingContainer undeploy INFO: Undeploy complete. Apr 10, 2018 3:55:05 AM com.datatorrent.bufferserver.server.Server$3 run INFO: Removing ln LogicalNode@68843994identifier=tcp://localhost:36284/16.output.16, upstream=16.output.16, group=stream17/17.input, partitions=[], iterator=com.datatorrent.bufferserver.internal.DataList$DataListIterator@26118550{da=com.datatorrent.bufferserver.internal.DataList$Block@84e69b2{identifier=16.output.16, data=1048576, readingOffset=0, writingOffset=47298, starting_window=5acc359300000001, ending_window=5acc359300000009, refCount=2, uniqueIdentifier=0, next=null, future=null}}} from dl com.datatorrent.bufferserver.internal.DataList@303b5e58 {16.output.16} Apr 10, 2018 3:55:05 AM com.datatorrent.stram.Journal write WARNING: Journal output stream is null. Skipping write to the WAL. Apr 10, 2018 3:55:05 AM com.datatorrent.stram.Journal write WARNING: Journal output stream is null. Skipping write to the WAL. Apr 10, 2018 3:55:05 AM com.datatorrent.stram.Journal write WARNING: Journal output stream is null. Skipping write to the WAL. Apr 10, 2018 3:55:05 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Undeploy request: [19, 20, 21] Apr 10, 2018 3:55:05 AM com.datatorrent.stram.engine.StreamingContainer undeploy INFO: Undeploy complete. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-5 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-5 msg: [container-5] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-5 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-9 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-7 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-1 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-1 msg: [container-1] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-1 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-9 msg: [container-9] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-9 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-7 msg: [container-7] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-7 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-2 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-2 msg: [container-2] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-10 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-2 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-10 msg: [container-10] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-10 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-8 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-0 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-8 msg: [container-8] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-8 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-0 msg: [container-0] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-0 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-6 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-6 msg: [container-6] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-6 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-4 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-4 msg: [container-4] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-4 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StreamingContainerManager processHeartbeat INFO: requesting idle shutdown for container container-3 Apr 10, 2018 3:55:06 AM com.datatorrent.stram.engine.StreamingContainer processHeartbeatResponse INFO: Received shutdown request type ABORT Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$UmbilicalProtocolLocalImpl log INFO: container-3 msg: [container-3] Exiting heartbeat loop.. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster$LocalStreamingContainerLauncher run INFO: Container container-3 terminating. Apr 10, 2018 3:55:06 AM com.datatorrent.stram.StramLocalCluster run INFO: Application finished. Apr 10, 2018 3:55:06 AM com.datatorrent.bufferserver.server.Server$1 run INFO: Server stopped listening at /0.0.0.0:36284 grep Foundation counts* counts-00111-of-00330:Foundation: 1 Verified Foundation: 1 [SUCCESS] Apr 10, 2018 3:55:12 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:55:09.811Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s). Apr 10, 2018 3:55:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:55:25.207Z: Workers have started successfully. Apr 10, 2018 3:55:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2018-04-10T03:55:39.022Z: java.io.FileNotFoundException: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv at org.apache.beam.sdk.util.GcsUtil$3.onFailure(GcsUtil.java:702) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:54) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:50) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseAndCallback(BatchUnparsedResponse.java:223) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseNextResponse(BatchUnparsedResponse.java:155) at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:253) at org.apache.beam.sdk.util.GcsUtil.lambda$executeBatches$0(GcsUtil.java:593) at org.apache.beam.sdk.util.MoreFutures.lambda$runAsync$2(MoreFutures.java:134) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Apr 10, 2018 3:55:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2018-04-10T03:55:40.798Z: java.io.FileNotFoundException: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv at org.apache.beam.sdk.util.GcsUtil$3.onFailure(GcsUtil.java:702) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:54) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:50) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseAndCallback(BatchUnparsedResponse.java:223) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseNextResponse(BatchUnparsedResponse.java:155) at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:253) at org.apache.beam.sdk.util.GcsUtil.lambda$executeBatches$0(GcsUtil.java:593) at org.apache.beam.sdk.util.MoreFutures.lambda$runAsync$2(MoreFutures.java:134) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Apr 10, 2018 3:55:47 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2018-04-10T03:55:46.702Z: java.io.FileNotFoundException: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv at org.apache.beam.sdk.util.GcsUtil$3.onFailure(GcsUtil.java:702) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:54) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:50) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseAndCallback(BatchUnparsedResponse.java:223) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseNextResponse(BatchUnparsedResponse.java:155) at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:253) at org.apache.beam.sdk.util.GcsUtil.lambda$executeBatches$0(GcsUtil.java:593) at org.apache.beam.sdk.util.MoreFutures.lambda$runAsync$2(MoreFutures.java:134) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Apr 10, 2018 3:55:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2018-04-10T03:55:49.807Z: java.io.FileNotFoundException: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/5000_gaming_data.csv at org.apache.beam.sdk.util.GcsUtil$3.onFailure(GcsUtil.java:702) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:54) at com.google.api.client.googleapis.batch.json.JsonBatchCallback.onFailure(JsonBatchCallback.java:50) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseAndCallback(BatchUnparsedResponse.java:223) at com.google.api.client.googleapis.batch.BatchUnparsedResponse.parseNextResponse(BatchUnparsedResponse.java:155) at com.google.api.client.googleapis.batch.BatchRequest.execute(BatchRequest.java:253) at org.apache.beam.sdk.util.GcsUtil.lambda$executeBatches$0(GcsUtil.java:593) at org.apache.beam.sdk.util.MoreFutures.lambda$runAsync$2(MoreFutures.java:134) at java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1626) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Apr 10, 2018 3:55:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process SEVERE: 2018-04-10T03:55:49.903Z: Workflow failed. Causes: S02:TextIO.Read/Read+ParseGameEvent+ExtractUserScore/MapElements/Map+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey+ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues/Partial+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Reify+ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/Write failed. Apr 10, 2018 3:55:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:55:50.017Z: Cleaning up. Apr 10, 2018 3:55:51 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:55:50.139Z: Stopping worker pool... Apr 10, 2018 3:57:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:57:13.545Z: Autoscaling: Resized worker pool from 2 to 0. Apr 10, 2018 3:57:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:57:13.576Z: Autoscaling: Would further reduce the number of workers but reached the minimum number allowed for the job. Apr 10, 2018 3:57:14 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-04-10T03:57:13.617Z: Worker pool stopped. Apr 10, 2018 3:57:22 AM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish INFO: Job 2018-04-09_20_54_29-4670947465499129879 failed with status FAILED. gsutil cat gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* | grep user19_BananaWallaby CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* CommandException: No URLs matched: gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/java-userscore-result-dataflow-runner.txt* [ERROR] Failed command :beam-runners-google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':beam-runners-direct-java:runMobileGamingJavaDirect'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * What went wrong: Execution failed for task ':beam-runners-google-cloud-dataflow-java:runMobileGamingJavaDataflow'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org BUILD FAILED in 4m 59s 5 actionable tasks: 5 executed Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Not sending mail to unregistered user sweg...@google.com Not sending mail to unregistered user eh...@google.com Not sending mail to unregistered user daniel.o.program...@gmail.com Not sending mail to unregistered user ankurgoe...@gmail.com Not sending mail to unregistered user sid...@google.com Not sending mail to unregistered user yifan...@yifanzou-linuxworkstation.sea.corp.google.com Not sending mail to unregistered user re...@relax-macbookpro2.roam.corp.google.com Not sending mail to unregistered user c...@google.com Not sending mail to unregistered user hero...@google.com Not sending mail to unregistered user git...@alasdairhodge.co.uk Not sending mail to unregistered user alan.myrv...@comcast.net Not sending mail to unregistered user aal...@gmail.com Not sending mail to unregistered user swegn...@gmail.com Not sending mail to unregistered user w...@google.com Not sending mail to unregistered user boyu...@google.com Not sending mail to unregistered user amyrv...@google.com Not sending mail to unregistered user xuming...@users.noreply.github.com Not sending mail to unregistered user mark...@google.com