See <https://builds.apache.org/job/beam_PerformanceTests_Spark/1627/display/redirect?page=changes>
Changes: [geet.kumar75] BEAM-4038: Support Kafka Headers in KafkaIO [geet.kumar75] BEAM-4038: Update license information [geet.kumar75] Update code formatting [geet.kumar75] Remove custom implementations of KafkaHeader, KafkaHeaders, etc. [geet.kumar75] Changes based on review comments [daniel.o.programmer] [BEAM-3513] Removing PrimitiveCombineGroupedValues override w/ FnAPI. [echauchot] Introduce MetricsPusher in runner core to regularly push aggregated [echauchot] Instanciate MetricsPusher in runner-specific code because we need [echauchot] Improve MetricsPusher: do not aggregate metrics when not needed, improve [echauchot] Create JsonMetricsSerializer [echauchot] Stop MetricsPusher thread by observing pipeline state and improve the [echauchot] Make metrics sink configurable through PipelineOptions, pass [echauchot] Add MetricsPusher tests specific to Spark (because Spark streaming tests [echauchot] Add a MetricksPusher test to runner-core (batch and streaming are done [echauchot] Push metrics at the end of a batch pipeline in flink runner [echauchot] improve MetricsPusher lifecycle and thread safety [echauchot] Make MetricsPusher merge a list a MetricsContainerStepMaps because there [echauchot] Fix thread synchronisation and replace usages of instance variable by [echauchot] Clear dummyMetricsSink before test [echauchot] Push metrics at the end of a batch pipeline in spark runner [echauchot] Improve MetricsPusher teardown to enable multiple pipelines in a single [echauchot] Manually generate json and remove jackson [echauchot] Replace use of http client by use of java.net.HttpUrlConnection and deal [echauchot] Remove DEFAULT_PERIOD constant in favor of already existing [echauchot] Remove unneeded null check, format [echauchot] convert MetricsSink to an interface with a single writeMetrics method [echauchot] Remove MetricsSerializer base class and inline serialization in [echauchot] Dynamically create the sinks by reflection [echauchot] Split DummyMetricsSink into NoOpMetricsSink (default sink) and [echauchot] Reduce overhead when no metricsSink is provided, do not start polling [echauchot] Make MetricsPusher a regular object instead of a singleton to allow [echauchot] Explicitely start MetricsPusher from the runners [echauchot] Separate MetricsHttpSink POC to a new runners-extensions artifact and [echauchot] Fix cycle bug between teardown() and pushmetrics() [echauchot] Update MetricsPusher and TestMetricsSink to new serializable [echauchot] Use regular jackson object mapper to serialize metrics now that they are [echauchot] Give MetricsPusher a bit of time to push before assert in test [echauchot] Make MetricsPusher thread a daemon [echauchot] Fix build and clean: dependencies, rat, checkstyle, findbugs, remove [geet.kumar75] Support kafka versions 0.10.1.0 and above [echauchot] Move build to gradle [echauchot] MetricsSink no more needs to be generic [echauchot] SparkRunnerDebugger does not need to export metrics as it does not run [tgroh] Use Existing Matchers in WatermarkManagerTest [echauchot] Move MetricsHttpSink and related classes to a new sub-module [kirpichov] Consistently handle EmptyMatchTreatment [rangadi] Add 10 millis sleep when there are no elements left in a partition. ------------------------------------------ [...truncated 69.74 KB...] 2018-04-24 00:17:50,212 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:18:15,592 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:18:19,180 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7575b9ff5d12d2be_00000162f503e5e3_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r7575b9ff5d12d2be_00000162f503e5e3_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7575b9ff5d12d2be_00000162f503e5e3_1 ... (0s) Current status: DONE 2018-04-24 00:18:19,180 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:18:49,107 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:18:52,674 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r2462e8db5cfb9f42_00000162f5046913_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r2462e8db5cfb9f42_00000162f5046913_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2462e8db5cfb9f42_00000162f5046913_1 ... (0s) Current status: DONE 2018-04-24 00:18:52,674 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:19:18,851 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:19:22,276 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r4d6c79f6698db4b0_00000162f504dcef_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r4d6c79f6698db4b0_00000162f504dcef_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4d6c79f6698db4b0_00000162f504dcef_1 ... (0s) Current status: DONE 2018-04-24 00:19:22,276 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:19:50,254 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:19:53,601 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_ra1a446f62d7b944_00000162f5055704_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_ra1a446f62d7b944_00000162f5055704_1 ... (0s) Current status: RUNNING Waiting on bqjob_ra1a446f62d7b944_00000162f5055704_1 ... (0s) Current status: DONE 2018-04-24 00:19:53,602 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:20:10,105 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:20:13,643 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7ee5e43f767728cb_00000162f505a48c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r7ee5e43f767728cb_00000162f505a48c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7ee5e43f767728cb_00000162f505a48c_1 ... (0s) Current status: DONE 2018-04-24 00:20:13,643 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:20:33,119 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:20:36,500 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7c1b35fee44eba92_00000162f505fe74_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r7c1b35fee44eba92_00000162f505fe74_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7c1b35fee44eba92_00000162f505fe74_1 ... (0s) Current status: DONE 2018-04-24 00:20:36,500 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:20:55,367 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:20:58,634 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r1a86aab6d73a6ce0_00000162f506555e_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r1a86aab6d73a6ce0_00000162f506555e_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1a86aab6d73a6ce0_00000162f506555e_1 ... (0s) Current status: DONE 2018-04-24 00:20:58,634 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:21:28,190 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:21:31,369 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6cd4cf55ec8d6489_00000162f506d596_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r6cd4cf55ec8d6489_00000162f506d596_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6cd4cf55ec8d6489_00000162f506d596_1 ... (0s) Current status: DONE 2018-04-24 00:21:31,369 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:21:49,441 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:21:52,587 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r3e1be144413c55fd_00000162f507288e_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r3e1be144413c55fd_00000162f507288e_1 ... (0s) Current status: RUNNING Waiting on bqjob_r3e1be144413c55fd_00000162f507288e_1 ... (0s) Current status: DONE 2018-04-24 00:21:52,588 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:22:19,210 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:22:22,388 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r2760f7dd0cf8244c_00000162f5079cdf_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r2760f7dd0cf8244c_00000162f5079cdf_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2760f7dd0cf8244c_00000162f5079cdf_1 ... (0s) Current status: DONE 2018-04-24 00:22:22,388 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:22:51,215 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:22:54,552 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r51f025f830665beb_00000162f50819e1_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r51f025f830665beb_00000162f50819e1_1 ... (0s) Current status: RUNNING Waiting on bqjob_r51f025f830665beb_00000162f50819e1_1 ... (0s) Current status: DONE 2018-04-24 00:22:54,553 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:23:12,083 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:23:15,421 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_re7f569f1edb96d8_00000162f5086b65_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_re7f569f1edb96d8_00000162f5086b65_1 ... (0s) Current status: RUNNING Waiting on bqjob_re7f569f1edb96d8_00000162f5086b65_1 ... (0s) Current status: DONE 2018-04-24 00:23:15,422 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:23:31,275 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:23:34,616 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r821c30cd3399c5d_00000162f508b669_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r821c30cd3399c5d_00000162f508b669_1 ... (0s) Current status: RUNNING Waiting on bqjob_r821c30cd3399c5d_00000162f508b669_1 ... (0s) Current status: DONE 2018-04-24 00:23:34,617 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:23:55,779 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:23:59,644 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r2b3c3413c5ee0c6a_00000162f5091896_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r2b3c3413c5ee0c6a_00000162f5091896_1 ... (0s) Current status: RUNNING Waiting on bqjob_r2b3c3413c5ee0c6a_00000162f5091896_1 ... (0s) Current status: DONE 2018-04-24 00:23:59,644 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:24:22,261 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:24:26,689 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r44b92973b7bfc5be_00000162f5097d88_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r44b92973b7bfc5be_00000162f5097d88_1 ... (0s) Current status: RUNNING Waiting on bqjob_r44b92973b7bfc5be_00000162f5097d88_1 ... (1s) Current status: RUNNING Waiting on bqjob_r44b92973b7bfc5be_00000162f5097d88_1 ... (1s) Current status: DONE 2018-04-24 00:24:26,690 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:24:45,124 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:24:48,684 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r12ffea5c1b990291_00000162f509d7d0_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r12ffea5c1b990291_00000162f509d7d0_1 ... (0s) Current status: RUNNING Waiting on bqjob_r12ffea5c1b990291_00000162f509d7d0_1 ... (0s) Current status: DONE 2018-04-24 00:24:48,684 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:25:06,412 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:25:10,032 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r8dc29e00a9680c3_00000162f50a2afa_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r8dc29e00a9680c3_00000162f50a2afa_1 ... (0s) Current status: RUNNING Waiting on bqjob_r8dc29e00a9680c3_00000162f50a2afa_1 ... (0s) Current status: DONE 2018-04-24 00:25:10,032 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:25:31,297 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:25:34,781 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6035179c65cd6349_00000162f50a8b30_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r6035179c65cd6349_00000162f50a8b30_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6035179c65cd6349_00000162f50a8b30_1 ... (0s) Current status: DONE 2018-04-24 00:25:34,781 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:25:50,719 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:25:54,588 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r1f0cbec8f7564956_00000162f50ad893_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r1f0cbec8f7564956_00000162f50ad893_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1f0cbec8f7564956_00000162f50ad893_1 ... (0s) Current status: DONE 2018-04-24 00:25:54,588 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:26:15,275 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:26:18,906 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r753f85abf1bae1f2_00000162f50b37ed_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r753f85abf1bae1f2_00000162f50b37ed_1 ... (0s) Current status: RUNNING Waiting on bqjob_r753f85abf1bae1f2_00000162f50b37ed_1 ... (0s) Current status: DONE 2018-04-24 00:26:18,906 50988614 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-24 00:26:37,982 50988614 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json> 2018-04-24 00:26:41,602 50988614 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/50988614/perfkit-bq-pubhyk_ye.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r14e1b3a4a5eaa9e4_00000162f50b909c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: Upload complete.Waiting on bqjob_r14e1b3a4a5eaa9e4_00000162f50b909c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r14e1b3a4a5eaa9e4_00000162f50b909c_1 ... (0s) Current status: DONE Traceback (most recent call last): File "PerfKitBenchmarker/pkb.py", line 21, in <module> sys.exit(Main()) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 977, in Main return RunBenchmarks() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 904, in RunBenchmarks collector.PublishSamples() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 927, in PublishSamples publisher.PublishSamples(self.samples) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 538, in PublishSamples vm_util.IssueRetryableCommand(load_cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 249, in WrappedFunction return f(*args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 387, in IssueRetryableCommand 'Command returned a non-zero exit code.\n') perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code. Build step 'Execute shell' marked build as failure
