See <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/1/display/redirect>
------------------------------------------ [...truncated 1000 B...] > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/4758/*:refs/remotes/origin/pr/4758/* > git rev-parse refs/remotes/origin/pr/4758/merge^{commit} # timeout=10 > git rev-parse refs/remotes/origin/origin/pr/4758/merge^{commit} # timeout=10 Checking out Revision 067916b9632840048317d2cf447dbd9bbdec0303 (refs/remotes/origin/pr/4758/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 067916b9632840048317d2cf447dbd9bbdec0303 Commit message: "Merge e5b6fd79420a4cfccf5c5dce40e0a0233c8bd7f2 into a7501289a23e2d8a3a7e5a3b9f5e920702d4d634" First time build. Skipping changelog. Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins3141918916483001521.sh + rm -rf PerfKitBenchmarker [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins8249273304052232983.sh + rm -rf .env [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins1667803911768100850.sh + virtualenv .env --system-site-packages New python executable in <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/bin/python> Installing setuptools, pip, wheel...done. [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins3291092877697212416.sh + .env/bin/pip install --upgrade setuptools pip Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins5734519250209219856.sh + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git Cloning into 'PerfKitBenchmarker'... [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins6117938483630699184.sh + .env/bin/pip install -r PerfKitBenchmarker/requirements.txt Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22)) <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning. SNIMissingWarning <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Installing collected packages: numpy Found existing installation: numpy 1.11.3 Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, outside environment <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env> Successfully installed numpy-1.13.3 [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins1879141133234415937.sh + .env/bin/pip install -e 'src/sdks/python/[gcp,test]' Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/sdks/python> Requirement already satisfied: avro<2.0.0,>=1.8.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: crcmod<2.0,>=1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: dill==0.2.6 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: grpcio<2,>=1.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0) <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning. SNIMissingWarning <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Requirement already satisfied: httplib2<0.10,>=0.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: mock<3.0.0,>=1.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: oauth2client<5,>=2.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: protobuf<4,>=3.5.0.post1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: pyyaml<4.0.0,>=3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: six<1.12,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: typing<3.7.0,>=3.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: futures<4.0.0,>=3.1.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: google-cloud-bigquery==0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: pyhamcrest<2.0,>=1.9 in /home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0) Requirement already satisfied: enum34>=1.0.4 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio<2,>=1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: requests>=2.7.0 in /home/jenkins/.local/lib/python2.7/site-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages (from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: funcsigs>=1; python_version < "3.3" in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0) Requirement already satisfied: pbr>=0.11 in /home/jenkins/.local/lib/python2.7/site-packages (from mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0) Requirement already satisfied: pyasn1>=0.1.7 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0) Requirement already satisfied: pyasn1-modules>=0.0.5 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0) Requirement already satisfied: rsa>=3.1.4 in /home/jenkins/.local/lib/python2.7/site-packages (from oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0) Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0) Requirement already satisfied: fasteners>=0.14 in /home/jenkins/.local/lib/python2.7/site-packages (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0) Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in /home/jenkins/.local/lib/python2.7/site-packages (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0) Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0) Requirement already satisfied: monotonic>=0.1 in /home/jenkins/.local/lib/python2.7/site-packages (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0) Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: google-auth-httplib2 in /home/jenkins/.local/lib/python2.7/site-packages (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in /home/jenkins/.local/lib/python2.7/site-packages (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: cachetools>=2.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: future<0.17dev,>=0.16.0 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Requirement already satisfied: ply==3.8 in /home/jenkins/.local/lib/python2.7/site-packages (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0) Installing collected packages: hdfs, apache-beam Found existing installation: apache-beam 2.4.0.dev0 Not uninstalling apache-beam at /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python, outside environment <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/.env> Running setup.py develop for apache-beam Successfully installed apache-beam hdfs-2.1.0 [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins7980878753411623657.sh + .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing --dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --bigquery_table=beam_performance.hadoopinputformatioit_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/> --official=true --kubeconfig=/home/jenkins/.kube/config --beam_it_timeout=1200 --benchmarks=beam_integration_benchmark --beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java --beam_it_module=sdks/java/io/hadoop-input-format --beam_it_class=org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIOIT '--beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--postgresPort=5432,--numberOfRecords=600000]' --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml> --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/pkb-config-local.yml> 2018-02-27 12:59:52,347 3618efe8 MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/pkb.log> 2018-02-27 12:59:52,348 3618efe8 MainThread INFO PerfKitBenchmarker version: v1.12.0-371-g2495168 2018-02-27 12:59:52,348 3618efe8 MainThread INFO Flag values: --beam_it_class=org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIOIT --beam_it_timeout=1200 --beam_it_module=sdks/java/io/hadoop-input-format --beam_sdk=java --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/> --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn --beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--postgresPort=5432,--numberOfRecords=600000] --beam_prebuilt --kubeconfig=/home/jenkins/.kube/config --project=apache-beam-testing --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml> --bigquery_table=beam_performance.hadoopinputformatioit_pkb_results --official --dpb_log_level=INFO --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/pkb-config-local.yml> --beam_it_profile=io-it --benchmarks=beam_integration_benchmark 2018-02-27 12:59:52,638 3618efe8 MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo. 2018-02-27 12:59:52,638 3618efe8 MainThread INFO Initializing the edw service decoder 2018-02-27 12:59:52,736 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark 2018-02-27 12:59:52,740 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark 2018-02-27 12:59:52,740 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: git clone https://github.com/apache/beam.git 2018-02-27 12:59:58,642 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config create -f <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml> 2018-02-27 12:59:58,944 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2018-02-27 12:59:58,947 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:09,102 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:19,243 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:29,487 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:39,611 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:49,737 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config get svc postgres-for-dev -ojsonpath={.status.loadBalancer.ingress[0].ip} 2018-02-27 13:00:49,906 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Using LoadBalancer IP Address: 35.192.230.254 2018-02-27 13:00:49,912 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: /home/jenkins/tools/maven/latest/bin/mvn -e verify -Dit.test=org.apache.beam.sdk.io.hadoop.inputformat.HadoopInputFormatIOIT -DskipITs=false -pl sdks/java/io/hadoop-input-format -Pio-it -Pdataflow-runner -DintegrationTestPipelineOptions=["--tempRoot=gs://temp-storage-for-perf-tests","--project=apache-beam-testing","--postgresPort=5432","--numberOfRecords=600000","--postgresServerName=35.192.230.254","--postgresUsername=postgres","--postgresPassword=uuinkks","--postgresDatabaseName=postgres","--postgresSsl=False","--runner=TestDataflowRunner"] 2018-02-27 13:11:37,292 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-02-27 13:11:37,292 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=/home/jenkins/.kube/config delete -f <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/src/.test-infra/kubernetes/postgres/postgres-service-for-local-dev.yml> 2018-02-27 13:11:37,680 3618efe8 MainThread beam_integration_benchmark(1/1) INFO Tearing down resources for benchmark beam_integration_benchmark 2018-02-27 13:11:37,688 3618efe8 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/perfkit-bq-pubRgpAX3.json> 2018-02-27 13:11:37,688 3618efe8 MainThread INFO Publishing 2 samples to beam_performance.hadoopinputformatioit_pkb_results 2018-02-27 13:11:37,688 3618efe8 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.hadoopinputformatioit_pkb_results <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/perfkit-bq-pubRgpAX3.json> 2018-02-27 13:11:41,388 3618efe8 MainThread INFO -------------------------PerfKitBenchmarker Complete Results------------------------- {'metadata': {'dpb_cluster_id': None, 'dpb_cluster_shape': 'n1-standard-1', 'dpb_cluster_size': 2, 'dpb_dataflow_runner': 'DataflowRunner', 'dpb_dataflow_sdk': None, 'dpb_service': 'dataflow', 'perfkitbenchmarker_version': 'v1.12.0-371-g2495168', 'run_number': 0}, 'metric': 'run_time', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '3618efe8-2c5309ec-092c-4b02-bca0-733e202caa97', 'sample_uri': '83070509-b1b7-4db3-a385-0492e0da0cc9', 'test': 'beam_integration_benchmark', 'timestamp': 1519737097.29174, 'unit': 'seconds', 'value': 647.38451} {'metadata': {'perfkitbenchmarker_version': 'v1.12.0-371-g2495168'}, 'metric': 'End to End Runtime', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '3618efe8-2c5309ec-092c-4b02-bca0-733e202caa97', 'sample_uri': 'b3df5a0b-1e69-4deb-a60c-2bbf14ac6692', 'test': 'beam_integration_benchmark', 'timestamp': 1519737097.68108, 'unit': 'seconds', 'value': 704.9445290565491} 2018-02-27 13:11:41,389 3618efe8 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/perfkitbenchmarker_results.json> 2018-02-27 13:11:41,389 3618efe8 MainThread INFO Benchmark run statuses: ------------------------------------------------------------------------------------ Name UID Status Failed Substatus ------------------------------------------------------------------------------------ beam_integration_benchmark beam_integration_benchmark0 SUCCEEDED ------------------------------------------------------------------------------------ Success rate: 100.00% (1/1) 2018-02-27 13:11:41,390 3618efe8 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/pkb.log> 2018-02-27 13:11:41,390 3618efe8 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/runs/3618efe8/completion_statuses.json> -------------------------PerfKitBenchmarker Results Summary------------------------- BEAM_INTEGRATION_BENCHMARK: dpb_cluster_shape="n1-standard-1" dpb_cluster_size="2" dpb_dataflow_runner="DataflowRunner" dpb_service="dataflow" run_number="0" run_time 647.384510 seconds End to End Runtime 704.944529 seconds ------------------------- For all tests: dpb_cluster_id="None" dpb_dataflow_sdk="None" perfkitbenchmarker_version="v1.12.0-371-g2495168" [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins4221416451891757465.sh + export 'KUBECONFIG=<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-Tue> Feb 27 12:54:48 UTC 2018' + KUBECONFIG='<https://builds.apache.org/job/beam_PerformanceTests_HadoopInputFormat/ws/config-hadoopinputformatioit-Tue> Feb 27 12:54:48 UTC 2018' [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins7971667743786581170.sh + gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug DEBUG: Running gcloud.container.clusters.get-credentials with Namespace(__calliope_internal_deepest_parser=ArgumentParser(prog='gcloud.container.clusters.get-credentials', usage=None, description='See https://cloud.google.com/container-engine/docs/kubectl for\nkubectl documentation.', version=None, formatter_class=<class 'argparse.HelpFormatter'>, conflict_handler='error', add_help=False), account=None, api_version=None, authority_selector=None, authorization_token_file=None, cmd_func=<bound method Command.Run of <googlecloudsdk.calliope.backend.Command object at 0x7fd7916ee710>>, command_path=['gcloud', 'container', 'clusters', 'get-credentials'], configuration=None, credential_file_override=None, document=None, format=None, h=None, help=None, http_timeout=None, log_http=None, name='io-datastores', project=None, quiet=None, trace_email=None, trace_log=None, trace_token=None, user_output_enabled=None, verbosity='debug', version=None, zone='us-central1-a'). WARNING: Accessing a Container Engine cluster requires the kubernetes commandline client [kubectl]. To install, run $ gcloud components install kubectl Fetching cluster endpoint and auth data. DEBUG: Saved kubeconfig to /home/jenkins/.kube/config kubeconfig entry generated for io-datastores. INFO: Display format "default". [beam_PerformanceTests_HadoopInputFormat] $ /bin/bash -xe /tmp/jenkins1965438784927016911.sh + kubectl create namespace hadoopinputformatioit-Tue Feb 27 12:54:48 UTC 2018 The Namespace "hadoopinputformatioit-Tue" is invalid: metadata.name: Invalid value: "hadoopinputformatioit-Tue": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name', or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?') Build step 'Execute shell' marked build as failure
