See 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/15/display/redirect?page=changes>

Changes:

[herohde] [BEAM-3250] Migrate Dataflow ValidatesRunner test to Gradle

[herohde] CR: use 4 forks in Dataflow test

[herohde] [BEAM-3250] Migrate Spark ValidatesRunner tests to Gradle

------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam7 (beam) in workspace 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 77207c3deec8648cde741b6984c6fbfbbe063008 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 77207c3deec8648cde741b6984c6fbfbbe063008
Commit message: "[BEAM-3250] Migrate Spark ValidatesRunner tests to Gradle"
 > git rev-list --no-walk 5056fd915e4d90a9ca0fcf716191469d96b7077b # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4442624536004254157.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a 
--verbosity=debug
DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: 
[--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"]
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
DEBUG: SDK update checks are disabled.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5086648304600127932.sh
+ cp /home/jenkins/.kube/config 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins125823683312007494.sh
+ kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 create namespace filebasedioithdfs-1523008864391
namespace "filebasedioithdfs-1523008864391" created
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins4735116346572113700.sh
++ kubectl config current-context
+ kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 config set-context gke_apache-beam-testing_us-central1-a_io-datastores 
--namespace=filebasedioithdfs-1523008864391
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7350676351820718925.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins34177832182622075.sh
+ rm -rf .env
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins444927881457005711.sh
+ virtualenv .env --system-site-packages
New python executable in 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins7196819135552081169.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6492195773344439283.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins6608199318696247050.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages 
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
  Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in 
/home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in 
/usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in 
/usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r 
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in 
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation != 
"PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from 
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in 
/usr/local/lib/python2.7/dist-packages (from 
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in 
/home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; 
platform_python_implementation != 
"PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r 
PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
  Found existing installation: numpy 1.12.1
    Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages, 
outside environment 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins1246687398989616052.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: dill==0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
  Using cached grpcio-1.10.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
  Using cached pytz-2018.3-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.5.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
  Using cached nose-1.3.7-py2-none-any.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
grpcio<2,>=1.8->apache-beam==2.5.0.dev0)
Requirement already satisfied: requests>=2.7.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages 
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pbr>=0.11 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages 
(from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0)
Requirement already satisfied: fasteners>=0.14 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: monotonic>=0.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth-httplib2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: ply==3.8 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Installing collected packages: grpcio, hdfs, pytz, nose, apache-beam
  Found existing installation: grpcio 1.4.0
    Not uninstalling grpcio at 
/home/jenkins/.local/lib/python2.7/site-packages, outside environment 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/.env>
  Found existing installation: apache-beam 2.5.0.dev0
    Not uninstalling apache-beam at 
/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_Python/src/sdks/python,
 outside environment 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/.env>
  Running setup.py develop for apache-beam
Successfully installed apache-beam grpcio-1.10.1 hdfs-2.1.0 nose-1.3.7 
pytz-2018.3
[beam_PerformanceTests_XmlIOIT_HDFS] $ /bin/bash -xe 
/tmp/jenkins5721058213910608021.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing 
--dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn 
--bigquery_table=beam_performance.xmlioit_hdfs_pkb_results 
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/>
 --official=true 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 
--beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java 
--beam_it_module=sdks/java/io/file-based-io-tests 
--beam_it_class=org.apache.beam.sdk.io.xml.XmlIOIT 
'--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=100000,--charset=UTF-8]'
 '--beam_extra_mvn_properties=[filesystem=hdfs]' 
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
 
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_XmlIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-06 12:03:58,410 72f45792 MainThread INFO     Verbose logging to: 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/runs/72f45792/pkb.log>
2018-04-06 12:03:58,415 72f45792 MainThread INFO     PerfKitBenchmarker 
version: v1.12.0-465-g7eb1e39
2018-04-06 12:03:58,416 72f45792 MainThread INFO     Flag values:
--beam_extra_mvn_properties=[filesystem=hdfs]
--beam_it_class=org.apache.beam.sdk.io.xml.XmlIOIT
--beam_it_timeout=1200
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=100000,--charset=UTF-8]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml,/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_XmlIOIT_HDFS/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
--bigquery_table=beam_performance.xmlioit_hdfs_pkb_results
--official
--beam_it_module=sdks/java/io/file-based-io-tests
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-06 12:03:58,837 72f45792 MainThread WARNING  The key "flags" was not in 
the default config, but was in user overrides. This may indicate a typo.
2018-04-06 12:03:58,837 72f45792 MainThread INFO     Initializing the edw 
service decoder
2018-04-06 12:03:58,961 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Provisioning resources for benchmark beam_integration_benchmark
2018-04-06 12:03:58,963 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Preparing benchmark beam_integration_benchmark
2018-04-06 12:03:58,964 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: git clone https://github.com/apache/beam.git
2018-04-06 12:04:05,711 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 create -f 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-06 12:04:05,986 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 create -f 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-06 12:04:06,153 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running benchmark beam_integration_benchmark
2018-04-06 12:04:06,159 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:04:16,312 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:04:26,439 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:04:36,582 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:04:46,702 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:04:56,826 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:06,946 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:17,066 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:27,188 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:37,308 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:47,482 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:05:57,606 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:07,754 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:17,874 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:28,006 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:38,223 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:48,343 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:06:58,473 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:07:08,611 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 get svc hadoop-external -ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-06 12:07:08,742 72f45792 MainThread beam_integration_benchmark(1/1) 
ERROR    Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 624, in RunBenchmark
    DoRunPhase(spec, collector, detailed_timer)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 527, in DoRunPhase
    samples = spec.BenchmarkRun(spec)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
 line 148, in Run
    dynamic_pipeline_options)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 100, in GenerateAllPipelineOptions
    EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 67, in EvaluateDynamicPipelineOptions
    argValue = RetrieveLoadBalancerIp(optionDescriptor)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 153, in RetrieveLoadBalancerIp
    raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, 
not str
2018-04-06 12:07:08,743 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Cleaning up benchmark beam_integration_benchmark
2018-04-06 12:07:08,743 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 delete -f 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster.yml>
2018-04-06 12:07:09,293 72f45792 MainThread beam_integration_benchmark(1/1) 
INFO     Running: kubectl 
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/config-filebasedioithdfs-1523008864391>
 delete -f 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/SmallITCluster/hdfs-single-datanode-cluster-for-local-dev.yml>
2018-04-06 12:07:09,494 72f45792 MainThread beam_integration_benchmark(1/1) 
ERROR    Exception running benchmark
Traceback (most recent call last):
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 758, in RunBenchmarkTask
    RunBenchmark(spec, collector)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 624, in RunBenchmark
    DoRunPhase(spec, collector, detailed_timer)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 527, in DoRunPhase
    samples = spec.BenchmarkRun(spec)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
 line 148, in Run
    dynamic_pipeline_options)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 100, in GenerateAllPipelineOptions
    EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 67, in EvaluateDynamicPipelineOptions
    argValue = RetrieveLoadBalancerIp(optionDescriptor)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
 line 153, in RetrieveLoadBalancerIp
    raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException, 
not str
2018-04-06 12:07:09,495 72f45792 MainThread beam_integration_benchmark(1/1) 
ERROR    Benchmark 1/1 beam_integration_benchmark (UID: 
beam_integration_benchmark0) failed. Execution will continue.
2018-04-06 12:07:09,505 72f45792 MainThread INFO     Benchmark run statuses:
---------------------------------------------------------------------------------
Name                        UID                          Status  Failed 
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark  beam_integration_benchmark0  FAILED                 
 
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-06 12:07:09,505 72f45792 MainThread INFO     Complete logs can be found 
at: 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/runs/72f45792/pkb.log>
2018-04-06 12:07:09,505 72f45792 MainThread INFO     Completion statuses can be 
found at: 
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT_HDFS/ws/runs/72f45792/completion_statuses.json>
Build step 'Execute shell' marked build as failure

Reply via email to