See 
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/56/display/redirect?page=changes>

Changes:

[chamikara] [BEAM-3060] Support for Perfkit execution of file-based-io-tests on 
HDFS

------------------------------------------
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/>
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision ad82ac140161cd5d88c972c88a1c8168cf26ff1c (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f ad82ac140161cd5d88c972c88a1c8168cf26ff1c
Commit message: "[BEAM-3060] Support for Perfkit execution of 
file-based-io-tests on HDFS cluster. (#4401)"
 > git rev-list 3c6899c001b28ff98db461df8b712c691403b67d # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins7525012335634357582.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins537005116997630131.sh
+ rm -rf .env
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins8939756250110714810.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins8189699785624850515.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins5167402494817098215.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied (use --upgrade to upgrade): absl-py in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied (use --upgrade to upgrade): jinja2>=2.7 in 
/usr/local/lib/python2.7/dist-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied (use --upgrade to upgrade): setuptools in 
./.env/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt 
(line 16))
Requirement already satisfied (use --upgrade to upgrade): 
colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages 
(from -r PerfKitBenchmarker/requirements.txt (line 17))
  Installing extra requirements: 'windows'
Requirement already satisfied (use --upgrade to upgrade): blinker>=1.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied (use --upgrade to upgrade): futures>=3.0.3 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied (use --upgrade to upgrade): PyYAML==3.12 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied (use --upgrade to upgrade): pint>=0.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied (use --upgrade to upgrade): numpy in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied (use --upgrade to upgrade): functools32 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied (use --upgrade to upgrade): contextlib2>=0.5.1 in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied (use --upgrade to upgrade): pywinrm in 
/home/jenkins/.local/lib/python2.7/site-packages (from -r 
PerfKitBenchmarker/requirements.txt (line 25))
Cleaning up...
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins8405484500491001678.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining 
file://<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python>
  Running setup.py 
(path:<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/setup.py)>
 egg_info for package from 
file://<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python>
    
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/setup.py>:66:
 UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is 
recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    no previously-included directories found matching 'doc/.build'
    
    Installed 
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/nose-1.3.7-py2.7.egg>
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution 
option: 'python_requires'
      warnings.warn(msg)
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
  Installing extra requirements: 'gcp,test'
Requirement already satisfied (use --upgrade to upgrade): avro>=1.8.1,<2.0.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): crcmod>=1.7,<2.0 in 
/usr/lib/python2.7/dist-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): dill==0.2.6 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): grpcio>=1.0,<2 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): httplib2>=0.8,<0.10 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): mock>=1.0.1,<3.0.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
oauth2client>=2.0.1,<5 in /home/jenkins/.local/lib/python2.7/site-packages 
(from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
protobuf>=3.5.0.post1,<4 in /home/jenkins/.local/lib/python2.7/site-packages 
(from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyyaml>=3.12,<4.0.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyvcf>=0.6.8,<0.7.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): six>=1.9,<1.12 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): typing>=3.6.0,<3.7.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): futures>=3.1.1,<4.0.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): hdfs3>=0.3.0,<0.4.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyhamcrest>=1.9,<2.0 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
google-apitools>=0.5.18,<=0.5.20 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
googledatastore==7.0.1 in /home/jenkins/.local/lib/python2.7/site-packages 
(from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
google-cloud-pubsub==0.26.0 in /home/jenkins/.local/lib/python2.7/site-packages 
(from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
google-cloud-bigquery==0.25.0 in 
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): enum34>=1.0.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
grpcio>=1.0,<2->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>=0.1.7 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): pyasn1-modules>=0.0.5 
in /home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): rsa>=3.1.4 in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
oauth2client>=2.0.1,<5->apache-beam==2.3.0.dev)
Requirement already satisfied (use --upgrade to upgrade): 
googleapis-common-protos>=1.5.2,<2.0dev in 
/home/jenkins/.local/lib/python2.7/site-packages (from 
proto-google-cloud-datastore-v1>=0.90.0,<=0.90.4->apache-beam==2.3.0.dev)
Installing collected packages: apache-beam
  Running setup.py develop for apache-beam
    
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/setup.py>:66:
 UserWarning: You are using version 1.5.4 of pip. However, version 7.0.0 is 
recommended.
      _PIP_VERSION, REQUIRED_PIP_VERSION
    /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution 
option: 'python_requires'
      warnings.warn(msg)
    
    warning: no files found matching 'README.md'
    warning: no files found matching 'NOTICE'
    warning: no files found matching 'LICENSE'
    Creating 
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/.env/lib/python2.7/site-packages/apache-beam.egg-link>
 (link to .)
    Adding apache-beam 2.3.0.dev to easy-install.pth file
    
    Installed 
<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python>
Successfully installed apache-beam
Cleaning up...
[beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe 
/tmp/jenkins2153495598447736521.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing 
--dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn 
--bigquery_table=beam_performance.tfrecordioit_pkb_results --official=true 
--benchmarks=beam_integration_benchmark --beam_it_timeout=1200 
--beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java 
--beam_it_module=sdks/java/io/file-based-io-tests 
--beam_it_class=org.apache.beam.sdk.io.tfrecord.TFRecordIOIT 
'--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--numberOfRecords=1000000,--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/56/]'
 '--beam_extra_mvn_properties=[filesystem=gcs]'
Traceback (most recent call last):
  File "PerfKitBenchmarker/pkb.py", line 21, in <module>
    sys.exit(Main())
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 869, in Main
    SetUpPKB()
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
 line 740, in SetUpPKB
    vm_util.GenTempDir()
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,>
 line 121, in GenTempDir
    temp_dir.CreateTemporaryDirectories()
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py";,>
 line 62, in CreateTemporaryDirectories
    os.makedirs(path)
  File 
"<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/.env/lib/python2.7/os.py";,>
 line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/tmp/perfkitbenchmarker/runs/7e997a83'
Build step 'Execute shell' marked build as failure

Reply via email to