Hi, I added a new Post Commit task "beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle" to Jenkins in https://github.com/apache/beam/pull/6322 but the jenkins job is failing with logs below. However the gradle task works fine when i execute it on my local machine using ./gradlew :beam-sdks-python:flinkCompatibilityMatrixBatch. I tried to trim down jenkins job to just have setupVirtualEnv and that also fails with the same exception. Here is an instance of failing run https://builds.apache.org/job/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/14/console#gradle-task-34 and a corresponding run for beam_PostCommit_Python_Verify which succeeded in setupVirtualEnv https://builds.apache.org/job/beam_PostCommit_Python_Verify_PR/60/console
Any pointers in resolving this will be greatly appreciated. Suspicious logs 17:41:10 Task ':beam-sdks-python:setupVirtualenv' is not up-to-date because: 17:41:10 No history is available. 17:41:10 Starting process 'command 'virtualenv''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python Command: virtualenv /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python/build/gradleenv 17:41:10 Successfully started process 'command 'virtualenv'' 17:41:10 New python executable in /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python/build/gradleenv/bin/python2 17:41:10 Also creating executable in /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python/build/gradleenv/bin/python 17:41:11 Installing setuptools, pkg_resources, pip, wheel...done. 17:41:11 Running virtualenv with interpreter /usr/bin/python2 17:41:11 Starting process 'command 'sh''. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python Command: sh -c . /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_PortableValidatesRunner_Flink_Gradle_PR/src/sdks/python/build/gradleenv/bin/activate && pip install --upgrade tox==3.0.0 grpcio-tools==1.3.5 17:41:11 Successfully started process 'command 'sh'' 17:41:11 Collecting tox==3.0.0 17:41:11 Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl 17:41:11 Collecting grpcio-tools==1.3.5 17:41:12 Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl 17:41:12 Collecting pluggy<1.0,>=0.3.0 (from tox==3.0.0) 17:41:12 Using cached https://files.pythonhosted.org/packages/f5/f1/5a93c118663896d83f7bcbfb7f657ce1d0c0d617e6b4a443a53abcc658ca/pluggy-0.7.1-py2.py3-none-any.whl 17:41:12 Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0) 17:41:12 Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /usr/lib/python2.7/dist-packages (from tox==3.0.0) (15.0.1) 17:41:12 Collecting py>=1.4.17 (from tox==3.0.0) 17:41:12 Using cached https://files.pythonhosted.org/packages/c8/47/d179b80ab1dc1bfd46a0c87e391be47e6c7ef5831a9c138c5c49d1756288/py-1.6.0-py2.py3-none-any.whl 17:41:12 Requirement not upgraded as not directly required: grpcio>=1.3.5 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (1.13.0) 17:41:12 Requirement not upgraded as not directly required: protobuf>=3.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio-tools==1.3.5) (3.6.0) 17:41:12 Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6) 17:41:12 Requirement not upgraded as not directly required: futures>=2.2.0 in /home/jenkins/.local/lib/python2.7/site-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (3.2.0) 17:41:12 Requirement not upgraded as not directly required: setuptools in /home/jenkins/.local/lib/python2.7/site-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (40.0.0) 17:41:13 gapic-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. 17:41:13 Installing collected packages: pluggy, py, tox, grpcio-tools 17:41:13 proto-google-cloud-datastore-v1 0.90.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. 17:41:13 proto-google-cloud-pubsub-v1 0.15.4 has requirement oauth2client<4.0dev,>=2.0.0, but you'll have oauth2client 4.1.2 which is incompatible. 17:41:13 googledatastore 7.0.1 has requirement httplib2<0.10,>=0.9.1, but you'll have httplib2 0.11.3 which is incompatible. 17:41:13 googledatastore 7.0.1 has requirement oauth2client<4.0.0,>=2.0.1, but you'll have oauth2client 4.1.2 which is incompatible. 17:41:13 Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pluggy-0.7.1.dist-info' 17:41:13 Consider using the `--user` option or check the permissions. Thanks, Ankur
